Scene building method and apparatus

Abstract

A scene building method and apparatus. The method comprises: a computing device provides a scene building page, and on the basis of a first position specified by a user in the scene building page and at least one built element comprised in a scene in the scene building page, displays N elements to be built at the first position in the scene building page; in response to the user's operation, the computing device determines at least one element from among the N elements to be built to build a three-dimensional scene. Users do not need to manually select required elements from among a large number of elements; thus, labor consumption in a scene building process is reduced, and the three-dimensional scene building efficiency can be improved.

Description

A scene construction method and device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on August 25, 2023, with application number 202311082799.9 and application name “A scene editing method, device and system”, and the Chinese patent application filed with the State Intellectual Property Office on November 30, 2023, with application number 202311654153.3 and application name “A scene building method and device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a scene construction method and device. Background Art

[0003] Image rendering refers to the use of a renderer (such as a rendering engine) to render raw model data such as a 3D model into an image at a specific angle. The rendering engine can build a rendering scene based on user operations, such as importing, moving, scaling, and deleting 3D models in the rendering scene to build the rendering scene. Usually, the 3D models in the rendering engine rely on the user to select and build them in sequence based on experience, or they can be built in large quantities by copying in repeated scenes. However, when the 3D models to be built are complex or there are many types, the efficiency of 3D model construction is low.

[0004] Summary of the Invention

[0005] The present application provides a scene construction method and device to solve the problem of low efficiency in building three-dimensional models when the three-dimensional models to be built are complex or of many types.

[0006] In the first aspect, the present application provides a scene building method. The scene building method can be applied to a computer system or to a computing device that supports the computer system to implement the scene building method. The computing device is such as a server, a terminal or a scene building device. The scene building method includes: the computing device provides a scene building page, and the scene in the scene building page includes at least one built element. The computing device obtains a first position specified by the user in the scene building page, and then displays N elements to be built in the scene building page in response to the specified first position and the information of at least one built element. The computing device deploys at least one element to be built in the first position in the scene building page in response to the user's operation on at least one of the N elements to be built in the scene building page. The information of the built element includes one or more of the second position of the built element on the scene building page, the category of the built element and the name of the built element, and N is a positive integer greater than or equal to 1.

[0007] In this application, the computing device displays N elements to be built at the first position specified by the user on the scene construction page, and uses the information of the existing built elements in the scene, such as the position, category, and name of the built elements, to recommend elements that may be built at the first position for the user to further select. In other words, the user does not need to manually select the required elements from a large number of elements, which reduces the manual consumption in the scene construction process and is conducive to improving the efficiency of three-dimensional scene construction. Furthermore, in response to the user's operation, the computing device determines at least one element from the N elements to build the three-dimensional scene, reducing the difficulty and complexity of element selection and further improving the efficiency of three-dimensional scene construction.

[0008] In one possible implementation, a computing device displays N elements to be constructed on a scene construction page in response to a specified first position and information about at least one constructed element, including: the computing device calculates a distance between a second position and the first position of the constructed element, determines from the scene construction page a portion of the constructed elements whose distance from the first position is less than or equal to a target distance, and then determines N elements to be constructed based on the information about the portion of the constructed elements, thereby displaying the N elements to be constructed on the scene construction page.

[0009] In the present application, the computing device uses the information of the elements that have been built and the distance between the second position and the first position to determine N elements to be built. The N elements to be built conform to the content representation of the built scene (including multiple elements), avoiding the user from manually selecting the required elements from a large number of elements, thereby reducing the difficulty and complexity of the user's element selection, improving the efficiency of element selection, and thus improving the efficiency of building three-dimensional scenes.

[0010] In one possible implementation, the scene construction device includes an element library, which includes multiple elements. The computing device displays N elements to be built on the scene construction page in response to the specified first position and information about at least one built element, including: the computing device determines the target information of the element to be built based on the information of the built element. The target information of the element to be built includes the target category of the element to be built and / or the target name of the element to be built. Then, a first similarity between the category of multiple elements in the element library and the target category of the element to be built is calculated, or a second similarity between the name of multiple elements in the element library and the target name of the element to be built is calculated, and the N elements to be built that are ranked higher in the first similarity or the second similarity are determined, thereby displaying the N elements to be built.

[0011] In this application, the computing device determines the information (category or name) of the N elements to be built that are ranked higher in similarity to the target information of the elements to be built by calculating the similarity, and then recommends the N elements to be built to the user. This is beneficial for the user to not have to manually select the required elements from a large number of elements, reducing the manual consumption in the scene construction process, and helping to improve the efficiency of scene construction.

[0012] In a possible implementation, the information of the constructed elements further includes: the order in which the constructed elements are added to the scene construction page.

[0013] In this application, the computing device introduces the order in which the built elements are added to the scene building page, and then when determining the N elements to be built, it uses an additional dimension of indicator (the order in which the built elements are added to the scene building page) to make the determination, which can improve the accuracy of the content representation of the N elements to be built that conform to the built elements.

[0014] In one possible implementation, before the computing device obtains the first position specified by the user in the scene building page, the scene building method further includes: the computing device receives a function activation instruction sent by the user, and the function activation instruction is used to activate the function of recommending elements to be built.

[0015] In this application, the computing device recommends elements to be built according to the location specified by the user in the scene building page according to the function activation instruction, avoiding the user from manually selecting the required elements from a large number of elements, reducing the labor consumption in the scene building process, and helping to improve the efficiency of scene building.

[0016] In one possible example, the computing device receives a function activation instruction triggered by a user's operation. For example, the operation may be a shortcut key tapped, a click operation, or a sliding operation by the user on an input device connected to the computing device.

[0017] In a possible implementation, the above elements include one or more combinations of objects, maps, materials, lights, and textures.

[0018] In one possible implementation, a computing device displays N elements to be constructed on a scene construction page in response to a specified first location and information about at least one constructed element, including: the computing device determines the N elements to be constructed using a recommendation model in response to the specified first location and information about at least one constructed element, and displays the N elements to be constructed on the scene construction page. The above-mentioned scene construction method also includes: training the recommendation model using training data, the training data including one or more of the categories of multiple training elements, the distance between two elements of the multiple training elements in the scene, and the order in which the multiple training elements are added to the scene, the training elements including at least one constructed element and / or at least one element of the N elements to be constructed.

[0019] In the present application, the computing device determines N elements to be built at the first position through a recommendation model, which can improve the efficiency of obtaining the N elements to be built, thereby improving the efficiency of building the three-dimensional scene.

[0020] In one possible implementation, the scene construction method further includes: obtaining a second location specified by a user on the scene construction page, and using a recommendation model to infer, based on the second location, at least one element to be constructed in the second location; obtaining user feedback regarding the at least one element to be constructed in the second location, the feedback including positive feedback or negative feedback; and training the recommendation model based on the positive feedback or negative feedback.

[0021] In this application, the computing device trains the recommendation model through reinforcement learning, which can further improve the accuracy of the recommendation model in determining the N elements to be built.

[0022] In one possible implementation, the computing device obtains the first position specified by the user in the scene building page, including: the computing device receives the user's operation on the scene building page, and then obtains the first position specified by the user in the scene building page in response to the operation.

[0023] In the present application, the computing device obtains the first position specified by the user in the scene construction page in response to the operation, thereby triggering the computing device to determine the N elements to be constructed at the first position, avoiding the user from manually selecting the required elements from a large number of elements, thereby reducing the difficulty and complexity of the user's element selection during the scene construction process, and is conducive to improving the efficiency of building three-dimensional scenes.

[0024] Exemplarily, the operation may be a click operation, a slide operation, voice control, a skeletal movement, a knuckle tap, etc.

[0025] In a possible implementation, the computing device displays N elements to be constructed in the scene construction page, including: the computing device displays the N elements to be constructed around a first position in the scene construction page.

[0026] Exemplarily, the N elements to be built are displayed in a circular or polygonal shape around the first position.

[0027] In the present application, the computing device displays N elements to be built around the first position in the scene building page. While realizing the visualization of the elements, displaying N elements to be built around the first position can improve the user's efficiency in selecting elements.

[0028] In a possible implementation, the computing device displays N elements to be constructed on a scene construction page, including: the computing device displays the N elements to be constructed on one side or multiple sides of the scene construction page.

[0029] In this application, the computing device displays N elements to be built on one or more sides of the scene building page. While realizing element visualization, it can avoid obstructing the elements that have been built when displaying N elements to be built, thereby improving the integrity of the scene building page.

[0030] In a possible implementation, the computing device displays N elements to be built on the scene building page, including: the computing device displays a pop-up window on the upper layer of the scene building page, and the pop-up window includes the N elements to be built.

[0031] In this application, the computing device displays N elements to be built in the form of a pop-up window, which can highlight the N elements to be built, allowing users to notice the N elements to be built in time and select them, thereby improving the efficiency of element selection and thus improving the efficiency of building three-dimensional scenes.

[0032] In the second aspect, the present application provides a method for training a recommendation model. The method for training the recommendation model can be applied to a computer system or to a computing device that supports the computer system to implement the training method for the recommendation model. The computing device is such as a server or a terminal. The method for training the recommendation model includes: obtaining training data, and using the training data to train the recommendation model. The training data includes one or more of the categories of multiple training elements, the distance between two training elements in the scene, and the order in which multiple training elements are added to the scene. The training elements include at least one built element and / or at least one element among N elements to be built. The recommendation model is used to obtain N elements to be built at the first position based on the first position specified by the user in the scene building page and the information of at least one built element in the scene building page. The information of the built element includes one or more of the second position of the built element on the scene building page, the category of the built element, and the name of the built element, and N is a positive integer greater than or equal to 1.

[0033] In the present application, by training the training model, the training model is made to obtain N elements to be built at the first position according to the first position specified by the user in the scene building page and the information of at least one built element in the scene building page. By using the information of the existing built elements in the scene, such as the position, category and name of the built elements, the elements that may be built at the first position are recommended for further selection by the user. In other words, the user does not need to manually select the required elements from a large number of elements, which reduces the labor consumption in the scene building process and is conducive to improving the efficiency of three-dimensional scene building. Furthermore, in response to the user's operation, at least one element is determined from the N elements to build the three-dimensional scene, which reduces the difficulty and complexity of element selection and further improves the efficiency of building the three-dimensional scene.

[0034] In one possible implementation, the recommendation model training method further includes: obtaining a second location specified by a user on a scene construction page, and using the recommendation model to infer, based on the second location, at least one element to be built in the second location; obtaining user feedback regarding the at least one element to be built in the second location, the feedback including positive feedback or negative feedback; and training the recommendation model based on the positive feedback or negative feedback.

[0035] In this application, the recommendation model is trained through reinforcement learning, which can further improve the accuracy of the recommendation model in determining the N elements to be built.

[0036] For more possible implementation contents of the second aspect, please refer to the description of any implementation method in the above-mentioned first aspect, which will not be repeated here.

[0037] In a third aspect, the present application provides a scene construction device. The scene construction device is applied to a computer system or a computing device that supports the computer system in implementing a scene construction method. The scene construction device includes various modules for executing the scene construction method of the first aspect or any optional implementation of the first aspect. For example, the scene construction device includes: a providing module, an acquiring module, a first response module, and a second response module. The providing module is configured to provide a scene construction page, wherein the scene in the scene construction page includes at least one constructed element. The acquiring module is configured to acquire a first position specified by a user in the scene construction page. The first response module is configured to display N elements to be constructed in the scene construction page in response to the specified first position and information about the at least one constructed element. The information about the constructed element includes one or more of the second position of the constructed element in the scene construction page, the category of the constructed element, and the name of the constructed element, where N is a positive integer greater than or equal to 1. The second response module is configured to deploy the at least one element to be constructed in the first position in the scene construction page in response to a user operation on at least one of the N elements to be constructed in the scene construction page.

[0038] For more detailed implementation details of the scene building device, please refer to the description of any implementation method in the first aspect above, as well as the content of the following specific implementation methods, which will not be repeated here.

[0039] In a fourth aspect, the present application provides a training device for a recommendation model. The training device for the recommendation model is applied to a computer system or a computing device that supports the computer system in implementing a training method for the recommendation model. The training device for the recommendation model includes various modules for executing the training method for the recommendation model in the first aspect or any optional implementation of the first aspect. For example, the training device for the recommendation model includes: an acquisition module and a training module. The acquisition module is configured to acquire training data. The training data includes one or more of the categories of multiple training elements, the distance between two of the multiple training elements in the scene, and the order in which the multiple training elements are added to the scene. The training elements include at least one already constructed element and / or at least one of N elements to be constructed. The training module is configured to train the recommendation model using the training data. The recommendation model is configured to obtain N elements to be constructed at a first position in the scene based on information about the first position and at least one already constructed element in the scene. The information about the already constructed element includes one or more of the second position of the already constructed element on the scene construction page, the category of the already constructed element, and the name of the already constructed element, where N is a positive integer greater than or equal to 1.

[0040] For more detailed implementation details of the training device for the recommendation model, please refer to the description of any implementation method in the second aspect above, as well as the content of the following specific implementation methods, which will not be repeated here.

[0041] In a fifth aspect, the present application provides a chip. The chip includes an interface circuit and a control circuit. The interface circuit is used to provide a scene building page, and the interface circuit and the control circuit cooperate to perform the method in the first aspect or any possible implementation of the first aspect; and / or the interface circuit is used to obtain training data, and the interface circuit and the control circuit cooperate to perform the method in the second aspect or any possible implementation of the second aspect.

[0042] In a sixth aspect, the present application provides a computing device cluster. The computing device cluster includes at least one computing device, each computing device including a memory and a processor. The memory of the at least one computing device is used to store computer instructions, and when the processor of the at least one computing device executes the computer instructions, it implements the method of the first aspect or any possible implementation of the first aspect; and / or, when the processor executes the computer instructions, it implements the method of the second aspect or any possible implementation of the second aspect.

[0043] In a seventh aspect, the present application provides a computer-readable storage medium. The storage medium stores a computer program or instructions, which, when executed by a processing device, implements the method of the first aspect or any possible implementation of the first aspect; and / or, when executed by a processing device, implements the method of the second aspect or any possible implementation of the second aspect.

[0044] In an eighth aspect, the present application provides a computer program product. The computer program product includes a computer program or instructions, which, when executed by a processing device, implements the method of the first aspect or any possible implementation of the first aspect; and / or, when executed by a processing device, implements the method of the second aspect or any possible implementation of the second aspect.

[0045] The beneficial effects of the second to eighth aspects above can be referred to the description of the first aspect or any possible implementation thereof, or the second aspect or any possible implementation thereof, and are not described in detail here. Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] FIG1 is a schematic diagram of the structure of a neural network provided by this application;

[0047] FIG2 is a schematic diagram of the structure of a convolutional neural network provided by this application;

[0048] FIG3 is a schematic diagram of the architecture of a computer system provided by the present application;

[0049] FIG4 is a flow chart of the training method of the recommendation model provided in this application;

[0050] FIG5 is a flow chart of a reinforcement learning method provided by this application;

[0051] FIG6 is a schematic diagram of the scene building page provided by this application;

[0052] FIG7 is a schematic diagram showing elements to be constructed provided by the present application;

[0053] FIG8 is a schematic diagram of the flow chart of the element determination method provided by this application;

[0054] FIG9 is a first structural diagram of a training device for a recommendation model provided in this application;

[0055] FIG10 is a second structural diagram of the training device for the recommendation model provided in this application;

[0056] FIG11A is a structural diagram 1 of the scene building device provided in this application;

[0057] FIG11B is a second structural diagram of the scene building device provided in this application;

[0058] FIG12 is a schematic diagram of the structure of a computing device provided by the present application;

[0059] FIG13 is a schematic diagram of the structure of a computing device cluster provided by the present application;

[0060] FIG14 is a schematic diagram of a connection between computing devices provided by this application. DETAILED DESCRIPTION

[0061] To facilitate understanding, the technical terms involved in this application are first introduced.

[0062] A rendering engine is a software or hardware component. It's primarily used to build 3D models or convert completed 3D models into visual results (such as images). The rendering engine is responsible for processing graphics data, calculating lighting effects, performing operations such as texture mapping and shading, and ultimately converting the 3D model into a pixel-level image output. Depending on the application and requirements, rendering engines can employ different technologies and algorithms, such as rasterization and ray tracing. Commonly used rendering engines include Unreal Engine, Unity, V-Ray, Corona, and Lumion.

[0063] The Rendering Engine Editor is an editor used to build elements in a 3D model during the 3D model building phase. Users can freely import, move, scale, and delete elements in the scene to construct the 3D model (i.e., the rendered scene).

[0064] Ray tracing is a method for presenting three-dimensional images on a two-dimensional screen. Ray tracing calculates and simulates the multiple refractions, reflections, and diffuse reflections of a large number of rays of light on a constructed three-dimensional model (i.e., the complete propagation path of light), thereby obtaining the lighting, color, and other values ​​of a single pixel. Multiple rays may be simulated for a single pixel.

[0065] Reinforcement learning aims to learn how to make the best decisions in a given environment to maximize cumulative rewards through the interaction between an agent (model) and its environment. In reinforcement learning, the model learns by acquiring environmental states, performing actions, and receiving rewards. In this application, continuous feedback is provided to the recommendation model to optimize its parameters, making its recommendations more accurate.

[0066] A neural network can be composed of neurons, which can be represented by xs The output of this operation unit satisfies the following formula.

[0067] Where, s = 1, 2, ... n, n is a natural number greater than 1, W s is x s The weight of the neuron, b is the bias of the neuron. f is the activation function of the neuron, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neuron into the output signal. The output signal of the activation function can be used as the input of the next layer, and the activation function can be a sigmoid function. A neural network is a network formed by connecting multiple single neurons mentioned above, that is, the output of one neuron can be the input of another neuron. The input of each neuron can be connected to the local receptive field of the previous layer to extract the features of the local receptive field. The local receptive field can be an area composed of several neurons. The weight represents the strength of the connection between different neurons. The weight determines the influence of the input on the output. A weight close to 0 means that changing the input does not change the output. A negative weight means that increasing the input reduces the output.

[0068] As shown in Figure 1, Figure 1 is a schematic diagram of the structure of a neural network provided by the present application. The neural network 100 includes N processing layers, where N is an integer greater than or equal to 3. The first layer of the neural network 100 is the input layer 110, which is responsible for receiving input signals, and the last layer of the neural network 100 is the output layer 130, which is responsible for outputting the processing results of the neural network. The other layers excluding the first and last layers are intermediate layers 140, and these intermediate layers 140 together constitute the hidden layer 120. Each intermediate layer 140 in the hidden layer 120 can both receive input signals and output signals. The hidden layer 120 is responsible for the processing of the input signal. Each layer represents a logical level of signal processing. Through multiple layers, the data signal can be processed by multiple levels of logic.

[0069] In some feasible embodiments, the input signal of the neural network can be a signal in various forms such as a video signal, a voice signal, a text signal, an image signal or a temperature signal. The video signal or image signal can be various sensor signals such as an image signal recorded or photographed by a camera (image sensor). The input signal of the neural network also includes various other computer-processable engineering signals, which are not listed here one by one. If a neural network is used to perform deep learning on the image signal, the quality of the image processed by the neural network can be improved.

[0070] A convolutional neural network (CNN) is a deep neural network with a convolutional architecture. A CNN consists of a feature extractor consisting of convolutional layers and subsampling layers. This feature extractor can be viewed as a filter, and the convolution process can be thought of as convolving an input image or feature map with a trainable filter. A convolutional layer is the layer of neurons in a CNN that performs convolution on the input signal. Within a CNN's convolutional layer, a neuron may only connect to a subset of neurons in adjacent layers. A convolutional layer can output several feature maps, which are intermediate results of the CNN's computational process. Neurons in the same feature map share weights, specifically the convolution kernel. Shared weights can be understood as extracting image information in a position-independent manner. In other words, the statistical information from one part of the image is the same as from other parts. This means that image information learned in one part can also be applied to other parts. Therefore, the same learned image information is available at all locations in the image. In the same convolutional layer, multiple convolution kernels can be used to extract different image information. Generally speaking, the more convolution kernels there are, the richer the image information reflected by the convolution operation.

[0071] Convolution kernels can be initialized as matrices of random size, and during the training process of the convolutional neural network, the convolution kernels can be learned to obtain reasonable weights. In addition, the direct benefit of shared weights is that they reduce the number of connections between the layers of the convolutional neural network, while also reducing the risk of overfitting.

[0072] For example, as shown in FIG2 , which is a schematic diagram of the structure of a convolutional neural network provided by the present application, the convolutional neural network 200 may include an input layer 210 , a convolutional layer / pooling layer 220 (wherein the pooling layer is optional), and a multi-layer perceptron 230 .

[0073] The convolution / pooling layer 220 may include, for example, layers 221 to 226. In one example, layer 221 may be a convolutional layer, layer 222 may be a pooling layer, layer 223 may be a convolutional layer, layer 224 may be a pooling layer, layer 225 may be a convolutional layer, and layer 226 may be a pooling layer. In another example, layers 221 and 222 may be convolutional layers, layer 223 may be a pooling layer, layers 224 and 225 may be convolutional layers, and layer 226 may be a pooling layer. The output of a convolutional layer may serve as the input to a subsequent pooling layer, or as the input to another convolutional layer to continue the convolution operation.

[0074] Taking convolutional layer 221 as an example, the internal working principle of a convolutional layer is introduced.

[0075] Convolution layer 221 may include multiple convolution operators, also known as kernels. In image processing, a convolution operator functions as a filter that extracts specific information from an input image matrix. A convolution operator is essentially a weight matrix, which is typically predefined. The size of this weight matrix is ​​related to the size of the image. It is important to note that the depth dimension of the weight matrix is ​​the same as the depth dimension of the input image. During the convolution operation, the weight matrix extends across the entire depth of the input image. Therefore, convolution with a single weight matrix produces a convolution output with a single depth dimension. However, in most cases, a single weight matrix is ​​not used. Instead, multiple weight matrices of the same size (rows × columns) are applied, i.e., multiple homogeneous matrices. The outputs of each weight matrix are stacked to form the depth dimension of the convolved image. Different weight matrices can be used to extract different features from an image. For example, one weight matrix can be used to extract edge information, another weight matrix can be used to extract specific colors, and yet another weight matrix can be used to blur unwanted noise in the image. The multiple weight matrices have the same size (row × column), and the feature maps extracted by the multiple weight matrices of the same size also have the same size. The multiple feature maps of the same size extracted are then merged to form the output of the convolution operation.

[0076] The weight values ​​in these weight matrices need to be obtained through a lot of training in practical applications. The weight matrices formed by the weight values ​​obtained through training can be used to extract information from the input image, so that the convolutional neural network 200 can make correct predictions.

[0077] When convolutional neural network 200 has multiple convolutional layers, the initial convolutional layers (e.g., layer 221) often extract more general features, which can also be called low-level features. As the depth of convolutional neural network 200 increases, the features extracted by later convolutional layers (e.g., layer 226) become increasingly complex, such as high-level semantic features. Features with higher semantics are more applicable to the problem being solved.

[0078] Because it's often necessary to reduce the number of training parameters, pooling layers are often periodically introduced after convolutional layers. In the example of convolutional / pooling layer 220 in Figure 2, layers 221 through 226 can be composed of a single convolutional layer followed by a single pooling layer, or multiple convolutional layers followed by one or more pooling layers. During image processing, the sole purpose of a pooling layer is to reduce the spatial size of an image. A pooling layer can include an average pooling operator and / or a max pooling operator, which are used to subsample the input image to produce a smaller image. The average pooling operator calculates the average value of pixel values ​​within a specific range as the average pooling result. The max pooling operator takes the pixel with the largest value within a specific range as the max pooling result. Furthermore, just as the size of the weight matrix used in the convolutional layer should be related to the image size, the operators in the pooling layer should also be related to the image size. The output image size after processing by the pooling layer can be smaller than the size of the image input to the pooling layer. Each pixel in the output image of the pooling layer represents the average or maximum value of the corresponding subregion of the image input to the pooling layer.

[0079] After being processed by the convolution layer / pooling layer 220, the convolution neural network 200 is still insufficient to output the required output information. Because as mentioned above, the convolution layer / pooling layer 220 extracts features and reduces the parameters brought by the input image. However, in order to generate the final output information (the required class information or other relevant information), the convolution neural network 200 needs to use a multi-layer perceptron 230 to generate one or a group of outputs of the required number of classes. Therefore, the multi-layer perceptron 230 may include multiple hidden layers (layers 231, 232 to 23n as shown in Figure 2) and an output layer 240. The parameters contained in the multiple hidden layers can be pre-trained based on relevant training data of a specific task type. For example, the task type may include image recognition, image classification, image super-resolution reconstruction, etc.

[0080] After the multiple hidden layers in the multi-layer perceptron 230, that is, the last layer of the entire convolutional neural network 200 is the output layer 240. The output layer 240 has a loss function similar to the classification cross entropy, which is specifically used to calculate the prediction error. Once the forward propagation of the entire convolutional neural network 200 (as shown in Figure 2, the propagation from the input layer 210 to the output layer 240 is forward propagation) is completed, the backpropagation (as shown in Figure 2, the propagation from the output layer 240 to the input layer 210 is backpropagation) will begin to update the weight values ​​and biases of the aforementioned layers to reduce the loss of the convolutional neural network 200 and the error between the result output by the convolutional neural network 200 through the output layer and the ideal result.

[0081] It should be noted that the convolutional neural network 200 shown in FIG2 is only an example of a convolutional neural network. In specific applications, the convolutional neural network can also exist in the form of other network models, such as U-Net, residual network (ResNet), etc.

[0082] A deep neural network (DNN), also known as a multi-layer neural network, can be understood as a neural network with many hidden layers. The "many" here does not have a specific metric. Based on the position of different layers in a DNN, the neural network inside the DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally speaking, the first layer is the input layer, the last layer is the output layer, and the layers in between are all hidden layers. The layers are fully connected, that is, any neuron in the i-th layer must be connected to any neuron in the i+1-th layer. Although DNN looks complicated, the work of each layer is actually not complicated. Simply put, it is the following linear relationship expression: Where x is the input vector, is the output vector, is the offset vector, W is the weight matrix (also called coefficient), and α() is the activation function. Each layer is just an input vector After such a simple operation, the output vector Since there are many DNN layers, the coefficient W and the offset vector The definition of these parameters in DNN is as follows: Take the coefficient W as an example: Assume that in a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as The superscript 3 represents the layer number of the coefficient W, while the subscript corresponds to the output of the third layer index 2 and the input of the second layer index 4. In summary, the coefficient from the kth neuron in the L-1th layer to the jth neuron in the Lth layer is defined as It's important to note that the input layer has no W parameter. In deep neural networks, more hidden layers allow the network to better capture complex real-world situations. Theoretically, a model with more parameters has higher complexity and greater "capacity," meaning it can handle more complex learning tasks. Training a deep neural network is essentially the process of learning the weight matrix, with the ultimate goal of obtaining the weight matrices for all layers of a trained deep neural network (a weight matrix formed by the vectors W across many layers).

[0083] Recurrent neural networks (RNNs) are used to process sequential data. In traditional neural network models, layers are fully connected, from the input layer to the hidden layer to the output layer, while nodes within each layer are disconnected. While these conventional neural networks have solved many difficult problems, they are still inadequate for many others. For example, to predict the next word in a sentence, you generally need to use the previous words, as the previous and next words in a sentence are not independent. RNNs are called recurrent neural networks because the current output of a sequence is dependent on the previous output. Specifically, the network memorizes previous information and applies it to the calculation of the current output. This means that nodes within the hidden layer are no longer disconnected but connected, and the input to a hidden layer includes not only the output of the input layer but also the output of the previous hidden layer. In theory, RNNs can process sequence data of any length. Training an RNN is similar to training a traditional CNN or DNN. This approach also uses the backpropagation algorithm, but with one difference: if the RNN is expanded, its parameters, such as W, are shared; this is not the case with traditional neural networks, as in the example above. Furthermore, when using gradient descent, the output of each step depends not only on the state of the network at the current step, but also on the state of the network at several previous steps.

[0084] Since we already have convolutional neural networks, why do we still need recurrent neural networks? The reason is simple. Convolutional neural networks assume that elements are independent of each other, and that inputs and outputs are also independent, such as cats and dogs. However, in the real world, many elements are interconnected, such as the changes in stock prices over time. Or, for example, someone says, "I love traveling, and my favorite place is Yunnan. I must visit it someday." Humans should know to fill in the blank with "Yunnan." This is because humans make inferences based on context, but how can machines do the same? That's where RNNs come in. RNNs aim to give machines the ability to remember, like humans do. Therefore, the output of an RNN depends on both the current input and historical memory.

[0085] In specific applications, recurrent neural networks can exist in the form of various network models, such as long short-term memory networks (LSTM).

[0086] When training a deep neural network, the goal is to get its output as close as possible to the desired target value. This is done by comparing the network's predictions with the desired target values, and then updating the weight vectors of each layer based on the difference between the two. (Of course, there's usually an initialization process before the first update, which pre-configures the parameters for each layer in the deep neural network.) For example, if the network's prediction is too high, the weight vectors are adjusted to predict a lower value. This adjustment is repeated until the deep neural network can predict the desired target value or a value very close to it. Therefore, it's necessary to predefine how to compare the difference between the predicted and target values. This is the loss function, or objective function, which is an important equation used to measure the difference between the predicted and target values. For example, a higher loss function output (loss) indicates a greater difference, so training a deep neural network becomes a process of minimizing this loss.

[0087] Convolutional neural networks can use the back propagation (BP) algorithm to correct the size of the parameters in the initial super-resolution model during training, reducing the reconstruction error loss of the super-resolution model. Specifically, the forward propagation of the input signal to the output generates an error loss. This error loss information is then backpropagated to update the parameters in the initial super-resolution model, thereby converging the error loss. The BP algorithm is a backpropagation movement dominated by the error loss, aiming to obtain the optimal super-resolution model parameters, such as the weight matrix.

[0088] Typically, building a 3D scene in a rendering engine editor requires users to select elements from a library and deploy them into the scene. This library contains a large number of elements, such as 5,000. These elements include object models, maps, materials, lights, or textures.

[0089] Since each element in the three-dimensional scene needs to be determined by the user from the element library, the efficiency of element selection is slow, and thus the efficiency of building the three-dimensional scene is low.

[0090] To address the aforementioned low build efficiency issue, Unreal Engine uses a copy strategy. This allows you to quickly select elements by copying previously deployed elements, which in turn speeds up 3D scene building. Alternatively, you can quickly find elements in the element library by searching for them in the Unreal Engine search box, further accelerating 3D scene building efficiency.

[0091] However, when the three-dimensional scene to be built is complex or the three-dimensional scene includes a wide variety of elements, using the above-mentioned copying or searching method to select elements will still result in slow element selection efficiency, and then using the selected elements to build the three-dimensional scene will result in low efficiency in building the three-dimensional scene.

[0092] Based on this, the present application provides a training method for a recommendation model, which includes: obtaining training data, and using the training data to train the recommendation model. The training data includes one or more of the categories of multiple training elements, the distance between two elements in the scene, and the order in which multiple training elements are added to the scene, and the training elements include at least one built element and / or at least one element among N elements to be built. Among them, the recommendation model is used to obtain N elements to be built at the first position based on the first position specified by the user in the scene building page and the information of at least one built element in the scene. The information of the built element includes one or more of the second position of the built element on the scene building page, the category of the built element, and the name of the built element, and N is a positive integer greater than or equal to 1.

[0093] In the present application, by training the training model, the training model is made to obtain N elements to be built at the first position according to the first position specified by the user in the scene building page and the information of at least one built element in the scene building page. By using the information of the existing built elements in the scene, such as the position, category and name of the built elements, the elements that may be built at the first position are recommended for further selection by the user. In other words, the user does not need to manually select the required elements from a large number of elements, which reduces the labor consumption in the scene building process and is conducive to improving the efficiency of three-dimensional scene building. Furthermore, in response to the user's operation, at least one element is determined from the N elements to build the three-dimensional scene, which reduces the difficulty and complexity of element selection and further improves the efficiency of building the three-dimensional scene.

[0094] In addition, the present application also provides a scene building method, which can be applied to a computing device. The scene building method includes: providing a scene building page, and the scene in the scene building page includes at least one built element. Obtaining a first position specified by the user in the scene building page, and then displaying N elements to be built in the scene building page in response to the specified first position and the information of at least one built element. In response to the user's operation on at least one of the N elements to be built in the scene building page, deploying at least one element in the first position in the scene building page. The information of the built element includes one or more of the second position of the built element on the scene building page, the category of the built element, and the name of the built element, and N is a positive integer greater than or equal to 1.

[0095] In this application, based on the first position specified by the user, N elements to be built at the first position are displayed on the scene construction page. The information of the existing built elements in the scene, such as the position, category and name of the built elements, is used to recommend elements that may be built at the first position for the user to further select. In other words, the user does not need to manually select the required elements from a large number of elements, which reduces the manual consumption in the scene construction process and is conducive to improving the efficiency of three-dimensional scene construction. Furthermore, in response to the user's operation, the computing device determines at least one element from the N elements to build the three-dimensional scene, reducing the difficulty and complexity of element selection and further improving the efficiency of three-dimensional scene construction.

[0096] Next, the training method and scenario building method of the recommendation model provided in this application are described in detail with reference to the accompanying drawings.

[0097] First, referring to Figure 3 , which is a schematic diagram of the architecture of a computer system provided by this application, as shown in Figure 3 , the computer system includes an execution device 310 , a training device 320 , a database 330 , a terminal device 340 , a data storage system 350 , and a data acquisition device 360 ​​.

[0098] The execution device 310 can be a terminal, such as a mobile phone terminal, a tablet computer, a laptop computer, a virtual reality (VR) device, an augmented reality (AR) device, a mixed reality (MR) device, an extended reality (ER) device, a camera or a vehicle-mounted terminal, etc. It can also be an edge device (for example, a box carrying a chip with processing capabilities), etc.

[0099] The training device 320 can be a terminal or other computing device, such as a server or cloud device.

[0100] As a possible embodiment, the execution device 310 and the training device 320 are different processors deployed on different physical devices (such as a server or a server in a cluster). For example, the execution device 310 can be a graphics processing unit (GPU), a central processing unit (CPU), other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The training device 320 can be a GPU, a neural network processing unit (NPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0101] In another possible embodiment, the execution device 310 and the training device 320 are deployed on the same physical device, or the execution device 310 and the training device 320 are the same physical device.

[0102] Data acquisition device 360 ​​is used to collect training data and store it in database 330. Data acquisition device 360, execution device 310, and training device 320 may be the same or different devices. The training data includes data in at least one of the following formats: images, speech, and text. For example, the training data includes image data, each of which contains multiple different types of elements.

[0103] The training device 320 is used to train the neural network using training data until the loss function in the neural network converges, and the neural network training is completed when the loss function value is less than a specific threshold, so that the neural network reaches a certain accuracy. Alternatively, all training data in the database 330 are used for training, and the neural network training is completed, so that the trained neural network has functions such as recommending elements to be built based on existing elements in the interface. Furthermore, the training device 320 configures the trained neural network 301 to the execution device 310. The execution device 310 is used to implement the function of processing input data according to the trained neural network 301.

[0104] In some embodiments, the execution device 310 and the training device 320 are the same computing device. The computing device can configure the trained neural network 301 to itself and use the trained neural network 301 to adjust the pixel values ​​of the pixels of the image or identify the categories of image data in various different fields.

[0105] In other embodiments, the training device 320 may deploy the trained neural network 301 to multiple execution devices 310. Each execution device 310 utilizes the trained neural network 301 to implement functions such as recommending elements to be built based on the built elements in the scene building page.

[0106] In conjunction with a computer system, the recommendation model training method provided in this embodiment can be applied to scene construction scenarios in a rendering engine. Specifically, the recommendation model training method of the embodiment of the present application can be applied to scenarios such as game scene construction, urban planning, and garden design. The following briefly introduces the application of the aforementioned recommendation model training method in game scene construction, urban planning, and garden design scenarios.

[0107] For example, the training device 320 receives training data input by the user or obtains training data from the database 330. When the training method of the recommendation model is applied to the construction of a game scene, the training data includes the layout information of each element in the game scene (including one or more of location, category, and name). The element may be a tree, stone, building, pool, etc. When the training method of the recommendation model is applied to urban planning, the training data may include the layout information of each element in the city. The element may be a telephone pole, an electric box, a building, a road, etc. When the training method of the recommendation model is applied to garden design, the training data may include the layout information of each element in a park or a community courtyard. The element may be various varieties of trees, flowers, and plants.

[0108] In other embodiments of the present application, the training method of the above-mentioned recommendation model can also be applied to planning scenarios such as community planning, home decoration design or exhibition design.

[0109] It should be noted that, in actual applications, the training data (e.g., image data) maintained in the database 330 does not necessarily all come from the data acquisition device 360, but may also be received from other devices. In addition, the training device 320 does not necessarily train the neural network entirely based on the training data maintained in the database 330, but may also obtain training data from the cloud or other places to train the neural network. The above description should not be construed as limiting the embodiments of the present application.

[0110] Furthermore, according to the functions executed by the execution device 310 , the execution device 310 can be further subdivided. As shown in FIG3 , the execution device 310 is configured with a calculation module 311 , an I / O interface 312 and a pre-processing module 313 .

[0111] The I / O interface 312 is used to exchange data with external devices. A user can input data into the I / O interface 312 through a terminal device 340. The input data may include images or videos. Alternatively, the input data may come from a database 330.

[0112] The pre-processing module 313 is used to perform pre-processing based on the input data received by the I / O interface 312. In the embodiment of the present application, the pre-processing module 313 can be used to improve the clarity of the image data received from the I / O interface 312, etc.

[0113] When the execution device 310 preprocesses the input data, or when the computing module 311 of the execution device 310 performs calculations and other related processing, the execution device 310 can call the data, code, etc. in the data storage system 350 for corresponding processing, and can also store the data and instructions obtained from the corresponding processing in the data storage system 350.

[0114] For example, the first neural network stored in execution device 310 can be applied to execution device 310. After execution device 310 acquires image data, computing module 311 inputs the image data into the first neural network to obtain a processing result. Because the first neural network is trained by training device 320 based on image data, each image data contains different types of elements. Therefore, using the first neural network to process the input image data or the location, category, or name of the elements can meet the requirement of recommending elements to the user.

[0115] Finally, the I / O interface 312 returns the processing result to the terminal device 340, thereby providing it to the user so that the user can view the processing result. It should be understood that the terminal device 340 and the execution device 310 can also be the same physical device.

[0116] In the scenario shown in FIG3 , the user can manually input data, which can be performed through the interface provided by I / O interface 312. Alternatively, terminal device 340 can automatically send input data to I / O interface 312. If user authorization is required for terminal device 340 to automatically send input data, the user can set the corresponding permissions in terminal device 340. The user can view the processing results output by execution device 310 on terminal device 340, which can be presented in the form of a display, sound, action, or other specific methods. Terminal device 340 can also serve as a data acquisition terminal, collecting input data input into I / O interface 312 and processing results output from I / O interface 312 as shown in FIG3 , as new sample data, and storing them in database 330. Of course, data collection can also be performed without going through terminal device 340, with I / O interface 312 storing the input data input into I / O interface 312 and processing results output from I / O interface 312 as new sample data in database 330.

[0117] Figure 3 is merely a schematic diagram of a system architecture provided by an embodiment of the present application. The positional relationships between the devices, components, modules, etc. shown in Figure 3 do not constitute any limitation. For example, in Figure 3, data storage system 350 is an external memory relative to execution device 310. In other cases, data storage system 350 can also be placed in execution device 310. In one embodiment provided by the present application, one or more of computing module 311, I / O interface 312, preprocessing module 313, training device 320, database 330, and data storage system 350 can be functional modules in software such as a rendering engine or a design engine. These functional modules can be deployed in a terminal device or at least one data center, and this application does not impose any restrictions on this.

[0118] On the one hand, the present application provides a method for training a recommendation model, which can be applied to the computer system shown in Figure 3. Figure 4 is a flow chart of the method for training a recommendation model provided by the present application. The method for training a recommendation model can be executed by a first processing device 400, which can be the training device 320 in Figure 3. The content of this embodiment is explained using the method shown in Figure 4 as an example of applying the method to game scene construction. As shown in Figure 4, the method for training a recommendation model may include the following steps S410 and S420.

[0119] S410: The first processing device 400 obtains training data.

[0120] The training data includes one or more of: categories of multiple training elements, distances between two elements in the multiple training elements in a scene, and an order in which the multiple training elements are added to the scene.

[0121] Illustratively, the category of the training element indicates the type to which the training element belongs, such as a chair, a table, and the like.

[0122] The distance between two training elements in the scene among multiple training elements represents the actual distance between the two training elements in the scene. For example, when the training data is a picture, the distance between the chair and the table in the picture is the distance between the two training elements in the scene. Because in the process of scene construction, the position layout of different elements has a strong correlation. For example, in the process of building a forest in a game scene, the positions of various types of trees are deployed close to each other, while houses are usually far away from the trees; or, in the process of building a house, the positions of tables and chairs are deployed close to each other, while the positions of tables, chairs and beds are far away. Therefore, using the distance between elements and the category / name of the elements (such as table, chair) as training data can enable the recommendation model to learn the relationship between the distance between elements. Furthermore, the trained recommendation model can recommend elements to be built to users based on the category / name of the elements and the distance between the elements.

[0123] The order in which multiple training elements are added to the scene represents the order in which the elements are built. For example, the order in which the elements are built in the scene could be (chair, 1), (chair, 2), (table, 3), etc., where 1, 2, 3 represent the order in which the elements are added to the scene building page.

[0124] During the scene construction process, the order in which elements are added to the scene often has a strong logical association. For example, in the process of building a house in a game scene, chairs and tables are usually added to the scene one after another; or, in the process of building a forest, various types of trees and grass are often added to the scene one after another. Therefore, on the basis of using the distance between elements and the category / name of the elements as training data, the construction order is also used as training data, so that the recommendation model can learn the logical association of the order in which the aforementioned elements are added to the scene. Furthermore, the trained recommendation model can more accurately recommend elements to be built to users based on the acquired construction order, the distance between elements, and the category / name of the elements.

[0125] The training method for the recommendation model shown in this embodiment is applied to the game scene construction process. Therefore, the training data can be extracted from image data or historical scene data. Each image in the image data indicates multiple categories of training elements, such as various combinations of different types of animals, grass, people (children or adults), trees, stones, etc., as well as the distance between each pair of training elements.

[0126] In a first possible example, the image data contains training elements of multiple categories, as well as distances between any two of the training elements, such as the distance between an animal and a person, or the distance between a person and a tree.

[0127] In a second possible example, the image data only shows a large number of training elements. The first processing device 400 performs semantic recognition or semantic segmentation on the image data to obtain the category of each training element in the image data and the distance between each training element.

[0128] In a third possible example, the historical scene data shows multiple categories of training elements, the distances between any two training elements, and the order in which the training elements are added to the scene.

[0129] Exemplarily, the training element is at least one constructed element that has been deployed in the scene.

[0130] It is worth noting that the above-mentioned image data is only a general term. In actual application scenarios, the image data can be images of completed game scenes or pictures of the real world. Similarly, historical scene data is also only a general term. In actual application scenarios, the historical scene data can be completed game scenes, such as scene data output by a rendering engine. In other embodiments provided in this application, the training data can be the final state data or intermediate data output by a model scene that has been built or is being built, and this embodiment of the application does not limit this.

[0131] Regarding the first processing device 400 acquiring training data, two possible examples are provided below.

[0132] Example 1: The first processing device 400 obtains training data from the database 330, such as the above-mentioned image data or historical scene data.

[0133] Example 2: The first processing device 400 receives training data input by a user, such as the aforementioned image data or historical scene data.

[0134] In a possible scenario, the above-mentioned training data also includes the name of the training element, or other identifiers or labels of the training element, such as the identity document (ID) of the element, etc., which is not limited in this application.

[0135] For example, when the category of the training element is chair, the name of the training element may be red table and chair, blue recliner, etc.

[0136] S420: The first processing device 400 trains the recommendation model using the training data.

[0137] The recommendation model is configured to obtain N elements to be constructed at a first position specified by a user on a scene construction page and information about at least one constructed element in a scene on the scene construction page. The information about the constructed element includes one or more of the second position of the constructed element on the scene construction page, the category of the constructed element, and the name of the constructed element, where N is a positive integer greater than or equal to 1.

[0138] In one possible scenario, the scene building page is a display interface of a rendering engine that uses the above-mentioned recommendation model.

[0139] Exemplarily, the scene building page is a display interface corresponding to the rendering engine editor in the rendering engine during the scene editing stage.

[0140] In a possible example, the first processing device 400 completes training of the recommendation model using the training data to obtain a trained recommendation model.

[0141] The following provides two possible examples of the network structure of the recommendation model.

[0142] In Example 1, the network structure of the recommendation model is a DNN network structure. For the description of DNN, please refer to the above-mentioned content of deep neural network and will not be elaborated here.

[0143] In Example 2, the recommendation model uses a transformer network. The transformer network is essentially an encoder-decoder structure. The encoder includes self-attention and a feed-forward neural network (FNN), while the decoder includes self-attention, an FNN, and encoder-decoder attention. These two attentions are used to calculate input and output weights, respectively. For example, the transformer has six layers each for the encoder and decoder.

[0144] The above content is only an example provided by this application and should not be understood as a limitation of this application. In other embodiments of this application, the network structure of the recommendation model may also be a CNN network structure, an RNN network structure, or a combination of RNN and CNN network structures.

[0145] Regarding the first processing device 400 using training data to train the recommendation model, three possible embodiments are provided below.

[0146] In a first possible embodiment, the first processing device 400 inputs the training data into the recommendation model to obtain a training result, and calculates a loss function based on the training result and the label (category or name) corresponding to the training data, and then performs backpropagation on the recommendation model based on the loss function to update the parameters of the recommendation model, thereby completing the pre-training of the recommendation model and obtaining a trained recommendation model.

[0147] In one possible example, the training process includes multiple rounds, each of which proceeds as described above and is not described in detail here. Training the recommendation model stops when the loss function converges, the loss function is less than a set threshold, or all training data is exhausted.

[0148] Furthermore, after training, the recommendation model can obtain N elements to be built at the first position based on the information of the elements already built on the scene construction page and the first position specified by the user. The content of the N elements to be built at the first position obtained by the recommendation model can be referred to the content shown in Figure 8 below and will not be repeated here.

[0149] In a second possible embodiment, the first processing device 400 performs reinforcement learning training on the recommendation model when using the recommendation model.

[0150] Regarding the first processing device 400 performing reinforcement learning training on the recommendation model during use of the recommendation model, a possible implementation is provided below, as shown in FIG5 , which is a flowchart of a reinforcement learning method provided in this application. The method may include the following steps S510 to S540 .

[0151] S510: The first processing device 400 obtains a second location specified by the user in the scene building page.

[0152] In a possible example, the current scene building page includes the category and name of the built element and the second position of the built element in the scene building page, wherein the second position specified by the user is obtained by the first processing device 400 from the front end.

[0153] The front end may be an input device built into the first processing device 400, such as a display screen, etc.; or, the front end may be an input device connected to the first processing device 400, such as a keyboard, a mouse, etc.

[0154] For the first processing device 400 to obtain more content of the second position specified by the user in the scene construction page, reference can be made to the following second processing device 600 to obtain content of the first position specified by the user in the scene construction page, which is not described in detail here.

[0155] S520: The first processing device 400 uses the recommendation model to infer, based on the second position, at least one element to be constructed at the second position.

[0156] For example, the first processing device 400 infers that the deployable to-be-built element at the second position is a telephone, a recorder, an armchair, or a recliner.

[0157] The recommendation model obtains the representation of N elements to be constructed according to the second position, which can be referred to the content shown in FIG8 below and will not be described in detail here.

[0158] S530: The first processing device 400 receives user feedback on at least one element to be constructed at the second position.

[0159] The feedback is used to indicate the user's selection result for at least one element to be built at the second position. If the selection result is that the user did not select N elements to be built, the feedback is negative feedback. If the selection result is that the user selected one or more elements from the N elements to be built, the feedback is positive feedback.

[0160] S540: The first processing device 400 trains the recommendation model according to the positive feedback or negative feedback.

[0161] In one possible implementation, the first processing device 400 sets rewards to give the recommendation model positive rewards when there is positive feedback, and gives the recommendation model negative rewards when there is negative feedback. The recommendation model is continuously trained based on the aforementioned rewards so that the rewards corresponding to the recommendation model continue to approach the positive rewards, thereby obtaining a trained recommendation model.

[0162] In the present application, the first processing device 400 trains the recommendation model through reinforcement learning, which can further improve the accuracy of the recommendation model in determining the N elements to be built.

[0163] Therefore, the training element in this embodiment includes: the at least one element to be built and the user's feedback on the at least one element to be built.

[0164] In a third possible embodiment, the first processing device 400 may pre-train the recommendation model according to the content shown in the first possible embodiment, and continuously train the recommendation model during use of the recommendation model according to the content shown in the second possible embodiment.

[0165] For the specific content of this embodiment, reference may be made to the content of the first possible embodiment and the second possible embodiment described above, and no further details will be given here.

[0166] The contents shown in the above three embodiments are only examples. In other embodiments of the present application, the first processing device 400 may also use a distributed training method to train the recommendation model.

[0167] Here are some possible examples of what the element might represent.

[0168] In Example 1, the above elements are objects, such as trees, tables and chairs, roads, and other entities that occupy part of the scene building page.

[0169] In Example 2, the above elements are textures, such as pictures of trees, tables, and chairs, which occupy very little space on the scene building page.

[0170] Example 3: The above element is light.

[0171] Example 4: The above elements are materials, such as the bark of a tree or the surface material of an iron object.

[0172] Example 5: The above element is texture, for example, the specific texture of the bark, whether it is rough or fine.

[0173] Example 6: The above elements are a combination of any two of objects, maps, materials, lights, and textures.

[0174] Example 7: The above elements are a combination of any three of objects, maps, materials, lights, and textures.

[0175] Example 8: The above elements are a combination of any four of objects, maps, materials, lights, and textures.

[0176] Example 9, the above elements are objects, maps, materials, lights and textures.

[0177] In addition, the present application also provides a scene construction method, which can be applied to the computer system shown in Figure 3. Figure 6 is a schematic diagram of the scene construction page provided by the present application, and the content of the scene construction method of the present application can be shown in Figure 6. The scene construction method can be executed by the second processing device 600, and specifically can be executed by the scene construction device in the second processing device 600. The second processing device 600 can be the execution device 310 in Figure 3. The content of this embodiment is explained by applying the method shown in Figure 6 to game scene construction as an example.

[0178] The second processing device 600 provides a scene building page (as shown in a of FIG6 , the scene building page may include a navigation bar control component 401 , an element selection control component 402 , an element editing control component 403 , an element recommendation control component 404 , a scene layout display control component 405 , etc.).

[0179] The navigation bar control component 401 is used to display various function keys of the rendering engine, such as "Insert", "Page", etc. When the user clicks various function keys, the second processing device 600 can perform corresponding processing or display a corresponding interface.

[0180] The element selection control component 402 is used to display the selectable elements in the element library and provide the user with a selection. For example, the element selection control component 402 displays the elements according to their types, that is, displays the elements of the corresponding type according to the original type determined by the user.

[0181] The element editing control component 403 is used to perform editing and scaling operations on the element selected by the user.

[0182] The element recommendation control component 404 is used to recommend N elements to be built based on the elements that have been built in the scene building page.

[0183] The scene layout display control component 405 is used to display the elements selected by the user, and the position of each element is specified by the user.

[0184] It is worth noting that the above-mentioned scene building page may also include more or fewer control components to achieve more or fewer functions, which is not limited in this application.

[0185] Since this application is applied to the game scene construction scene, the scene construction page is the display interface of the rendering engine during the scene construction process. For example, the rendering engine can be a game rendering engine.

[0186] In a possible example, the second processing device 600 displays the scene construction page on a front end, which is a display built into the second processing device 600 or a display device connected to the second processing device 600.

[0187] In a possible embodiment, the second processing device 600 receives a function activation instruction sent by a user, and then executes the step shown in b in FIG. 6 .

[0188] This function enable command is used to enable the function of recommending elements to be built.

[0189] For example, the user may trigger the function activation instruction in the scene building page. For example, the user triggers operation a on the scene building page to send the function activation instruction to the second processing device 600.

[0190] For operation a, two possible examples are provided below.

[0191] Example 1: Operation a may be a shortcut key triggered by the user on an input device, or a preset key (such as the alt key + the B key).

[0192] For example, after receiving the shortcut key or the preset key, the second processing device 600 triggers the function start instruction and executes the function shown in c in FIG. 6 .

[0193] In Example 2, the operation a may be a gesture operation performed by the user on an input device, such as a sliding operation performed by the user on the front end of the second processing device 600. The front end may be a display screen built into the second processing device 600, or a display device or mouse connected to the second processing device 600.

[0194] For example, after recognizing the gesture operation, the second processing device 600 triggers the function start instruction and executes the function shown in c in FIG. 6 .

[0195] Example 3: Operation a can recommend a control component for the user to click on an element on the scene building page.

[0196] The above examples are only optional implementations provided in this embodiment and should not be understood as limiting the present application. In other embodiments of the present application, the operation a may also be airborne operation, voice control, skeletal movement, or knuckle tapping.

[0197] The second processing device 600 obtains a first position specified by the user in the scene building page (as shown in b in FIG. 6 , the user clicks on the specified position in the scene building page).

[0198] In a possible example, the first position specified by the user is located in the scene layout display control component 405 in the scene construction page, that is, in the scene that the user has constructed.

[0199] Regarding the second processing device 600 obtaining the content of the first position specified by the user in the scene building page, please refer to the following description of the second operation, which will not be repeated here.

[0200] In response to the designated first position and information about at least one constructed element, the second processing device 600 displays N elements to be constructed on the scene construction page (as shown in c in FIG6 , the second processing device 600 displays N elements to be constructed on the left side of the scene construction page).

[0201] The information of the built element includes one or more of the second position of the built element on the scene building page, the category of the built element, and the name of the built element, and N is a positive integer greater than or equal to 1.

[0202] In a possible implementation, the second processing device 600 obtains N elements to be constructed at the first position according to the first position specified by the user and information about at least one constructed element, and then displays the N elements to be constructed on the scene construction page.

[0203] For the content of the above implementation method, please refer to the content shown in Figure 8 below, which will not be repeated here.

[0204] Regarding the second processing device 600 displaying the contents of N elements to be built in the scene building page, three possible examples are shown below. As shown in FIG7 , FIG7 is a schematic diagram of displaying elements to be built provided by the present application.

[0205] Example 1, as shown in a of FIG7 , the second processing device 600 displays N elements to be constructed around a first position in the scene construction page.

[0206] For example, N elements to be built are displayed in a circular or polygonal shape around the first position.

[0207] In the present application, the second processing device 600 displays N elements to be built around the first position in the scene building page. While realizing the visualization of the elements, displaying N elements to be built around the first position can improve the user's efficiency in selecting elements.

[0208] Example 2, as shown in FIG7 b , the second processing device 600 displays N elements to be constructed on multiple sides of the scene construction page.

[0209] In the present application, the second processing device 600 displays N elements to be built on one or more sides of the scene building page. While realizing element visualization, it can avoid obstructing the built elements when displaying the N elements to be built, thereby improving the integrity of the scene building page.

[0210] Example 3, as shown in c in FIG7 , the second processing device 600 displays N elements to be constructed within a preset range from the first position in the scene construction page.

[0211] In a possible scenario, two possible examples are provided below for the display form of N elements to be constructed.

[0212] Example 1: The second processing device 600 displays the element to be built in a proportionally scaled manner. For example, the second processing device 600 displays the element to be built in a reduced scale, or the second processing device 600 displays the element to be built in an enlarged scale.

[0213] Example 2: The second processing device 600 displays the elements to be constructed in the form of dotted lines.

[0214] The above content is only an example provided by the present application and should not be understood as limiting the present application. In other examples of the present application, the second processing device 600 displays the elements to be constructed in the form of solid lines.

[0215] In response to a first operation of the user on at least one of the N elements to be built on the scene building page, the second processing device 600 deploys the at least one element to be built at a first position on the scene building page (as shown in d in FIG6 ).

[0216] In one possible implementation, the second processing device 600 receives a first operation by a user on at least one of N elements to be built on the scene building page, and then, in response to the first operation, deploys the at least one element to be built at a first position on the scene building page.

[0217] In a possible example, the second processing device 600 obtains a first operation of a user on a control component on a scene building page through various input devices (keyboard, mouse, touch screen, etc.), which are connected to the second processing device 600.

[0218] Regarding the specific implementation of the first operation, three possible examples are provided below.

[0219] In Example 1, the first operation may be a user confirming at least one of the N elements to be built using the keyboard, such as the first operation being a confirmation (enter) key triggered by the user, or the user pre-selecting the elements to be built using the arrow keys and confirming at least one element to be built using the enter key.

[0220] Example 2: The first operation may be a user clicking and confirming at least one of the N elements to be constructed using a mouse.

[0221] Example 3: The first operation may be a user clicking or sliding on the touch screen on at least one of the N elements to be built.

[0222] The above examples are only optional implementations provided in this embodiment and should not be understood as limiting the present application. In other embodiments of the present application, the first operation may also be air operation, voice control, gesture control, skeletal movement or knuckle tapping.

[0223] With respect to the content of at least one element selected by the user in the scene building page, two possible examples are provided below.

[0224] In a possible example, if N elements to be built are displayed on the scene building page in a proportionally scaled manner, the second processing device 600 displays the at least one element to be built in the original size when deploying the at least one element to be built to the scene layout display control component 405 .

[0225] In a possible example, if N elements to be built are displayed in dotted line form on the scene building page, after the user determines the at least one element to be built, the at least one element to be built will be converted from dotted line form to solid line form.

[0226] In a possible embodiment, when the second processing device 600 displays N elements to be built on the scene building page, it displays them in a pop-up window on an upper layer of the scene building page, and the pop-up window includes the N elements to be built.

[0227] In a possible example, the pop-up window may be in a square shape. This application does not limit the form of the pop-up window. In other embodiments of this application, the pop-up window may be in a circular or polygonal shape.

[0228] In the present application, the second processing device 600 displays N elements to be built in the form of a pop-up window, which can highlight the N elements to be built, so that the user can notice the N elements to be built in time and select them, which can improve the efficiency of selecting elements and thus improve the efficiency of building the three-dimensional scene.

[0229] With respect to the second processing device 600 acquiring the content of the first position specified by the user in the scene building page, a possible implementation manner is provided below.

[0230] The second processing device 600 receives a second operation of the user on the scene building page, and then obtains a first position specified by the user on the scene building page in response to the second operation.

[0231] For the specific implementation of the second operation, three possible examples are provided below.

[0232] Example 1: The second operation may be a shortcut key triggered by the user on the input device, or a preset key (such as the alt key + the B key).

[0233] For example, after receiving the shortcut key or the preset key, the second processing device 600 determines the position of the cursor as the first position.

[0234] In Example 2, the second operation may be a gesture operation performed by the user on an input device, such as a sliding operation performed by the user on the front end of the second processing device 600. The front end may be a display screen built into the second processing device 600, or a display device or mouse connected to the second processing device 600.

[0235] For example, after recognizing a gesture operation, the second processing device 600 triggers element recommendation for the termination position of the gesture operation, that is, the termination position of the gesture operation is used as the first position.

[0236] Example 3: The second operation may be a click operation in the scene layout display control component 405 after the user clicks the element recommendation control component on the scene building page. The click operation indicates the position in the scene layout display control component 405, namely the first position.

[0237] For example, after the user clicks the element recommendation control component on the scene building page, each time the user clicks a position in the scene layout display control component 405, the position is determined to be the first position.

[0238] The above content is only an optional example provided in this embodiment and should not be understood as limiting the present application. In other examples of the present application, the second operation can also be air operation, voice control, bone movement or knuckle tapping, etc.

[0239] In this application, the second processing device 600 receives a second operation performed by the user in various ways on the scene construction page and, in response to the second operation, obtains the first position specified by the user on the scene construction page. This triggers the second processing device 600 to determine N elements to be constructed at the first position, thereby avoiding the user manually selecting the required elements from a large number of elements, thereby reducing the difficulty and complexity of the user's element selection and improving the efficiency of building a three-dimensional scene.

[0240] Regarding the second processing device 600 displaying the contents of N elements to be constructed on the scene construction page in response to the specified first position and information of at least one constructed element, three possible embodiments are provided below.

[0241] In a first possible embodiment, as shown in Figure 8, which is a flow chart of the element determination method provided by the present application, the method may include the following steps S810 to S830.

[0242] S810: The second processing device 600 calculates the distance between the second position and the first position of the constructed element, and determines from the scene construction page some constructed elements whose distance from the first position is less than or equal to the target distance.

[0243] In a possible implementation, the second processing device 600 determines some of the constructed elements within a circular range with the first position as the center and the target distance (such as the actual distance of 2 meters in the scene construction page) as the radius.

[0244] The first position and the second position are both positions in the scene layout display control component 405, that is, actual positions in the game scene.

[0245] S820: The second processing device 600 obtains N elements to be constructed based on information of some constructed elements.

[0246] In a possible scenario, the second processing device 600 obtains N elements to be constructed according to the second positions in the information of some constructed elements, the categories or names of the constructed elements, and the first positions.

[0247] Regarding the second processing device 600 obtaining the contents of N elements to be constructed based on the information of some constructed elements, three possible implementations are provided below.

[0248] In a first possible implementation, the second processing device 600 determines a target category of the element to be built based on the information of the already built element, and then calculates a first similarity between the categories of multiple elements in the element library and the target category of the element to be built, thereby determining N elements to be built that are ranked highest by the first similarity. In this implementation, the information of the already built element includes the category of the already built element and the second position of the already built element in the scene construction page.

[0249] In one possible example, the second processing device 600 calculates the distance a between the second position and the first position, and then inputs the distance a and the category of the already constructed element into the recommendation model to determine the target category of the element to be constructed. The second processing device 600 then calculates a first similarity between the categories of multiple elements in the element library and the target category of the element to be constructed, thereby determining the N elements to be constructed that are ranked highest by the first similarity.

[0250] For example, if the second position is (1, 3) and the first position is (2, 4), the distance a is

[0251] In another possible example, the second processing device 600 inputs the second position, the first position, and the category of the already constructed element into the recommendation model to determine the target category of the element to be constructed. The second processing device 600 then calculates a first similarity between the categories of multiple elements in the element library and the target category of the element to be constructed, thereby determining N elements to be constructed that are ranked highest by the first similarity.

[0252] In the present application, the second processing device 600 determines N elements to be built at the first position through the recommendation model, which can improve the efficiency of obtaining the N elements to be built, thereby improving the efficiency of building the three-dimensional scene.

[0253] Exemplarily, the above-mentioned recommendation model is a recommendation model trained with the content shown in FIG. 4 or FIG. 5 , which will not be described in detail here.

[0254] After the second processing device 600 determines that the target category of the element to be built is a chair, it calculates a first similarity between the categories of multiple elements in the element library (e.g., chair, sofa, ergonomic chair, etc.) and the chair. For example, the similarity between chairs is 1, the similarity between chairs and sofas is 0.5, and the similarity between chairs and ergonomic chairs is 0.9.

[0255] The second processing device 600 selects the categories of the M elements ranked first by the first similarity, for example, the first two, namely, chair and ergonomic chair. Furthermore, the second processing device 600 determines N elements to be constructed from the elements categorized as chair and ergonomic chair.

[0256] Regarding the above content of determining N elements to be constructed from multiple categories of elements (such as chairs or ergonomic chairs), three possible examples are provided below.

[0257] Example 1: The second processing device 600 selects the same number of elements from multiple categories of elements as elements to be built.

[0258] In Example 2, the second processing device 600 selects elements as to-be-built elements in proportion based on the first similarity corresponding to the categories. For example, if it is determined that 10 to-be-built elements are displayed on the scene building page, 7 to-be-built elements are selected from the elements in the chair category and 3 to-be-built elements are selected from the elements in the ergonomic chair category.

[0259] Example 3: The second processing device 600 determines the similarity a between the elements in the categories of chair and ergonomic chair and the target category of the element to be built, and then selects N elements ranked top in similarity a as the elements to be built.

[0260] The similarity between the element category and the element can be set in advance by the user, for example, by associating one category of elements with multiple element instances, or calculated by the second computing device 600. The similarity between the element category and the element can represent the similarity between the element category and the element name.

[0261] For example, if the element category is chair, multiple elements associated with chair may be bench, armchair, recliner, etc. If the element category is microphone, multiple elements associated with microphone may be telephone, tape recorder, airplane, etc.

[0262] The similarity between the above elements and the target category can be determined by the semantic distance between the name / ID of the element and the target category. The smaller the semantic distance, the greater the similarity, and the larger the semantic distance, the smaller the similarity. Then, the N elements ranked first in the similarity ranking are the N elements ranked last in the semantic distance ranking, and the ranking is arranged in descending order.

[0263] Exemplarily, the second processing device 600 inputs the category of the constructed element and the distance between the second position and the first position into the recommendation model trained using the method shown in FIG4 , and determines that the target category of the element to be constructed at the first position is a microphone or a chair. The second processing device 600 determines two elements with the highest similarity to the microphone, such as a telephone and a tape recorder, from among the telephones (e.g., 0.05), tape recorders (e.g., 0.02), and airplanes (e.g., 0.3) associated with the microphone. Furthermore, the second processing device 600 determines two elements with the highest similarity to the chair, such as an armchair and a deckchair, from among the benches (0.025), armchairs (e.g., 0.02), and recliners (e.g., 0.3) associated with the chair.

[0264] The content in the above brackets represents the semantic distance, such as the telephone (such as 0.05) represents the semantic distance between the microphone and the telephone.

[0265] In other examples of the present application, the second processing device 600 may directly input the category, the second position, and the first position of the constructed element into the recommendation model.

[0266] In a second possible implementation, the second processing device 600 determines the target name of the element to be built based on the information of the already built element, and then calculates a second similarity between the names of multiple elements in the element library and the target name of the element to be built, thereby determining N elements to be built that have the highest second similarity ranking. In this implementation, the information of the already built element includes the name of the already built element and the second position of the already built element in the scene construction page.

[0267] In one possible example, the second processing device 600 calculates a distance b between the second position and the first position, and then inputs the distance b and the name of the already constructed element into the recommendation model to determine a target name for the element to be constructed. The second processing device 600 then calculates a first similarity between the names of multiple elements in the element library and the target name of the element to be constructed, thereby determining N elements to be constructed that are ranked highest by the first similarity.

[0268] For example, after the second processing device 600 determines that the target name of the element to be built is a gaming chair, it calculates the second similarity between the names of multiple elements in the element library (such as a beanbag, desk chair, gaming chair, gaming recliner, etc.) and the gaming chair. For example, the similarity between a beanbag and a gaming chair is 0.2, the similarity between a desk chair and a gaming chair is 0.5, the similarity between a gaming chair and a gaming chair is 1, and the similarity between a gaming recliner and a gaming chair is 0.8. The second processing device 600 determines that the two elements to be built that rank highest in the first similarity are the gaming chair and the gaming recliner.

[0269] In other examples of the present application, the above-mentioned N may also be 3 or 7, etc. The second processing device 600 may directly input the name, second position and first position of the constructed element into the recommendation model, which is not limited in the present application.

[0270] In a third possible implementation, the first possible implementation is combined with the second possible implementation, that is, the second processing device 600 can determine the name and category of the element to be built based on the category and name of the built element, and then determine the top N elements to be built with the highest similarity.

[0271] For the details of this implementation, please refer to the first possible implementation and the second possible implementation. The above content is only a possible implementation provided by this application and should not be understood as a limitation of this application. In other implementations of this application, the second processing device 600 determines the target ID of the element to be built based on the ID and the second position of the built element, and then calculates the third similarity between the IDs of multiple elements in the element library and the target ID of the element to be built, thereby determining the N elements to be built that are ranked top by the third similarity.

[0272] Please continue to refer to FIG. 8 . The method provided in this embodiment further includes step S830 .

[0273] S830: The second processing device 600 displays N elements to be constructed on the scene construction page.

[0274] As in the above example, the second processing device 600 displays the telephone, recorder, armchair, and recliner to be built in the scene building page.

[0275] For the content of N elements to be built displayed on the scene building page by the second processing device 600, reference may be made to the content shown in FIG. 6 and FIG. 7 , which will not be described in detail here.

[0276] In the present application, the second processing device 600 uses the label (category or name) of the constructed elements and the distance between the second position and the first position to determine N elements to be constructed. The N elements to be constructed are more consistent with the content representation of the constructed scene (including multiple elements), avoiding the user from manually selecting the required elements from a large number of elements, thereby reducing the difficulty and complexity of the user's element selection, improving the efficiency of element selection, and thus improving the efficiency of building the three-dimensional scene.

[0277] In a second possible embodiment, the second processing device 600 determines N elements to be constructed based on information of at least one constructed element in the scene construction interface, and then displays the N elements to be constructed in the scene construction page.

[0278] In this embodiment, the second processing device 600 inputs the third position of at least one built element in the scene building page, the category or name of at least one built element, and the first position into the recommendation model to obtain N elements to be built at the first position, and then displays the N elements to be built in the scene building page.

[0279] For example, the constructed elements in the scene construction page include a chair (1, 1) and a table (1, 2), and the user-specified first position is (2, 2). The second processing device 600 inputs the chair (1, 1), the table (1, 2), and the first position (2, 2) into the recommendation model to obtain the elements to be constructed: a chair and a recliner.

[0280] For more details of this embodiment, please refer to the contents of the first possible embodiment mentioned above, which will not be described in detail here.

[0281] In the third possible embodiment, the contents of the first possible embodiment and the second possible embodiment are combined. The specific contents can be referred to the first possible embodiment and the second possible embodiment, which are not described in detail here.

[0282] In another possible embodiment of the present application, the information about the constructed elements further includes the order in which the constructed elements were added to the scene construction page, i.e., the construction order. Furthermore, the second processing device 600 inputs the category, construction order, and second position of the constructed elements, along with the user-specified first position, into the recommendation model to obtain N elements to be constructed at the first position. Then, in response to the user's operation on at least one of the N elements to be constructed, the at least one element to be constructed is deployed at the first position.

[0283] For the details of the above embodiment, please refer to the contents shown in Figures 6 to 8 above, and will not be described in detail here.

[0284] In the present application, the first processing device 400 introduces the order in which the built elements are added to the scene building page, and then when determining the N elements to be built, an additional dimension of indicator (the order in which the built elements are added to the scene building page) is used for determination, which can improve the accuracy of the content representation of the N elements to be built that conforms to the built elements.

[0285] It is understandable that in order to implement the functions in the above embodiments, the first processing device 400 and the second processing device 600 can be the same or different terminals, and the terminals include hardware structures and / or software modules corresponding to the execution of each function. It should be easy for those skilled in the art to realize that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, this application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0286] The training method of the recommendation model provided in accordance with the present embodiment is described in detail above in conjunction with FIG. 3 to FIG. 5 . The training device of the recommendation model provided in accordance with the present embodiment will be described below in conjunction with FIG. 9 .

[0287] Figure 9 is a first structural diagram of a training device for a recommendation model provided in this application. The training device for a recommendation model can be used to implement the functions of the training device in the above-described method embodiment, thereby also achieving the beneficial effects of the above-described method embodiment. In this embodiment, the training device for the recommendation model can be the training device 320 shown in Figure 3, or a module (e.g., a chip) applied to a server.

[0288] The training device 900 for the recommendation model includes an acquisition module 910 and a training module 920. The training device 900 for the recommendation model is used to implement the function of the first processing device 400 in the method embodiment shown in FIG.

[0289] Acquisition module 910 is configured to acquire training data. The training data includes one or more of the following: the categories of multiple training elements, the distance between two of the multiple training elements in the scene, and the order in which the multiple training elements are added to the scene. The training elements include at least one already constructed element and / or at least one of the N elements to be constructed.

[0290] Training module 920 is configured to train a recommendation model using training data. The recommendation model is configured to obtain N elements to be constructed at a first position specified by a user on a scene construction page and information about at least one constructed element on the scene construction page. The information about the constructed element includes one or more of the second position of the constructed element on the scene construction page, the category of the constructed element, and the name of the constructed element, where N is a positive integer greater than or equal to 1.

[0291] In one possible scenario, the recommendation model training device 900 may deploy the trained recommendation model to a terminal-side device, such as the second processing device 600 described above.

[0292] To further implement the functions of the method embodiments shown in Figures 3 to 5 above, the present application also provides a training device for a recommendation model, as shown in Figure 10 , which is a second structural diagram of the training device for a recommendation model provided by the present application. The training device 900 for the recommendation model also includes an enhanced training module 930 .

[0293] The enhanced training module 930 is configured to obtain a second position specified by the user on the scene construction page and, based on the second position, use the recommendation model to infer at least one element to be constructed at the second position. The module also obtains user feedback on the at least one element to be constructed at the second position, including positive or negative feedback, and trains the recommendation model based on the positive or negative feedback.

[0294] The scene building method provided in accordance with the present embodiment is described in detail above in conjunction with FIG. 3 and FIG. 6 to FIG. 8 . The scene building device provided in accordance with the present embodiment will be described below in conjunction with FIG. 11A .

[0295] Figure 11A is a schematic diagram of the structure of the scene construction device provided in this application. The scene construction device can be used to implement the functions of the execution device or the second processing device 600 in the above method embodiment, thereby also achieving the beneficial effects of the above method embodiment. In this embodiment, the scene construction device can be the execution device 310 shown in Figure 3, the second processing device 600 in Figure 6, or a module (such as a chip) applied to the server.

[0296] The scene construction device 1100 includes a providing module 1110, an acquiring module 1120, a first responding module 1130, and a second responding module 1140. The scene construction device 1100 is used to implement the functions of the computing device in the method embodiments shown in Figures 6 to 8 above.

[0297] A module 1110 is provided for providing a scene construction page, where the scene in the scene construction page includes at least one constructed element.

[0298] The acquisition module 1120 is used to acquire the first position specified by the user in the scene building page.

[0299] The first response module 1130 is used to display N elements to be built on the scene construction page in response to the specified first position and information of at least one built element, where the information of the built elements includes one or more of the second position of the built element on the scene construction page, the category of the built element, and the name of the built element, where N is a positive integer greater than or equal to 1.

[0300] The second responding module 1140 is configured to deploy the at least one element at a first position in the scene building page in response to a user's operation on at least one element among the N elements to be built in the scene building page.

[0301] In one possible scenario, the scene construction apparatus 1100 may send the obtained elements to be constructed to the terminal device, or display the elements to be constructed on a display device corresponding to the computing device.

[0302] To further implement the functions of the method embodiments shown in Figures 3 and 6 to 8 above, the present application also provides a scene construction device, as shown in Figure 11B , which is a second structural diagram of the scene construction device provided by the present application. The scene construction device 1100 also includes: a receiving module 1150 , a training module 1160 , and a reinforcement learning module 1170 .

[0303] The receiving module 1150 is used to receive a function activation instruction sent by a user, where the function activation instruction is used to activate the function of recommending elements to be built.

[0304] The training module 1160 is used to train the recommendation model using training data, where the training data includes one or more of the categories of multiple training elements, the distance between two elements in the scene, and the order in which the multiple training elements are added to the scene, and the training elements include at least one constructed element and / or at least one element among N elements to be constructed.

[0305] The reinforcement learning module 1170 is used to obtain the second position specified by the user in the scene construction page, and use the recommendation model to infer at least one element to be built in the second position based on the second position; obtain the user's feedback on the at least one element to be built in the second position, the feedback including positive feedback or negative feedback; and train the recommendation model based on the positive feedback or negative feedback.

[0306] It is worth noting that the providing module 1110, the obtaining module 1120, the first response module 1130, and the second response module 1140 can all be implemented via software or hardware. For example, the implementation of the providing module 1110 will be described below using the providing module 1110 as an example. Similarly, the implementation of the obtaining module 1120, the first response module 1130, and the second response module 1140 can refer to the implementation of the providing module 1110.

[0307] As an example of a software functional unit, the module 1110 provided may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the module 1110 provided may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0308] Similarly, the multiple hosts / virtual machines / containers running the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is located within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC. Interconnection between VPCs is achieved through the communication gateway.

[0309] As an example of a hardware functional unit, providing module 1110 may include at least one computing device, such as a server. Alternatively, providing module 1110 may be implemented using an ASIC or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), an FPGA, a generic array logic (GAL), or any combination thereof.

[0310] The multiple computing devices included in the provisioning module 1110 can be distributed in the same region or in different regions. The multiple computing devices included in the provisioning module 1110 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the provisioning module 1110 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0311] It should be noted that, in other embodiments, the providing module 1110 can be used to execute any step in the scene building method, the acquiring module 1120 can be used to execute any step in the scene building method, the first response module 1130 can be used to execute any step in the scene building method, and the second response module 1140 can be used to execute any step in the scene building method. The steps that the providing module 1110, the acquiring module 1120, the first response module 1130, and the second response module 1140 are responsible for implementing can be specified as needed. By having the providing module 1110, the acquiring module 1120, the first response module 1130, and the second response module 1140 respectively implement different steps in the scene building method, the full functions of the scene building device are realized.

[0312] For the software or hardware implementation of each module in the training device 900 of the recommendation model shown in Figure 9, please refer to the description of module 1110 provided in the above-mentioned scene building device 1100, which will not be repeated here.

[0313] The present application also provides a chip system comprising a control circuit and an interface circuit. The interface circuit is used to provide a scene building page, and the interface circuit and the control circuit work together to implement the functions of the execution device 320 or the second processing device 600 in the above method. And / or, the interface circuit is used to obtain training data, and the interface circuit and the control circuit work together to implement the functions of the training device 310 or the first processing device 400 in the above method.

[0314] In a possible design, the chip system further includes a memory for storing program instructions and / or data. The chip system can be composed of a chip or include a chip and other discrete devices.

[0315] The present application also provides a computing device. Please refer to Figure 12, which is a schematic diagram of the structure of a computing device provided by the present application. Computing device 1200 includes a bus 1202, a processor 1204, a memory 1206, and a communication interface 1208. The processor 1204, the memory 1206, and the communication interface 1208 are connected to each other via bus 1202. Computing device 1200 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in computing device 1200.

[0316] Bus 1202 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG12 shows a single bus line, but this does not imply a single bus or type of bus. Bus 1204 may include a path for transmitting information between various components of computing device 1200 (e.g., processor 1204, memory 1206, and communication interface 1208).

[0317] The processor 1204 may include any one or more processors such as a CPU, a GPU, a microprocessor (MP), or a DSP.

[0318] The memory 1206 may include volatile memory, such as random access memory (RAM). The processor 1204 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0319] Memory 1206 stores executable program code, which processor 104 executes to implement the functions of providing module 1110, acquiring module 1120, first response module 1130, and second response module 1140, thereby implementing the scene building method. In other words, memory 106 stores instructions for executing the scene building method.

[0320] Alternatively, the memory 1206 stores executable program code, and the processor 104 executes the executable program code to respectively implement the functions of the aforementioned acquisition module 910 and training module 920, thereby implementing the training method of the recommendation model. In other words, the memory 106 stores instructions for executing the training method of the recommendation model.

[0321] Communication interface 1208 uses a transceiver module, such as, but not limited to, a network interface card or a transceiver, to enable communication between computing device 1200 and other devices or communication networks. Computing device 1200 can be a computer (e.g., a server) in a cloud data center, a computer in an edge data center, or a terminal.

[0322] For the functions of the training device 320 and the functions of the execution device 310 to be deployed within the same computing device 1200 , the training device 320 can communicate with the execution device 312 via the bus 1202 .

[0323] For the functions of the training device 320 and the functions of the execution device 310 to be deployed in different computing devices 1200 , the training device 320 may communicate with the execution device 310 through a communication network.

[0324] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device, which can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0325] As shown in Figure 13, Figure 13 is a schematic diagram of the structure of a computing device cluster provided by this application. The computing device cluster includes at least one computing device 1200. The memory 1206 of one or more computing devices 1200 in the computing device cluster may store the same instructions for executing the scene building method.

[0326] In some possible implementations, the memory 1206 of one or more computing devices 1200 in the computing device cluster may also store some instructions for executing the scene building method. In other words, the combination of one or more computing devices 1200 can jointly execute the instructions for executing the scene building method.

[0327] It should be noted that the memory 1206 in different computing devices 1200 in the computing device cluster can store different instructions, each used to execute a portion of the functions of the scene building system. In other words, the instructions stored in the memory 1206 in different computing devices 1200 can implement the functions of one or more modules among the providing module 1110, the obtaining module 1120, the first response module 1130, and the second response module 1140.

[0328] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network. The network may be a wide area network or a local area network, etc. FIG14 shows a possible implementation. As shown in FIG14 , FIG14 is a schematic diagram of a connection between computing devices provided by the present application, wherein two computing devices 1200A and 1200B are connected via a network. Specifically, the connection to the network is made through a communication interface in each computing device. In this type of possible implementation, the memory 1206 in the computing device 1200A stores instructions for executing the functions of the providing module 1110 and the acquiring module 1120. At the same time, the memory 1206 in the computing device 1200B stores instructions for executing the functions of the first response module 1130 and the second response module 1140.

[0329] It should be understood that the functionality of the computing device 1200A shown in FIG14 may also be implemented by multiple computing devices 1200. Similarly, the functionality of the computing device 1200B may also be implemented by multiple computing devices 1200.

[0330] The present application also provides a computer program product comprising instructions. The computer program product may be software or a program product comprising instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the above-described scenario building method or recommendation model training method.

[0331] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute a scenario building method or a training method for a recommendation model.

[0332] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the process or function described in the embodiments of the present application is performed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instruction can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired or wireless means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, a hard disk, or a tape; it can also be an optical medium, such as a digital video disc (DVD); it can also be a semiconductor medium, such as a solid state drive (SSD).

[0333] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A scene building method, characterized in that: The method is applied to a scene building device, and the method comprises: Providing a scene building page, wherein the scene in the scene building page includes at least one built element; Obtaining the first position specified by the user in the scene building page; In response to the designated first position and the information of the at least one constructed element, N elements to be constructed are displayed in the scene construction page, the information of the constructed elements including one or more of the second position of the constructed element in the scene construction page, the category of the constructed element and the name of the constructed element, where N is a positive integer greater than or equal to 1; In response to the user's operation on at least one of the N elements to be built in the scene building page, the at least one element to be built is deployed in the first position in the scene building page.

2. The method according to claim 1, characterized in that In response to the designated first position and the information of the at least one constructed element, displaying N elements to be constructed in the scene construction page includes: Calculating the distance between the second position of the constructed element and the first position, and determining, from the scene construction page, some of the constructed elements whose distance to the first position is less than or equal to the target distance; Determining the N elements to be constructed according to the information of the part of the constructed elements; The N elements to be constructed are displayed in the scene construction page.

3. The method according to claim 1, characterized in that The scene construction device includes an element library, the element library includes a plurality of elements, and in response to the specified first position and the information of the at least one constructed element, N elements to be constructed are displayed in the scene construction page, including: Determining target information of the element to be built according to the information of the built element, wherein the target information of the element to be built includes a target category of the element to be built and / or a target name of the element to be built; Calculating a first similarity between the categories of the multiple elements in the element library and the target categories of the elements to be built, or calculating a second similarity between the names of the multiple elements in the element library and the target names of the elements to be built, and determining the N elements to be built that are ranked higher in the first similarity or the second similarity; The N elements to be constructed are displayed.

4. The method according to any one of claims 1 to 3, characterized in that The information of the constructed elements also includes the order in which the constructed elements are added to the scene construction page.

5. The method according to any one of claims 1 to 4, characterized in that: Before obtaining the first position specified by the user in the scene building page, the method further includes: A function activation instruction sent by a user is received, where the function activation instruction is used to activate a function of recommending the element to be constructed.

6. The method according to any one of claims 1 to 5, characterized in that: The built elements and the elements to be built include one or more of objects, maps, materials, lights or textures.

7. The method according to any one of claims 1 to 6, characterized in that: In response to the designated first position and the information of the at least one constructed element, displaying N elements to be constructed in the scene construction page includes: In response to the specified first position and the information of the at least one constructed element, determining N elements to be constructed using a recommendation model; Displaying N elements to be built in the scene building page; The method further comprises: The recommendation model is trained using training data, wherein the training data includes one or more of categories of multiple training elements, distances between two training elements of the multiple training elements in a scene, and an order in which the multiple training elements are added to the scene, and the training elements include the at least one built element and / or the at least one element of the N elements to be built.

8. The method according to claim 7, characterized in that The method further comprises: Obtaining a second position specified by the user in the scene building page; According to the second position, using the recommendation model to infer at least one element to be built in the second position; Obtaining user feedback on at least one element to be constructed at the second position, wherein the feedback includes positive feedback or negative feedback; The recommendation model is trained based on the positive feedback or the negative feedback.

9. A scene building device, characterized in that: The device comprises: A module is provided, for providing a scene building page, wherein the scene in the scene building page includes at least one built element; An acquisition module, used to acquire a first position specified by a user in the scene building page; A first response module, configured to display N elements to be constructed in the scene construction page in response to the specified first position and the information of the at least one constructed element, wherein the information of the constructed element includes one or more of the second position of the constructed element in the scene construction page, the category of the constructed element, and the name of the constructed element, where N is a positive integer greater than or equal to 1; The second response module is used to deploy the at least one to-be-built element in the first position in the scene building page in response to the user's operation on at least one to-be-built element among the N to-be-built elements in the scene building page.

10. The device according to claim 9, characterized in that The first response module is specifically used to calculate the distance between the second position of the built element and the first position, determine from the scene construction page the part of the built elements whose distance to the first position is less than the target distance, and determine the N elements to be built based on the information of the part of the built elements, and display the N elements to be built in the scene construction page.

11. The device according to claim 9, characterized in that The scene building device includes an element library, which includes multiple elements; the first response module is specifically used to determine the target information of the element to be built based on the information of the built elements, the target information of the element to be built includes the target category of the element to be built and / or the target name of the element to be built, and calculate the first similarity between the category of the multiple elements in the element library and the target category of the element to be built, or calculate the second similarity between the name of the multiple elements in the element library and the target name of the element to be built, determine the N elements to be built that are ranked high in the first similarity or the second similarity; and display the N elements to be built.

12. The device according to any one of claims 9 to 11, characterized in that The information of the constructed elements also includes the order in which the constructed elements are added to the scene construction page.

13. The device according to any one of claims 9 to 12, characterized in that The scene building device further includes: The receiving module is used to receive a function activation instruction sent by a user, where the function activation instruction is used to activate the function of recommending the element to be built.

14. The device according to any one of claims 9 to 13, characterized in that The built elements and the elements to be built include one or more of objects, maps, materials, lights or textures.

15. The device according to any one of claims 9 to 14, characterized in that The first response module is further specifically configured to determine N elements to be constructed using a recommendation model in response to the specified first position and the information of the at least one constructed element; Displaying N elements to be built in the scene building page; The scene building device further includes: A training module is used to train the recommendation model using training data, wherein the training data includes one or more of categories of multiple training elements, a distance between two elements of the multiple training elements in a scene, and an order in which the multiple training elements are added to the scene, and the training elements include the at least one constructed element and / or the at least one element of the N elements to be constructed.

16. The device according to claim 15, characterized in that The scene building device further includes: A reinforcement learning module is used to obtain a second position specified by a user in the scene construction page, and based on the second position, use the recommendation model to infer at least one element to be built in the second position; obtain user feedback on the at least one element to be built in the second position, wherein the feedback includes positive feedback or negative feedback; and train the recommendation model based on the positive feedback or the negative feedback.

17. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 8.

18. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 8.

19. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 8.

Images