Scene building method and device

Abstract

The invention discloses a scene building method and device, and relates to the technical field of computers. The scene building method comprises the steps that a computing device provides a scene building page, and according to a first position specified by a user in the scene building page and at least one built element included in a scene in the scene building page, N to-be-built elements at the first position are displayed in the scene building page, so that the N to-be-built elements in the first position are displayed on the basis of information of the built elements in the scene; according to the method, the element of the first position is recommended to the user from a large number of elements, so that the user does not need to manually select the required element from the large number of elements, the labor consumption in the scene building process is reduced, and the three-dimensional scene building efficiency is improved. And furthermore, the computing equipment responds to the operation of the user, and determines at least one element from the N elements to construct the three-dimensional scene, so that the difficulty and complexity of element selection are reduced, and the construction efficiency of the three-dimensional scene is further improved.

Description

[0001] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on August 25, 2023, with application number 202311082799.9 and application name “A scene editing method, device and system”, all contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a scene building method and device. Background Art

[0003] Image rendering refers to the use of a renderer (such as a rendering engine) to render original model data such as a 3D model into an image at a specific angle. The rendering engine can build a rendering scene based on user operations, such as importing, moving, scaling, and deleting 3D models in the rendering scene to build a rendering scene. Usually, the 3D models in the rendering engine rely on users to select and build them in sequence based on experience, or they can be built in large quantities by copying in repeated scenes. However, when the 3D models to be built are complex or of many types, the efficiency of building 3D models is low. Summary of the invention

[0004] The present application provides a scene construction method and device to solve the problem of low efficiency in building three-dimensional models when the three-dimensional models to be built are complex or of many types.

[0005] In the first aspect, the present application provides a scene building method. The scene building method can be applied to a computer system or to a computing device that supports the computer system to implement the scene building method. The computing device is such as a server, a terminal or a scene building device. The scene building method includes: the computing device provides a scene building page, and the scene in the scene building page includes at least one built element. The computing device obtains a first position specified by a user in the scene building page, and then displays N elements to be built in the scene building page in response to the specified first position and the information of at least one built element. The computing device responds to the user's operation on at least one of the N elements to be built in the scene building page, and deploys at least one element to be built in the first position of the scene building page. Among them, the information of the built element includes one or more of the second position of the built element on the scene building page, the category of the built element and the name of the built element, and N is a positive integer greater than or equal to 1.

[0006] In the present application, the computing device displays N elements to be built at the first position in the scene building page according to the first position specified by the user, and uses the information of the existing built elements in the scene, such as the position, category and name of the built elements, to recommend the elements that may be built at the first position for further selection by the user. In other words, the user does not need to manually select the required elements from a large number of elements, which reduces the manual consumption in the scene building process and is conducive to improving the efficiency of three-dimensional scene building. Furthermore, the computing device determines at least one element from the N elements to build a three-dimensional scene in response to the user's operation, which reduces the difficulty and complexity of element selection and further improves the efficiency of building a three-dimensional scene.

[0007] In a possible implementation, a computing device displays N elements to be built in a scene building page in response to a specified first position and information about at least one built element, including: the computing device calculates the distance between the second position and the first position of the built element, determines from the scene building page some of the built elements whose distance to the first position is less than or equal to the target distance, and then determines N elements to be built based on the information of some of the built elements, thereby displaying the N elements to be built in the scene building page.

[0008] In the present application, the computing device uses the information of the elements that have been built and the distance between the second position and the first position to determine N elements to be built. The N elements to be built conform to the content representation of the built scene (including multiple elements), thereby avoiding the user from manually selecting the required elements from a large number of elements, thereby reducing the difficulty and complexity of the user's element selection, improving the efficiency of element selection, and thus improving the efficiency of building the three-dimensional scene.

[0009] In a possible implementation, the scene construction device includes an element library, which includes multiple elements. The computing device displays N elements to be built in the scene construction page in response to the specified first position and the information of at least one built element, including: the computing device determines the target information of the element to be built based on the information of the built element. The target information of the element to be built includes the target category of the element to be built and / or the target name of the element to be built. Then, the first similarity between the category of multiple elements in the element library and the target category of the element to be built is calculated, or the second similarity between the name of multiple elements in the element library and the target name of the element to be built is calculated, and the N elements to be built that are ranked higher in the first similarity or the second similarity are determined, thereby displaying the N elements to be built.

[0010] In the present application, the computing device determines the information (category or name) of N elements to be built that are ranked top in similarity to the target information of the elements to be built by calculating the similarity, and then recommends the N elements to be built to the user. This helps the user not to manually select the required elements from a large number of elements, reduces the labor consumption in the scene building process, and helps to improve the efficiency of scene building.

[0011] In a possible implementation, the information of the constructed elements also includes: the order in which the constructed elements are added to the scene construction page.

[0012] In the present application, the computing device introduces the order in which the already built elements are added to the scene building page, and then when determining the N elements to be built, an additional dimension of indicator (the order in which the already built elements are added to the scene building page) is used for determination, thereby improving the accuracy of the content representation of the N elements to be built that conform to the already built elements.

[0013] In a possible implementation, before the computing device obtains the first position specified by the user in the scene building page, the scene building method further includes: the computing device receives a function activation instruction sent by the user, and the function activation instruction is used to activate the function of recommending the element to be built.

[0014] In the present application, the computing device enables the recommendation of elements to be built according to the location specified by the user in the scene building page according to the function activation instruction, thereby avoiding the user from manually selecting the required elements from a large number of elements, reducing the labor consumption in the scene building process, and helping to improve the efficiency of scene building.

[0015] In a possible example, the computing device receives a user's operation triggering function activation instruction. For example, the operation may be a shortcut key tapped by the user on an input device connected to the computing device, a click operation, or a sliding operation.

[0016] In a possible implementation, the above elements include one or more combinations of objects, maps, materials, lights, and textures.

[0017] In a possible implementation, a computing device displays N elements to be built in a scene building page in response to a specified first position and information about at least one built element, including: the computing device determines N elements to be built using a recommendation model in response to the specified first position and information about at least one built element, and displays the N elements to be built in the scene building page. The above-mentioned scene building method also includes: training the recommendation model using training data, the training data including one or more of the categories of multiple training elements, the distance between two elements of the multiple training elements in the scene, and the order in which the multiple training elements are added to the scene, and the training elements include at least one built element and / or at least one element of the N elements to be built.

[0018] In the present application, the computing device determines the N elements to be built at the first position through the recommendation model, which can improve the efficiency of obtaining the N elements to be built, thereby improving the efficiency of building the three-dimensional scene.

[0019] In a possible implementation, the scene construction method further includes: obtaining a second position specified by a user in the scene construction page, and using the recommendation model to infer at least one element to be constructed in the second position according to the second position. Obtaining user feedback on the at least one element to be constructed in the second position, the feedback including positive feedback or negative feedback; and training the recommendation model based on the positive feedback or negative feedback.

[0020] In the present application, the computing device trains the recommendation model through reinforcement learning, which can further improve the accuracy of the recommendation model in determining the N elements to be built.

[0021] In one possible implementation, the computing device obtains the first position specified by the user in the scene building page, including: the computing device receives an operation of the user on the scene building page, and then obtains the first position specified by the user in the scene building page in response to the operation.

[0022] In the present application, the computing device obtains the first position specified by the user in the scene construction page in response to the operation, thereby triggering the computing device to determine the N elements to be constructed at the first position, avoiding the user from manually selecting the required elements from a large number of elements, thereby reducing the difficulty and complexity of the user's element selection during the scene construction process, which is conducive to improving the efficiency of building three-dimensional scenes.

[0023] Exemplarily, the operation may be a click operation, a slide operation, voice control, a skeletal movement, a knuckle tap, etc.

[0024] In a possible implementation, the computing device displays N elements to be constructed in a scene construction page, including: the computing device displays the N elements to be constructed around a first position in the scene construction page.

[0025] Exemplarily, the N elements to be constructed are displayed in a circular or polygonal shape around the first position.

[0026] In the present application, the computing device displays N elements to be built around the first position in the scene building page. While realizing the visualization of the elements, displaying N elements to be built around the first position can improve the user's efficiency in selecting elements.

[0027] In a possible implementation, the computing device displays N elements to be built in the scene building page, including: the computing device displays the N elements to be built on one side or multiple sides of the scene building page.

[0028] In the present application, the computing device displays N elements to be built on one or more sides of the scene building page. While realizing element visualization, it can avoid obstruction of the built elements when displaying the N elements to be built, thereby improving the integrity of the scene building page.

[0029] In a possible implementation, the computing device displays N elements to be built in the scene building page, including: the computing device displays a pop-up window on the upper layer of the scene building page, and the pop-up window includes the N elements to be built.

[0030] In the present application, the computing device displays N elements to be built in the form of a pop-up window, which can highlight the N elements to be built, allowing the user to promptly notice the N elements to be built and select them, thereby improving the efficiency of selecting elements and further improving the efficiency of building three-dimensional scenes.

[0031] In the second aspect, the present application provides a training method for a recommendation model. The training method for the recommendation model can be applied to a computer system or to a computing device that supports the computer system to implement the training method for the recommendation model. The computing device is such as a server or a terminal. The training method for the recommendation model includes: obtaining training data, and using the training data to train the recommendation model. The training data includes one or more of the categories of multiple training elements, the distance between two training elements in the scene, and the order in which multiple training elements are added to the scene, and the training elements include at least one built element and / or at least one element of N elements to be built. Among them, the recommendation model is used to obtain N elements to be built at the first position according to the first position specified by the user in the scene building page and the information of at least one built element in the scene building page. The information of the built element includes one or more of the second position of the built element on the scene building page, the category of the built element, and the name of the built element, and N is a positive integer greater than or equal to 1.

[0032] In the present application, by training the training model, the training model obtains N elements to be built at the first position according to the first position specified by the user in the scene building page and the information of at least one built element in the scene building page. By using the information of the existing built elements in the scene, such as the position, category and name of the built elements, the elements that may be built at the first position are recommended for further selection by the user. In other words, the user does not need to manually select the required elements from a large number of elements, which reduces the manual consumption in the scene building process and is conducive to improving the efficiency of three-dimensional scene building. Furthermore, in response to the user's operation, at least one element is determined from the N elements to build a three-dimensional scene, which reduces the difficulty and complexity of element selection and further improves the efficiency of building a three-dimensional scene.

[0033] In a possible implementation, the training method of the recommendation model further includes: obtaining a second position specified by a user in the scene construction page, and using the recommendation model to infer at least one element to be built in the second position according to the second position. Obtaining user feedback on the at least one element to be built in the second position, the feedback including positive feedback or negative feedback; and training the recommendation model based on the positive feedback or negative feedback.

[0034] In this application, the recommendation model is trained through reinforcement learning, which can further improve the accuracy of the recommendation model in determining the N elements to be built.

[0035] For more possible implementation contents of the second aspect, reference may be made to the description of any implementation method in the above-mentioned first aspect, which will not be elaborated here.

[0036] In a third aspect, the present application provides a scene building device. The scene building device is applied to a computer system or a computing device that supports the computer system to implement a scene building method. The scene building device includes various modules for executing the scene building method in the first aspect or any optional implementation of the first aspect. For example, the scene building device includes: a providing module, an acquiring module, a first response module, and a second response module. Among them, the providing module is used to provide a scene building page, and the scene in the scene building page includes at least one built element. The acquiring module is used to acquire a first position specified by a user in the scene building page. The first response module is used to display N elements to be built in the scene building page in response to the specified first position and the information of at least one built element. The information of the built element includes one or more of the second position of the built element in the scene building page, the category of the built element, and the name of the built element, and N is a positive integer greater than or equal to 1. The second response module is used to respond to the user's operation on at least one of the N elements to be built in the scene building page, and deploy at least one element to be built in the first position of the scene building page.

[0037] For more detailed implementation content of the scene building device, please refer to the description of any implementation method in the first aspect above, as well as the content of the following specific implementation methods, which will not be repeated here.

[0038] In a fourth aspect, the present application provides a training device for a recommendation model. The training device for the recommendation model is applied to a computer system or a computing device that supports the computer system to implement a training method for the recommendation model. The training device for the recommendation model includes various modules for executing the training method for the recommendation model in the first aspect or any optional implementation of the first aspect. For example, the training device for the recommendation model includes: an acquisition module and a training module. Among them, the acquisition module is used to acquire training data. The training data includes one or more of the categories of multiple training elements, the distance between two elements in the scene, and the order in which multiple training elements are added to the scene. The training elements include at least one built element and / or at least one element of N elements to be built. The training module is used to train the recommendation model using the training data. Among them, the recommendation model is used to obtain N elements to be built at the first position according to the first position in the scene and the information of at least one built element in the scene. The information of the built element includes one or more of the second position of the built element on the scene building page, the category of the built element, and the name of the built element, and N is a positive integer greater than or equal to 1.

[0039] For more detailed implementation content of the training device for the recommendation model, please refer to the description of any implementation method in the second aspect above, as well as the content of the following specific implementation methods, which will not be repeated here.

[0040] In a fifth aspect, the present application provides a chip. The chip includes an interface circuit and a control circuit, the interface circuit is used to provide a scene building page, the interface circuit and the control circuit cooperate to execute the method in the first aspect or any possible implementation of the first aspect; and / or, the interface circuit is used to obtain training data, the interface circuit and the control circuit cooperate to execute the method in the second aspect or any possible implementation of the second aspect.

[0041] In a sixth aspect, the present application provides a computing device cluster. The computing device cluster includes at least one computing device, each computing device includes a memory and a processor, the memory of the at least one computing device is used to store computer instructions, and when the processor of the at least one computing device executes the computer instructions, the method in the first aspect or any possible implementation of the first aspect is implemented; and / or, when the processor executes the computer instructions, the method in the second aspect or any possible implementation of the second aspect is implemented.

[0042] In a seventh aspect, the present application provides a computer-readable storage medium. The storage medium stores a computer program or instruction, and when the computer program or instruction is executed by a processing device, the method in the first aspect or any possible implementation of the first aspect is implemented; and / or, when the computer program or instruction is executed by a processing device, the method in the second aspect or any possible implementation of the second aspect is implemented.

[0043] In an eighth aspect, the present application provides a computer program product. The computer program product includes a computer program or an instruction, and when the computer program or the instruction is executed by a processing device, the method in the first aspect or any possible implementation of the first aspect is implemented; and / or, when the computer program or the instruction is executed by a processing device, the method in the second aspect or any possible implementation of the second aspect is implemented.

[0044] The beneficial effects of the second to eighth aspects above can refer to the description of the first aspect or any possible implementation of the first aspect, or the second aspect or any possible implementation of the second aspect, and will not be repeated here. Based on the implementations provided in the above aspects, the present application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram of the structure of a neural network provided for this application;

[0046] Figure 2 A schematic diagram of the structure of a convolutional neural network provided for this application;

[0047] Figure 3 A schematic diagram of the architecture of a computer system provided for this application;

[0048] Figure 4 A flowchart of the training method of the recommendation model provided in this application;

[0049] Figure 5 A flowchart of a reinforcement learning method provided for this application;

[0050] Figure 6 A schematic diagram of the scene building page provided for this application;

[0051] Figure 7 A schematic diagram showing elements to be built provided for this application;

[0052] Figure 8 A schematic diagram of the process of element determination provided in this application;

[0053] Fig. 9 Schematic diagram of the structure of the training device for the recommended model provided in this application Figure 1 ;

[0054] Fig.10 Schematic diagram of the structure of the training device for the recommended model provided in this application Figure 2 ;

[0055] Fig.11A Structural diagram of the scene building device provided for this application Figure 1 ;

[0056] Fig. 11B Structural diagram of the scene building device provided for this application Figure 2 ;

[0057] Fig.12 A schematic diagram of the structure of a computing device provided for this application;

[0058] Fig.13 A schematic diagram of the structure of a computing device cluster provided for this application;

[0059] Fig.14 A schematic diagram of the connection between computing devices provided for this application. DETAILED DESCRIPTION

[0060] To facilitate understanding, the technical terms involved in this application are first introduced.

[0061] A rendering engine is a software or hardware component. The rendering engine is mainly used to build three-dimensional models, or to convert the built three-dimensional models into visual results (such as images). The rendering engine is responsible for processing graphic data, calculating lighting effects, performing operations such as texture mapping and shadow processing, and finally converting the three-dimensional model into pixel-level image output. Depending on different application scenarios and requirements, the rendering engine can use different technologies and algorithms, such as rasterization rendering, ray tracing rendering, etc. Commonly used rendering engines include Unreal Engine, Unity, Vray, Corona, Lumion, etc.

[0062] The rendering engine editor is an editor used to build elements in a 3D model during the 3D model building phase. Users can freely import, move, scale, and delete elements in the scene to build a 3D model (i.e., a rendering scene) using the rendering engine editor.

[0063] Ray tracing is a method of presenting 3D images on a 2D screen. Ray tracing calculates and simulates the multiple refractions, reflections, diffuse reflections, etc. of a large number of light rays on the constructed 3D model (i.e. the complete propagation path of the light rays), thereby obtaining the illumination, color, and other values ​​of a pixel. For a pixel, multiple rays may be simulated.

[0064] Reinforcement learning aims to learn how to make the best decision in a given environment to obtain the maximum cumulative reward through the interaction between the agent (model) and the environment. In reinforcement learning, the model learns by obtaining the state of the environment, performing actions, and receiving rewards. In this application, the recommendation model is continuously fed back to optimize the parameters of the recommendation model, making the recommendation results of the recommendation model more accurate.

[0065] A neural network can be composed of neurons, where neurons can refer to s The output of this operation unit satisfies the following formula.

[0066]

[0067] Where, s = 1, 2, ... n, n is a natural number greater than 1, W s For x s , b is the bias of the neuron. f is the activation function of the neuron, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neuron into the output signal. The output signal of the activation function can be used as the input of the next layer, and the activation function can be a sigmoid function. A neural network is a network formed by connecting multiple single neurons mentioned above, that is, the output of one neuron can be the input of another neuron. The input of each neuron can be connected to the local receptive field of the previous layer to extract the features of the local receptive field. The local receptive field can be an area composed of several neurons. The weight characterizes the strength of the connection between different neurons. The weight determines the influence of the input on the output. A weight close to 0 means that changing the input does not change the output. A negative weight means that increasing the input reduces the output.

[0068] like Figure 1 As shown, Figure 1 A schematic diagram of the structure of a neural network provided for the present application. The neural network 100 includes N processing layers, where N is an integer greater than or equal to 3. The first layer of the neural network 100 is the input layer 110, which is responsible for receiving input signals, and the last layer of the neural network 100 is the output layer 130, which is responsible for outputting the processing results of the neural network. The other layers except the first and last layers are intermediate layers 140, and these intermediate layers 140 together constitute the hidden layer 120. Each intermediate layer 140 in the hidden layer 120 can both receive input signals and output signals. The hidden layer 120 is responsible for the processing of input signals. Each layer represents a logical level of signal processing. Through multiple layers, the data signal can be processed by multiple levels of logic.

[0069] In some feasible embodiments, the input signal of the neural network can be a signal in various forms such as a video signal, a voice signal, a text signal, an image signal or a temperature signal. The video signal or the image signal can be various sensor signals such as an image signal recorded or photographed by a camera (image sensor). The input signal of the neural network also includes various other computer-processable engineering signals, which are not listed here one by one. If the neural network is used to perform deep learning on the image signal, the quality of the image processed by the neural network can be improved.

[0070] Convolutional neural network (CNN) is a deep neural network with a convolutional structure. Convolutional neural network contains a feature extractor consisting of a convolution layer and a subsampling layer. The feature extractor can be regarded as a filter, and the convolution process can be regarded as using a trainable filter to convolve with an input image or feature map. The convolution layer refers to the neuron layer in the convolutional neural network that performs convolution processing on the input signal. In the convolution layer of the convolutional neural network, a neuron can only be connected to some neurons in the adjacent layer. A convolutional layer can output several feature maps, which can refer to the intermediate results of the convolutional neural network operation process. Neurons in the same feature map share weights, and the shared weights here are the convolution kernels. Shared weights can be understood as the way to extract image information is independent of position. That is, the statistical information of a part of the image is the same as that of other parts. This means that the image information learned in one part can also be used in another part. Therefore, the same learned image information can be used for all positions on the image. In the same convolution layer, multiple convolution kernels can be used to extract different image information. Generally speaking, the more convolution kernels there are, the richer the image information reflected by the convolution operation.

[0071] The convolution kernel can be initialized in the form of a matrix of random size, and the convolution kernel can obtain reasonable weights through learning during the training process of the convolutional neural network. In addition, the direct benefit of shared weights is to reduce the connections between the layers of the convolutional neural network, while reducing the risk of overfitting.

[0072] For example, Figure 2 As shown, Figure 2 A schematic diagram of the structure of a convolutional neural network provided in the present application. The convolutional neural network 200 may include an input layer 210, a convolutional layer / pooling layer 220 (wherein the pooling layer is optional) and a multi-layer perceptron 230.

[0073] The convolution layer / pooling layer 220 may include, for example, layers 221 to 226. In one example, layer 221 may be, for example, a convolution layer, layer 222 may be, for example, a pooling layer, layer 223 may be, for example, a convolution layer, layer 224 may be, for example, a pooling layer, layer 225 may be, for example, a convolution layer, and layer 226 may be, for example, a pooling layer. In another example, layers 221 and 222 may be, for example, convolution layers, layer 223 may be, for example, a pooling layer, layers 224 and 225 may be, for example, convolution layers, and layer 226 may be, for example, a pooling layer. The output of a convolution layer may be used as the input of a subsequent pooling layer, or as the input of another convolution layer to continue the convolution operation.

[0074] The convolution layer 221 will be taken as an example to introduce the internal working principle of a convolution layer.

[0075] The convolution layer 221 may include a plurality of convolution operators, which may also be referred to as kernels. The role of the convolution operator in image processing is equivalent to a filter that extracts specific information from an input image matrix. The convolution operator may essentially be a weight matrix, which is usually predefined. The size of the weight matrix is ​​related to the size of the image. It should be noted that the depth dimension of the weight matrix is ​​the same as the depth dimension of the input image. During the convolution operation, the weight matrix extends to the entire depth of the input image. Therefore, convolution with a single weight matrix produces a convolution output of a single depth dimension, but in most cases a single weight matrix is ​​not used, but multiple weight matrices of the same size (row × column), i.e., multiple isotype matrices, are applied. The output of each weight matrix is ​​stacked to form the depth dimension of the convolution image. Different weight matrices can be used to extract different features in an image, for example, one weight matrix is ​​used to extract image edge information, another weight matrix is ​​used to extract specific colors of an image, and another weight matrix is ​​used to blur unwanted noise in an image. The multiple weight matrices have the same size (rows × columns), and the sizes of the feature maps extracted by the multiple weight matrices with the same size are also the same. The multiple feature maps with the same size extracted are then merged to form the output of the convolution operation.

[0076] The weight values ​​in these weight matrices need to be obtained through a lot of training in practical applications. The weight matrices formed by the weight values ​​obtained through training can be used to extract information from the input image, so that the convolutional neural network 200 can make correct predictions.

[0077] When the convolutional neural network 200 has multiple convolutional layers, the initial convolutional layer (e.g., layer 221) often extracts more general features, which can also be called low-level features. As the depth of the convolutional neural network 200 increases, the features extracted by the later convolutional layers (e.g., layer 226) become more and more complex, such as high-level semantic features. Features with higher semantics are more suitable for the problem to be solved.

[0078] Since it is often necessary to reduce the number of training parameters, it is often necessary to periodically introduce a pooling layer after the convolution layer. Figure 2 Each layer 221 to 226 of the example of the convolution layer / pooling layer 220 may be a convolution layer followed by a pooling layer, or may be multiple convolution layers followed by one or more pooling layers. In the process of image processing, the only purpose of the pooling layer is to reduce the spatial size of the image. The pooling layer may include an average pooling operator and / or a maximum pooling operator for sampling the input image to obtain an image of smaller size. The average pooling operator may calculate the pixel values ​​in the image within a specific range to generate an average value as the result of average pooling. The maximum pooling operator may take the pixel with the largest value within the range as the result of maximum pooling within a specific range. In addition, just as the size of the weight matrix used in the convolution layer should be related to the image size, the operator in the pooling layer should also be related to the image size. The size of the image output after processing by the pooling layer may be smaller than the size of the image input to the pooling layer, and each pixel in the image output by the pooling layer represents the average value or maximum value of the corresponding sub-region of the image input to the pooling layer.

[0079] After being processed by the convolution layer / pooling layer 220, the convolution neural network 200 is not sufficient to output the required output information. As mentioned above, the convolution layer / pooling layer 220 extracts features and reduces the parameters brought by the input image. However, in order to generate the final output information (the required class information or other related information), the convolution neural network 200 needs to use the multi-layer perceptron 230 to generate one or a group of outputs of the required number of classes. Therefore, the multi-layer perceptron 230 may include multiple hidden layers (such as Figure 2 As shown in layers 231, 232 to 23n) and output layer 240, the parameters contained in the multiple hidden layers can be pre-trained according to relevant training data of a specific task type. For example, the task type may include image recognition, image classification, image super-resolution reconstruction, etc.

[0080] After the multiple hidden layers in the multi-layer perceptron 230, that is, the last layer of the entire convolutional neural network 200 is the output layer 240, which has a loss function similar to the classification cross entropy, specifically used to calculate the prediction error. Once the forward propagation of the entire convolutional neural network 200 (such as Figure 2The propagation from the input layer 210 to the output layer 240 is forward propagation), and the reverse propagation (such as Figure 2 The propagation from the output layer 240 to the input layer 210 is called back propagation) and then the weight values ​​and biases of the aforementioned layers will be updated to reduce the loss of the convolutional neural network 200 and the error between the result output by the convolutional neural network 200 through the output layer and the ideal result.

[0081] It should be noted that if Figure 2 The convolutional neural network 200 shown is only an example of a convolutional neural network. In specific applications, the convolutional neural network can also exist in the form of other network models, such as U-Net, residual network (ResNet), etc.

[0082] A deep neural network (DNN), also known as a multi-layer neural network, can be understood as a neural network with many hidden layers. There is no special metric for "many" here. From the position of different layers of DNN, the neural network inside DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally speaking, the first layer is the input layer, the last layer is the output layer, and the layers in between are all hidden layers. The layers are fully connected, that is, any neuron in the i-th layer must be connected to any neuron in the i+1-th layer. Although DNN looks complicated, the work of each layer is actually not complicated. Simply put, it is the following linear relationship expression: Where x is the input vector, is the output vector, is the offset vector, W is the weight matrix (also called coefficient), and α() is the activation function. Each layer is just an input vector After such a simple operation, the output vector Since DNN has many layers, the coefficient W and the offset vector The definition of these parameters in DNN is as follows: Take coefficient W as an example: Assume that in a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as The superscript 3 represents the layer number of the coefficient W, while the subscripts correspond to the output third layer index 2 and the input second layer index 4. In summary, the coefficients from the kth neuron in the L-1th layer to the jth neuron in the Lth layer are defined as It should be noted that the input layer does not have a W parameter. In a deep neural network, more hidden layers allow the network to better describe complex situations in the real world. Theoretically, the more parameters a model has, the higher its complexity and the greater its "capacity", which means it can complete more complex learning tasks. Training a deep neural network is the process of learning the weight matrix, and its ultimate goal is to obtain the weight matrix of all layers of the trained deep neural network (the weight matrix formed by many layers of vector W).

[0083] Recurrent neural networks (RNN) are used to process sequence data. In traditional neural network models, from the input layer to the hidden layer and then to the output layer, the layers are fully connected, and the nodes within each layer are disconnected. Although this ordinary neural network has solved many difficult problems, it is still powerless for many problems. For example, if you want to predict the next word in a sentence, you generally need to use the previous word, because the previous and next words in a sentence are not independent. The reason why RNN is called a recurrent neural network is that the current output of a sequence is also related to the previous output. The specific manifestation is that the network will remember the previous information and apply it to the calculation of the current output, that is, the nodes between the hidden layers are no longer disconnected but connected, and the input of the hidden layer includes not only the output of the input layer but also the output of the hidden layer at the previous moment. In theory, RNN can process sequence data of any length. The training of RNN is the same as the training of traditional CNN or DNN. The same error back propagation algorithm is used, but there is a difference: if the RNN is expanded, the parameters, such as W, are shared; this is not the case with the traditional neural network mentioned above. And when using the gradient descent algorithm, the output of each step depends not only on the network at the current step, but also on the state of the network at the previous steps.

[0084] Since we already have convolutional neural networks, why do we need recurrent neural networks? The reason is simple. In convolutional neural networks, there is a premise that the elements are independent of each other, and the input and output are also independent, such as cats and dogs. But in the real world, many elements are interconnected, such as the change of stocks over time, or someone said: I like traveling, and my favorite place is Yunnan. I must go there if I have the chance in the future. Fill in the blank here, humans should all know to fill in "Yunnan". Because humans will make inferences based on the content of the context, but how can machines do this? RNN came into being. RNN is designed to give machines the ability to remember like humans. Therefore, the output of RNN needs to rely on the current input information and historical memory information.

[0085] In specific applications, recurrent neural networks can exist in the form of various network models, such as long short term memory networks (LSTM).

[0086] Loss function, in the process of training deep neural networks, because we hope that the output of the deep neural network is as close as possible to the value we really want to predict, we can compare the predicted value of the current network with the target value we really want, and then update the weight vector of each layer of the neural network according to the difference between the two (of course, there is usually an initialization process before the first update, that is, pre-configuring parameters for each layer in the deep neural network). For example, if the predicted value of the network is high, adjust the weight vector to make it predict lower, and keep adjusting until the deep neural network can predict the target value we really want or a value very close to the target value we really want. Therefore, it is necessary to pre-define "how to compare the difference between the predicted value and the target value", which is the loss function or objective function, which are important equations used to measure the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so the training of the deep neural network becomes a process of minimizing this loss as much as possible.

[0087] Back propagation algorithm, convolutional neural network can use error back propagation (BP) algorithm to correct the size of the parameters in the initial super-resolution model during the training process, so that the reconstruction error loss of the super-resolution model becomes smaller and smaller. Specifically, the forward transmission of the input signal to the output will generate error loss, and the error loss information is back-propagated to update the parameters in the initial super-resolution model, so that the error loss converges. The back propagation algorithm is a back propagation movement dominated by error loss, aiming to obtain the optimal parameters of the super-resolution model, such as the weight matrix.

[0088] Generally, when building a 3D scene in a rendering engine editor, the user selects the elements to be built from the element library and deploys the selected elements to be built into the scene. The element library includes a large number of elements, such as 5,000 elements. The elements include object models, maps, materials, lights or textures, etc.

[0089] Since each element in the three-dimensional scene needs to be determined by the user from the element library, the efficiency of element selection is slow, and thus the efficiency of building the three-dimensional scene is low.

[0090] To solve the problem of low building efficiency, Unreal Engine adopts a copy strategy, which can copy the elements of the previous deployment to quickly select elements, thereby speeding up the building efficiency of the 3D scene. Alternatively, by searching for elements in the element library in the search box in Unreal Engine, you can quickly find elements to speed up the building efficiency of the 3D scene.

[0091] However, when the three-dimensional scene to be built is complex or the three-dimensional scene includes many types of elements, the element selection efficiency is still slow when the above-mentioned copying or searching method is used to select elements. Then, when the three-dimensional scene is built using the selected elements, there is a problem of low efficiency in building the three-dimensional scene.

[0092] Based on this, the present application provides a training method for a recommendation model, which includes: obtaining training data, and using the training data to train the recommendation model. The training data includes one or more of the categories of multiple training elements, the distance between two elements in the scene, and the order in which the multiple training elements are added to the scene, and the training elements include at least one built element and / or at least one element in N elements to be built. Among them, the recommendation model is used to obtain N elements to be built at the first position according to the first position specified by the user in the scene building page and the information of at least one built element in the scene. The information of the built element includes one or more of the second position of the built element on the scene building page, the category of the built element, and the name of the built element, and N is a positive integer greater than or equal to 1.

[0093] In the present application, by training the training model, the training model obtains N elements to be built at the first position according to the first position specified by the user in the scene building page and the information of at least one built element in the scene building page. By using the information of the existing built elements in the scene, such as the position, category and name of the built elements, the elements that may be built at the first position are recommended for further selection by the user. In other words, the user does not need to manually select the required elements from a large number of elements, which reduces the manual consumption in the scene building process and is conducive to improving the efficiency of three-dimensional scene building. Furthermore, in response to the user's operation, at least one element is determined from the N elements to build a three-dimensional scene, which reduces the difficulty and complexity of element selection and further improves the efficiency of building a three-dimensional scene.

[0094] In addition, the present application also provides a scene building method, which can be applied to a computing device. The scene building method includes: providing a scene building page, and the scene in the scene building page includes at least one built element. Obtain a first position specified by a user in the scene building page, and then display N elements to be built in the scene building page in response to the specified first position and the information of at least one built element. In response to the user's operation on at least one of the N elements to be built in the scene building page, at least one element is deployed in the first position of the scene building page. Among them, the information of the built element includes one or more of the second position of the built element on the scene building page, the category of the built element, and the name of the built element, and N is a positive integer greater than or equal to 1.

[0095] In the present application, according to the first position specified by the user, N elements to be built at the first position are displayed in the scene building page, and the information of the existing built elements in the scene, such as the position, category and name of the built elements, is used to recommend the elements that may be built at the first position for the user to further select. In other words, the user does not need to manually select the required elements from a large number of elements, which reduces the manual consumption in the scene building process and is conducive to improving the efficiency of three-dimensional scene building. Furthermore, the computing device determines at least one element from the N elements in response to the user's operation to build a three-dimensional scene, which reduces the difficulty and complexity of element selection and further improves the efficiency of building a three-dimensional scene.

[0096] Next, the training method and scenario building method of the recommendation model provided in this application are described in detail with reference to the accompanying drawings.

[0097] First, see Figure 3 , Figure 3 This is a schematic diagram of the architecture of a computer system provided in this application. Figure 3 As shown, the computer system includes an execution device 310 , a training device 320 , a database 330 , a terminal device 340 , a data storage system 350 and a data acquisition device 360 ​​.

[0098] The execution device 310 can be a terminal, such as a mobile phone terminal, a tablet computer, a laptop computer, a virtual reality (VR) device, an augmented reality (AR) device, a mixed reality (MR) device, an extended reality (ER) device, a camera or a vehicle-mounted terminal, etc., and can also be an edge device (for example, a box carrying a chip with processing capability), etc.

[0099] The training device 320 may be a terminal or other computing device, such as a server or a cloud device.

[0100] As a possible embodiment, the execution device 310 and the training device 320 are different processors deployed on different physical devices (such as a server or a server in a cluster). For example, the execution device 310 can be a graphics processing unit (GPU), a central processing unit (CPU), other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The training device 320 can be a GPU, a neural network processing unit (NPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0101] In another possible embodiment, the execution device 310 and the training device 320 are deployed on the same physical device, or the execution device 310 and the training device 320 are the same physical device.

[0102] The data acquisition device 360 ​​is used to collect training data and store the training data in the database 330. The data acquisition device 360 ​​and the execution device 310 and the training device 320 can be the same or different devices. The training data includes data in at least one form of image, voice and text. For example, the training data includes image data, and each image in the image data has multiple different types of elements.

[0103] The training device 320 is used to train the neural network using the training data until the loss function in the neural network converges, and the neural network training is completed when the loss function value is less than a specific threshold, so that the neural network reaches a certain accuracy. Alternatively, all the training data in the database 330 are used for training, and the neural network training is completed, so that the trained neural network has the function of recommending elements to be built based on existing elements in the interface. Then, the training device 320 configures the trained neural network 301 to the execution device 310. The execution device 310 is used to implement the function of processing input data according to the trained neural network 301.

[0104] In some embodiments, the execution device 310 and the training device 320 are the same computing device, and the computing device can configure the trained neural network 301 to itself, and use the trained neural network 301 to implement functions such as adjusting the pixel values ​​of the pixel points of the image or identifying the categories of image data in various different fields.

[0105] In other embodiments, the training device 320 may configure the trained neural network 301 to multiple execution devices 310. Each execution device 310 uses the trained neural network 301 to implement functions such as recommending elements to be built based on the built elements in the scene building page.

[0106] In combination with a computer system, the training method of the recommendation model provided in this embodiment can be applied to the scene construction scene in the rendering engine. Specifically, the training method of the recommendation model of the embodiment of the present application can be applied to game scene construction, urban planning, garden design and other scenes. The following briefly introduces the application of the training method of the recommendation model in game scene construction, urban planning, and garden design scenes.

[0107] For example, the training device 320 receives training data input by the user or obtains training data from the database 330. When the training method of the recommendation model is applied to the construction of a game scene, the training data includes the layout information of each element in the game scene (including one or more of the location, category, and name), and the element may be a tree, a stone, a building, a pool, etc. When the training method of the recommendation model is applied to urban planning, the training data may include the layout information of each element in the city, and the element may be a telephone pole, an electric box, a building, a road, etc. When the training method of the recommendation model is applied to garden design, the training data may include the layout information of each element in a park or a community courtyard, and the element may be various varieties of trees, flowers, and plants, etc.

[0108] In other embodiments of the present application, the training method of the above-mentioned recommendation model can also be applied to planning scenarios such as community planning, home decoration design or exhibition design.

[0109] It should be noted that, in actual applications, the training data (e.g., image data) maintained in the database 330 does not necessarily all come from the data acquisition device 360, but may also be received from other devices. In addition, the training device 320 does not necessarily train the neural network entirely based on the training data maintained by the database 330, but may also obtain training data from the cloud or other places to train the neural network. The above description should not be used as a limitation on the embodiments of the present application.

[0110] Furthermore, according to the functions performed by the execution device 310, the execution device 310 can be further subdivided, such as Figure 3 As shown, the execution device 310 is configured with a calculation module 311 , an I / O interface 312 and a pre-processing module 313 .

[0111] The I / O interface 312 is used to exchange data with external devices. A user can input data to the I / O interface 312 through the terminal device 340. The input data may include images or videos. In addition, the input data may also come from the database 330.

[0112] The preprocessing module 313 is used to perform preprocessing according to the input data received by the I / O interface 312. In the embodiment of the present application, the preprocessing module 313 can be used to improve the clarity of the image data received from the I / O interface 312, etc.

[0113] When the execution device 310 preprocesses the input data, or when the computing module 311 of the execution device 310 performs calculations and other related processing, the execution device 310 can call the data, code, etc. in the data storage system 350 for corresponding processing, and can also store the data and instructions obtained from the corresponding processing in the data storage system 350.

[0114] For example, the first neural network stored in the execution device 310 can be applied to the execution device 310. After the execution device 310 obtains the image data, the calculation module 311 inputs the image data into the first neural network to obtain a processing result. Since the first neural network is trained by the training device 320 based on the image data, each of the image data has different types of elements. Therefore, the input image data or the position and category or name of the element is processed by the first neural network to meet the requirement of recommending elements to be built to the user.

[0115] Finally, the I / O interface 312 returns the processing result to the terminal device 340, thereby providing it to the user so that the user can view the processing result. It should be understood that the terminal device 340 and the execution device 310 can also be the same physical device.

[0116] exist Figure 3In the case shown, the user can manually input data, which can be operated through the interface provided by the I / O interface 312. In another case, the terminal device 340 can automatically send input data to the I / O interface 312. If the terminal device 340 needs to obtain the user's authorization to automatically send the input data, the user can set the corresponding authority in the terminal device 340. The user can view the processing results output by the execution device 310 on the terminal device 340, and the specific presentation form can be display, sound, action and other specific methods. The terminal device 340 can also be used as a data acquisition terminal to collect data such as Figure 3 The input data of the input I / O interface 312 and the processing result of the output I / O interface 312 are taken as new sample data and stored in the database 330. Of course, it is also possible not to collect through the terminal device 340, but to have the I / O interface 312 collect the sample data. Figure 3 The input data of the input I / O interface 312 and the processing results of the output I / O interface 312 are stored in the database 330 as new sample data.

[0117] Figure 3 This is only a schematic diagram of a system architecture provided by an embodiment of the present application. Figure 3 The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. Figure 3 In the embodiment, the data storage system 350 is an external memory relative to the execution device 310. In other cases, the data storage system 350 can also be placed in the execution device 310. In an embodiment provided by the present application, one or more of the computing module 311, the I / O interface 312, the preprocessing module 313, the training device 320, the database 330, and the data storage system 350 can be functional modules in software such as a rendering engine and a design engine. These functional modules can be deployed in a terminal device or at least one data center, and the present application does not limit this.

[0118] On the one hand, the present application provides a training method for a recommendation model, which can be applied to Figure 3 In the computer system shown. Figure 4 The flowchart of the training method of the recommendation model provided in the present application is shown in FIG. 4 . The training method of the recommendation model can be executed by a first processing device 400. The first processing device 400 can be Figure 3 The training device 320 in this embodiment is as follows Figure 4 The method shown is applied to the game scene construction as an example for explanation. Figure 4 As shown, the training method of the recommendation model may include the following steps S410 and S420.

[0119] S410: The first processing device 400 obtains training data.

[0120] The training data includes one or more of: categories of multiple training elements, distances between two elements in the multiple training elements in the scene, and an order in which the multiple training elements are added to the scene.

[0121] Exemplarily, the category of the training element refers to the type to which the training element belongs, such as a chair, a table, etc.

[0122] The distance between two training elements in the scene among multiple training elements represents the actual distance between the two training elements in the scene. For example, when the training data is a picture, the distance between the chair and the table in the picture is the distance between the two training elements in the scene. Because in the process of scene construction, the position layout of different elements has a strong correlation. For example, in the process of building a forest in a game scene, the positions of various types of trees are deployed close to each other, while houses are usually far away from trees; or, in the process of building a house, the positions of tables and chairs are deployed close to each other, while the positions of tables, chairs and beds are far away. Therefore, using the distance between elements and the category / name of the elements (such as table, chair) as training data can enable the recommendation model to learn the relationship between the distance between elements. Furthermore, the trained recommendation model can recommend elements to be built to users based on the category / name of the elements and the distance between the elements.

[0123] The order in which multiple training elements are added to the scene represents the order in which the elements are built in the scene. For example, the order in which the elements are built in the scene may be (chair, 1), (chair, 2), (table, 3), etc., where 1, 2, 3 represent the order in which the elements are added to the scene building page.

[0124] During the scene construction process, the order in which elements are added to the scene often has a strong logical association. For example, in the process of building a house in a game scene, chairs and tables are usually added to the scene one after another; or, in the process of building a forest, various types of trees and grass are often added to the scene one after another. Therefore, on the basis of using the distance between elements and the category / name of the elements as training data, the construction order is also used as training data, so that the recommendation model can learn the logical association of the order in which the aforementioned elements are added to the scene. Furthermore, the trained recommendation model can more accurately recommend elements to be built to users based on the acquired construction order, the distance between elements, and the category / name of the elements.

[0125] The training method of the recommendation model shown in this embodiment is applied to the game scene construction process, so the training data can be extracted from image data or historical scene data. Each image in the image data indicates multiple categories of training elements, such as multiple combinations of different types of animals, grass, people (children or adults), trees, stones, etc.; and the distance between two training elements.

[0126] In a first possible example, the image data contains training elements of various categories, as well as distances between any two of the training elements, such as the distance between an animal and a person, the distance between a person and a tree, and the like.

[0127] In a second possible example, the above image data only shows a large number of training elements. The first processing device 400 performs semantic recognition or semantic segmentation on the image data to obtain the category of each training element in the image data and the distance between two training elements.

[0128] In a third possible example, the historical scene data shows multiple categories of training elements, the distances between any two training elements, and the order in which the training elements are added to the scene.

[0129] Exemplarily, the training element is at least one constructed element that has been deployed in the scene.

[0130] It is worth noting that the above-mentioned image data is only a general term. In actual application scenarios, the image data may be an image of a completed game scene, or a picture of the real world, etc. Similarly, the historical scene data is also only a general term. In actual application scenarios, the historical scene data may be a completed game scene, such as scene data output by a rendering engine, etc. In other embodiments provided in this application, the training data may be the final state data or intermediate data output by a model scene that has been built or is being built, and this embodiment of the application does not limit this.

[0131] With respect to the first processing device 400 acquiring training data, two possible examples are provided below.

[0132] Example 1: The first processing device 400 obtains training data from the database 330, such as the above-mentioned image data or historical scene data.

[0133] Example 2: The first processing device 400 receives training data input by a user, such as the above-mentioned image data or historical scene data.

[0134] In a possible scenario, the training data also includes the name of the training element, or other identifiers or labels of the training element, such as an identity document (ID) of the element, which is not limited in this application.

[0135] For example, when the category of the training element is chair, the name of the training element may be red table chair, blue recliner, etc.

[0136] S420: The first processing device 400 trains the recommendation model using the training data.

[0137] The recommendation model is used to obtain N elements to be built at the first position according to the first position specified by the user in the scene building page and the information of at least one built element in the scene of the scene building page. The information of the built element includes one or more of the second position of the built element in the scene building page, the category of the built element and the name of the built element, and N is a positive integer greater than or equal to 1.

[0138] In one possible scenario, the scene building page is a display interface of a rendering engine that uses the above-mentioned recommendation model.

[0139] Exemplarily, the scene building page is a display interface corresponding to the rendering engine editor in the rendering engine during the scene editing stage.

[0140] In a possible example, the first processing device 400 completes training of the recommendation model using the training data to obtain a trained recommendation model.

[0141] Two possible examples are provided below for the network structure of the recommendation model.

[0142] Example 1: The network structure of the recommendation model is a DNN network structure. For the description of DNN, please refer to the content of the above-mentioned deep neural network, which will not be described in detail here.

[0143] Example 2: The network structure of the recommendation model is a transformer network structure. The transformer network structure is essentially an encoder-decoder structure. The encoder includes self-attention and a feed forward neural network (FNN), and the decoder includes self-attention, FNN, and encoder-decoder attention. The two attentions are used to calculate the weights of the input and output, respectively. For example, the transformer has six layers of encoder and decoder.

[0144] The above content is only an example provided by the present application and should not be construed as a limitation of the present application. In other embodiments of the present application, the network structure of the recommendation model may also be a CNN network structure, a RNN network structure, or a combination of RNN and CNN network structures.

[0145] Regarding the first processing device 400 using training data to train the recommendation model, three possible embodiments are provided below.

[0146] In a first possible embodiment, the first processing device 400 inputs the training data into the recommendation model to obtain a training result, and calculates a loss function based on the training result and the label (category or name) corresponding to the training data, and then performs backpropagation on the recommendation model according to the loss function to update the parameters of the recommendation model, thereby completing the pre-training of the recommendation model and obtaining a trained recommendation model.

[0147] In a possible example, the training process includes multiple rounds, and the training process of each round is as described above, which is not described in detail here. When the loss function converges, the loss function is less than a set threshold, or the training data is used up, the training of the recommendation model is stopped.

[0148] Then, after training, the recommendation model can obtain N elements to be built at the first position according to the information of the elements that have been built in the scene building page and the first position specified by the user. Figure 8 The contents shown will not be repeated here.

[0149] In a second possible embodiment, the first processing device 400 performs reinforcement learning training on the recommendation model when using the recommendation model.

[0150] In the process of using the recommendation model, the first processing device 400 performs reinforcement learning training on the recommendation model. The following provides a possible implementation method, such as Figure 5 As shown, Figure 5 A schematic diagram of a process flow of a reinforcement learning method provided by the present application. The method may include the following steps S510 to S540.

[0151] S510: The first processing device 400 obtains a second position specified by a user in a scene building page.

[0152] In a possible example, the current scene building page includes the category and name of the built element and the second position of the built element in the scene building page, wherein the second position specified by the user is obtained by the first processing device 400 from the front end.

[0153] The front end may be an input device built into the first processing device 400, such as a display screen, etc.; or, the front end may be an input device connected to the first processing device 400, such as a keyboard, a mouse, etc.

[0154] For the first processing device 400 to obtain more content of the second position specified by the user in the scene building page, reference can be made to the second processing device 600 to obtain the content of the first position specified by the user in the scene building page, which will not be described in detail here.

[0155] S520: The first processing device 400 uses the recommendation model to infer at least one element to be built at the second position according to the second position.

[0156] For example, the first processing device 400 infers that the deployable to-be-built element at the second position is a telephone, a recorder, an armchair, or a recliner.

[0157] The recommended model obtains the representation of N elements to be built based on the second position, which can be referred to as follows Figure 8 The contents shown will not be repeated here.

[0158] S530: The first processing device 400 receives user feedback on at least one element to be constructed at the second position.

[0159] The feedback is used to indicate the user's selection result of at least one element to be built at the second position. If the selection result is that the user did not select N elements to be built, the feedback is negative feedback. If the selection result is that the user selected one or more elements among the N elements to be built, the feedback is positive feedback.

[0160] S540: The first processing device 400 trains the recommendation model according to the positive feedback or the negative feedback.

[0161] In one possible implementation, the first processing device 400 sets rewards to give the recommendation model positive rewards when there is positive feedback, and gives the recommendation model negative rewards when there is negative feedback, and continuously trains the recommendation model based on the aforementioned rewards so that the rewards corresponding to the recommendation model continue to approach the positive rewards, thereby obtaining a trained recommendation model.

[0162] In the present application, the first processing device 400 trains the recommendation model through reinforcement learning, which can further improve the accuracy of the recommendation model in determining the N elements to be built.

[0163] Therefore, the training element in this embodiment includes: the at least one element to be built and the user's feedback on the at least one element to be built.

[0164] In a third possible embodiment, the first processing device 400 may pre-train the recommendation model according to the content shown in the first possible embodiment, and continuously train the recommendation model during the use of the recommendation model according to the content shown in the second possible embodiment.

[0165] For the specific content of this embodiment, reference may be made to the contents shown in the aforementioned first possible embodiment and the second possible embodiment, which will not be described in detail here.

[0166] The contents shown in the above three embodiments are only examples. In other embodiments of the present application, the first processing device 400 may also use a distributed training method to train the recommendation model.

[0167] Here are some possible examples of what the element represents.

[0168] Example 1: The above elements are objects, such as trees, tables and chairs, roads, etc., which occupy part of the area in the scene building page.

[0169] Example 2: The above elements are textures, such as pictures of trees, tables and chairs, etc., which occupy very little area in the scene building page.

[0170] Example 3, the above element is light.

[0171] Example 4: The above elements are materials, such as the bark of a tree or the surface material of an iron object.

[0172] Example 5, the above element is a texture, for example, the specific texture of the bark, a rough or fine texture.

[0173] Example 6: The above elements are a combination of any two of objects, maps, materials, lights and textures.

[0174] Example 7: The above elements are a combination of any three of objects, maps, materials, lights and textures.

[0175] Example 8: The above elements are a combination of any four of objects, maps, materials, lights and textures.

[0176] Example 9, the above elements are objects, maps, materials, lights and textures.

[0177] In addition, the present application also provides a scene construction method, which can be applied in Figure 3 In the computer system shown. Figure 6 This is a schematic diagram of the scene building page provided by this application. The content of the scene building method of this application can be obtained by Figure 6 The scene construction method can be executed by the second processing device 600, and specifically can be executed by the scene construction device in the second processing device 600. The second processing device 600 can be Figure 3 The execution device 310 in this embodiment is as follows Figure 6 The method shown is applied to the construction of a game scene as an example for explanation.

[0178] The second processing device 600 provides a scene building page (such as Figure 6As shown in a, the scene building page may include a navigation bar control component 401, an element selection control component 402, an element editing control component 403, an element recommendation control component 404, a scene layout display control component 405, etc.).

[0179] The navigation bar control component 401 is used to display various function keys of the rendering engine, such as "Insert", "Page", etc. When the user clicks various function keys, the second processing device 600 can perform corresponding processing or display a corresponding interface.

[0180] The element selection control component 402 is used to display the selectable elements in the element library and provide the user with options. For example, the element selection control component 402 displays the elements according to their types, that is, displays the elements of the corresponding types according to the original types determined by the user.

[0181] The element editing control component 403 is used to perform editing and other zooming operations on the element selected by the user.

[0182] The element recommendation control component 404 is used to recommend N elements to be built according to the elements that have been built in the scene building page.

[0183] The scene layout display control component 405 is used to display the elements selected by the user, and the position of each element is specified by the user.

[0184] It is worth noting that the above-mentioned scene building page may also include more or fewer control components to achieve more or fewer functions, which is not limited in this application.

[0185] Since the present application is applied to the game scene construction scene, the scene construction page is the display interface of the rendering engine during the scene construction process. For example, the rendering engine can be a game rendering engine.

[0186] In a possible example, the second processing device 600 displays the scene building page on a front end, which is a display built into the second processing device 600 or a display device connected to the second processing device 600.

[0187] In a possible embodiment, the second processing device 600 receives a function activation instruction sent by a user, and then executes the following steps: Figure 6 Steps shown in b.

[0188] This function enable command is used to enable the function of recommending elements to be built.

[0189] Exemplarily, the user may trigger the function activation instruction in the scene building page. For example, the user triggers operation a on the scene building page to send the function activation instruction to the second processing device 600.

[0190] For operation a, two possible examples are provided below.

[0191] Example 1: The operation a may be a shortcut key triggered by the user on the input device, or a preset key (such as alt key + B key).

[0192] For example, after receiving the shortcut key or the preset key, the second processing device 600 triggers the function start instruction and executes the following steps: Figure 6 The function shown in c.

[0193] Example 2: Operation a may be a gesture operation performed by a user on an input device, such as a sliding operation performed by a user on the front end of the second processing device 600. The front end may be a display screen built into the second processing device 600, or a display device or a mouse connected to the second processing device 600.

[0194] For example, after the second processing device 600 recognizes the gesture operation, it triggers the function start instruction and executes the following steps: Figure 6 The function shown in c.

[0195] Example 3, the operation a can recommend a control component for the user to click on an element on the scene building page.

[0196] The above examples are only optional implementations provided in this embodiment and should not be construed as limiting the present application. In other embodiments of the present application, the operation a may also be remote operation, voice control, bone movement or knuckle tapping.

[0197] The second processing device 600 obtains the first position specified by the user in the scene building page (eg Figure 6 As shown in b, the user clicks a specified location in the scene building page).

[0198] In a possible example, the first position specified by the user is located in the scene layout display control component 405 in the scene building page, that is, in the scene built by the user.

[0199] Regarding the second processing device 600 obtaining the content of the first position specified by the user in the scene building page, reference may be made to the following description of the second operation, which will not be described in detail here.

[0200] The second processing device 600 displays N elements to be constructed (such as Figure 6 As shown in c, the second processing device 600 displays N elements to be built on the left side of the scene building page).

[0201] The information of the built element includes one or more of the second position of the built element on the scene building page, the category of the built element and the name of the built element, and N is a positive integer greater than or equal to 1.

[0202] In a possible implementation, the second processing device 600 obtains N elements to be constructed at the first position according to the first position specified by the user and information of at least one constructed element, and then displays the N elements to be constructed in the scene construction page.

[0203] For the content of the above implementation, please refer to the following Figure 8 The contents shown will not be repeated here.

[0204] Regarding the second processing device 600 displaying the contents of N elements to be constructed in the scene construction page, three possible examples are shown below. Figure 7 As shown, Figure 7 A schematic diagram showing the elements to be built provided for this application.

[0205] Example 1, such as Figure 7 As shown in a, the second processing device 600 displays N elements to be built around the first position in the scene building page.

[0206] For example, N elements to be constructed are displayed in a circle or polygon shape around the first position.

[0207] In the present application, the second processing device 600 displays N elements to be built around the first position in the scene building page. While realizing the visualization of the elements, displaying N elements to be built around the first position can improve the user's efficiency in selecting elements.

[0208] Example 2, such as Figure 7 As shown in b, the second processing device 600 displays N elements to be built on multiple sides of the scene building page.

[0209] In the present application, the second processing device 600 displays N elements to be built on one or more sides of the scene building page. While realizing element visualization, it can avoid obstruction of the built elements when displaying the N elements to be built, thereby improving the integrity of the scene building page.

[0210] Example 3, such as Figure 7 As shown in c, the second processing device 600 displays N elements to be built within a preset range from the first position in the scene building page.

[0211] In a possible scenario, two possible examples are provided below for the display form of N elements to be constructed.

[0212] Example 1: The second processing device 600 displays the element to be built in a proportional manner. For example, the second processing device 600 displays the element to be built in a reduced size, or the second processing device 600 displays the element to be built in an enlarged size.

[0213] Example 2: The second processing device 600 displays the element to be constructed in the form of dotted lines.

[0214] The above content is only an example provided by the present application and should not be understood as limiting the present application. In other examples of the present application, the second processing device 600 displays the elements to be constructed in the form of solid lines.

[0215] The second processing device 600 deploys the at least one to-be-built element at a first position (e.g., Figure 6 (shown in d).

[0216] In a possible implementation, the second processing device 600 receives a first operation of a user on at least one of N elements to be built on the scene building page, and then in response to the first operation, deploys the at least one element to be built at a first position on the scene building page.

[0217] In a possible example, the second processing device 600 obtains a first operation of a user on a control component on a scene building page through various input devices (keyboard, mouse, touch screen, etc.). The input device is connected to the second processing device 600.

[0218] Regarding the specific implementation of the first operation, three possible examples are provided below.

[0219] Example 1: The first operation may be a user confirming at least one of the N elements to be built through the keyboard, such as the first operation is a confirmation (enter) key triggered by the user, or the user pre-selects the elements to be built through the arrow keys and confirms at least one element to be built through the enter key.

[0220] Example 2: The first operation may be a user clicking and confirming at least one of the N elements to be constructed by using a mouse.

[0221] Example 3: The first operation may be a user clicking or sliding on the touch screen on at least one of the N elements to be built.

[0222] The above examples are only optional implementations provided in this embodiment and should not be construed as limiting the present application. In other embodiments of the present application, the first operation may also be air operation, voice control, gesture control, bone movement or knuckle tapping.

[0223] With respect to the content of at least one element selected by the user in the scene building page, two possible examples are provided below.

[0224] In a possible example, if N elements to be built are displayed in a proportional scaling form on the scene building page, the second processing device 600 displays the at least one element to be built in the original size when deploying the at least one element to be built to the scene layout display control component 405 .

[0225] In a possible example, if N elements to be built are displayed in dotted line form on the scene building page, after the user determines the at least one element to be built, the at least one element to be built will be converted from dotted line form to solid line form.

[0226] In a possible embodiment, when the second processing device 600 displays N elements to be built in the scene building page, it displays them in an upper layer of the scene building page in the form of a pop-up window, and the pop-up window includes the N elements to be built.

[0227] In a possible example, the pop-up window may be in a square shape. The present application does not limit the form of the pop-up window. In other embodiments of the present application, the pop-up window may be in a circular or polygonal shape.

[0228] In the present application, the second processing device 600 displays N elements to be built in the form of a pop-up window, which can highlight the N elements to be built, so that the user can promptly notice the N elements to be built and select them, which can improve the efficiency of selecting elements and thereby improve the efficiency of building the three-dimensional scene.

[0229] With respect to the second processing device 600 acquiring the content of the first position specified by the user in the scene building page, a possible implementation manner is provided below.

[0230] The second processing device 600 receives a second operation of the user on the scene building page, and then obtains a first position specified by the user on the scene building page in response to the second operation.

[0231] For the specific implementation of the second operation, three possible examples are provided below.

[0232] Example 1: The second operation may be a shortcut key triggered by the user on the input device, or a preset key (such as alt key + B key), etc.

[0233] For example, after receiving the shortcut key or the preset key, the second processing device 600 determines the position of the cursor as the first position.

[0234] Example 2: The second operation may be a gesture operation performed by the user on the input device, such as a sliding operation performed by the user on the front end of the second processing device 600. The front end may be a display screen built into the second processing device 600, or a display device or a mouse connected to the second processing device 600.

[0235] For example, after recognizing a gesture operation, the second processing device 600 triggers element recommendation for the termination position of the gesture operation, that is, the termination position of the gesture operation is used as the first position.

[0236] Example 3: The second operation may be a click operation in the scene layout display control component 405 after the user clicks the element recommendation control component on the scene building page. The click operation indicates the position in the scene layout display control component 405, namely the first position.

[0237] For example, after the user clicks the element recommendation control component on the scene building page, each time the user clicks a position in the scene layout display control component 405, the position is determined to be the first position.

[0238] The above contents are only optional examples provided in this embodiment and should not be construed as limiting the present application. In other examples of the present application, the second operation may also be remote operation, voice control, bone movement or knuckle tapping.

[0239] In the present application, the second processing device 600 receives the second operation performed by the user in various ways on the scene building page, and in response to the second operation, obtains the first position specified by the user on the scene building page. This triggers the second processing device 600 to determine the N elements to be built at the first position, thereby avoiding the user from manually selecting the required elements from a large number of elements, thereby reducing the difficulty and complexity of the user's element selection and improving the efficiency of building a three-dimensional scene.

[0240] With respect to the second processing device 600 displaying the contents of N elements to be constructed in the scene construction page in response to the designated first position and the information of at least one constructed element, three possible embodiments are provided below.

[0241] In a first possible embodiment, Figure 8 As shown, Figure 8 The schematic diagram of the flow chart of the element determination method provided in the present application. The method may include the following steps S810 to S830.

[0242] S810: The second processing device 600 calculates the distance between the second position and the first position of the constructed element, and determines from the scene construction page some constructed elements whose distance from the first position is less than or equal to the target distance.

[0243] In a possible implementation, the second processing device 600 determines some of the constructed elements within a circular range with the first position as the center and the target distance (such as the actual distance of 2 meters in the scene construction page) as the radius.

[0244] The first position and the second position are both positions in the scene layout display control component 405, that is, actual positions in the game scene.

[0245] S820: The second processing device 600 obtains N elements to be constructed according to information of some constructed elements.

[0246] In a possible scenario, the second processing device 600 obtains N elements to be constructed according to the second positions in the information of the partially constructed elements and the categories or names of the constructed elements and the first positions.

[0247] With respect to the second processing device 600 obtaining the contents of N elements to be constructed according to the information of some constructed elements, three possible implementations are provided below.

[0248] In a first possible implementation, the second processing device 600 determines the target category of the element to be built based on the information of the built element, and then calculates the first similarity between the categories of multiple elements in the element library and the target category of the element to be built, thereby determining the N elements to be built that are ranked first in the first similarity. In this implementation, the information of the built element includes the category of the built element and the second position of the built element in the scene building page.

[0249] In a possible example, the second processing device 600 calculates the distance a between the second position and the first position, and then inputs the distance a and the category of the built element into the recommendation model to determine the target category of the element to be built. Then, the second processing device 600 calculates the first similarity between the categories of multiple elements in the element library and the target category of the element to be built, thereby determining the N elements to be built that are ranked top in the first similarity.

[0250] For example, if the second position is (1, 3) and the first position is (2, 4), the distance a is

[0251] In another possible example, the second processing device 600 inputs the second position, the first position, and the category of the built element into the recommendation model to determine the target category of the element to be built. Then, the second processing device 600 calculates the first similarity between the categories of the multiple elements in the element library and the target category of the element to be built, thereby determining the N elements to be built that are ranked top in the first similarity.

[0252] In the present application, the second processing device 600 determines N elements to be built at the first position through the recommendation model, which can improve the efficiency of obtaining the N elements to be built, thereby improving the efficiency of building the three-dimensional scene.

[0253] Exemplarily, the above recommendation model is Figure 4 or Figure 5 The recommendation model after training the content shown is not described in detail here.

[0254] After the second processing device 600 obtains that the target category of the element to be built is a chair, the first similarity between the categories of multiple elements in the element library (such as chairs, sofas, ergonomic chairs, etc.) and the chair is calculated. For example, the similarity between chairs is 1, the similarity between chairs and sofas is 0.5, and the similarity between chairs and ergonomic chairs is 0.9.

[0255] The second processing device 600 selects the categories of the M elements ranked first in the first similarity order, such as the first two, namely, chair and ergonomic chair. Then, the second processing device 600 determines N elements to be constructed from the elements of the categories of chair and ergonomic chair.

[0256] Regarding the above content of determining N elements to be constructed from multiple categories of elements (such as chairs or ergonomic chairs), three possible examples are provided below.

[0257] Example 1: The second processing device 600 selects the same number of elements from multiple categories of elements as elements to be constructed.

[0258] Example 2: The second processing device 600 selects elements as the elements to be built in proportion according to the first similarity corresponding to the category. For example, when it is determined that 10 elements to be built are displayed in the scene building page, 7 elements to be built are selected from the elements of the category of chairs, and 3 elements to be built are selected from the elements of the category of ergonomic chairs.

[0259] Example 3: The second processing device 600 determines the similarity a between the elements in the categories of chairs and ergonomic chairs and the target category of the elements to be built, and then selects N elements ranked top in similarity a as the elements to be built.

[0260] The similarity between the above-mentioned element category and the element can be set in advance by the user, for example, one category of elements is associated with multiple element instances, or calculated by the second computing device 600. The similarity between the element category and the element can represent the similarity between the element category and the element name.

[0261] For example, if the category of the element is chair, multiple elements associated with the chair may be benches, armchairs, recliners, etc. If the category of the element is microphone, multiple elements associated with the microphone may be telephones, recorders, airplanes, etc.

[0262] The similarity between the above elements and the target category can be determined by the semantic distance between the name / ID of the element and the target category. The smaller the semantic distance, the greater the similarity, and the larger the semantic distance, the smaller the similarity. Therefore, the N elements ranked first in the above similarity are the N elements ranked last in the semantic distance, and the ranking is arranged in descending order.

[0263] Exemplarily, the second processing device 600 inputs the type of the constructed element and the distance between the second position and the first position into the Figure 4 In the recommendation model trained by the method shown, the target category of the element to be built at the first position is determined to be a microphone or a chair. The second processing device 600 determines two elements with the highest similarity to the microphone from the telephone (such as 0.05), tape recorder (such as 0.02), and airplane (such as 0.3) associated with the microphone, such as a telephone and a tape recorder. In addition, the second processing device 600 determines two elements with the highest similarity to the chair from the bench (0.025), armchair (such as 0.02), and recliner (such as 0.3) associated with the chair, such as an armchair and a recliner.

[0264] The content in the above brackets represents the semantic distance, such as telephone (such as 0.05) represents the semantic distance between the microphone and the telephone.

[0265] In other examples of the present application, the second processing device 600 may directly input the category, the second position, and the first position of the constructed element into the recommendation model.

[0266] In a second possible implementation, the second processing device 600 determines the target name of the element to be built based on the information of the built element, and then calculates the second similarity between the names of multiple elements in the element library and the target name of the element to be built, thereby determining the N elements to be built that are ranked top by the second similarity. In this implementation, the information of the built element includes the name of the built element and the second position of the built element in the scene building page.

[0267] In a possible example, the second processing device 600 calculates the distance b between the second position and the first position, and then inputs the distance b and the name of the built element into the recommendation model to determine the target name of the element to be built. Then, the second processing device 600 calculates the first similarity between the names of multiple elements in the element library and the target name of the element to be built, thereby determining the N elements to be built that are ranked top in the first similarity.

[0268] Exemplarily, after the second processing device 600 obtains that the target name of the element to be built is a gaming chair, the second similarity between the names of multiple elements in the element library (such as lazy sofas, desks and chairs, gaming chairs, gaming recliners, etc.) and the gaming chair is calculated. For example, the similarity between the lazy sofa and the gaming chair is 0.2, the similarity between the desks and chairs and the gaming chair is 0.5, the similarity between the gaming chair and the gaming chair is 1, and the similarity between the gaming recliner and the gaming chair is 0.8. The second processing device 600 determines the two elements to be built that are ranked higher in the first similarity, namely, the gaming chair and the gaming recliner.

[0269] In other examples of the present application, the above N may also be 3 or 7, etc. The second processing device 600 may directly input the name, the second position and the first position of the constructed element into the recommendation model, which is not limited in the present application.

[0270] In a third possible implementation, the first possible implementation is combined with the second possible implementation, that is, the second processing device 600 can determine the name and category of the element to be built according to the category and name of the built element, and then determine the top N elements to be built with the highest similarity.

[0271] For the details of this implementation, please refer to the first possible implementation and the second possible implementation. The above content is only a possible implementation provided by this application and should not be understood as a limitation of this application. In other implementations of this application, the second processing device 600 determines the target ID of the element to be built based on the ID and the second position of the built element, and then calculates the third similarity between the IDs of multiple elements in the element library and the target ID of the element to be built, thereby determining the N elements to be built that are ranked top in the third similarity.

[0272] Please continue to see Figure 8 , the method provided in this embodiment also includes step S830.

[0273] S830: The second processing device 600 displays N elements to be constructed in the scene construction page.

[0274] As in the above example, the second processing device 600 displays the telephone, recorder, armchair and recliner to be built in the scene building page.

[0275] For the second processing device 600 to display the contents of N elements to be constructed in the scene construction page, please refer to the above Figure 6 , Figure 7 The contents shown will not be repeated here.

[0276] In the present application, the second processing device 600 uses the label (category or name) of the built elements and the distance between the second position and the first position to determine N elements to be built. The N elements to be built are more in line with the content representation of the built scene (including multiple elements), avoiding the user's manual selection of required elements from a large number of elements, thereby reducing the difficulty and complexity of the user's element selection, improving the efficiency of element selection, and thus improving the efficiency of building the three-dimensional scene.

[0277] In a second possible embodiment, the second processing device 600 determines N elements to be constructed according to information of at least one constructed element in the scene construction interface, and then displays the N elements to be constructed in the scene construction page.

[0278] In this embodiment, the second processing device 600 inputs the third position of at least one built element in the scene building page, the category or name of at least one built element, and the first position into the recommendation model to obtain N elements to be built at the first position, and then displays the N elements to be built in the scene building page.

[0279] Exemplarily, the constructed elements in the scene construction page include chair (1, 1) and table (1, 2), and the first position specified by the user is (2, 2). The second processing device 600 inputs the chair (1, 1), table (1, 2) and the first position (2, 2) into the recommendation model to obtain the elements to be constructed: chair and recliner.

[0280] For more details of this embodiment, please refer to the contents of the first possible embodiment mentioned above, which will not be elaborated here.

[0281] In the third possible embodiment, it is a combination of the contents of the first possible embodiment and the second possible embodiment. The specific contents can refer to the first possible embodiment and the second possible embodiment, which will not be described in detail here.

[0282] In another possible embodiment of the present application, the information of the constructed elements also includes: the order in which the constructed elements are added to the scene construction page, i.e., the construction order. Then, the second processing device 600 inputs the category, construction order, second position and first position specified by the user of the constructed elements into the recommendation model, and obtains N elements to be constructed at the first position, and then in response to the user's operation on at least one of the N elements to be constructed, the at least one element to be constructed is deployed at the first position.

[0283] For details of the above embodiments, please refer to the above Figure 6-Figure 8 The contents shown will not be repeated here.

[0284] In the present application, the first processing device 400 introduces the order in which the built elements are added to the scene building page, and then when determining the N elements to be built, an additional dimension of indicator (the order in which the built elements are added to the scene building page) is used for determination, thereby improving the accuracy of the content representation of the N elements to be built that conforms to the built elements.

[0285] It is understandable that, in order to implement the functions in the above-mentioned embodiments, the first processing device 400 and the second processing device 600 may be the same or different terminals, and the terminals include hardware structures and / or software modules corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0286] Combination of the above Figures 3 to 5 The training method of the recommendation model provided in this embodiment is described in detail. Fig. 9 , describing the training device of the recommendation model provided according to this embodiment.

[0287] Fig. 9 Schematic diagram of the structure of the training device for the recommended model provided in this application Figure 1 The training device of the recommendation model can be used to implement the function of the training device in the above method embodiment, and thus can also achieve the beneficial effects possessed by the above method embodiment. In this embodiment, the training device of the recommendation model can be as follows: Figure 3 The training device 320 shown may also be a module (such as a chip) applied to a server.

[0288] The training device 900 for the recommendation model includes an acquisition module 910 and a training module 920. The training device 900 for the recommendation model is used to implement the above Figure 4 The functions of the first processing device 400 in the method embodiment shown in FIG.

[0289] The acquisition module 910 is used to acquire training data. The training data includes one or more of the categories of multiple training elements, the distance between two elements in the scene, and the order in which the multiple training elements are added to the scene. The training elements include at least one built element and / or at least one element in the N elements to be built.

[0290] The training module 920 is used to train the recommendation model using the training data. The recommendation model is used to obtain N elements to be built at the first position according to the first position specified by the user in the scene building page and the information of at least one built element in the scene building page. The information of the built element includes one or more of the second position of the built element in the scene building page, the category of the built element, and the name of the built element, where N is a positive integer greater than or equal to 1.

[0291] In a possible scenario, the recommendation model training device 900 may deploy the trained recommendation model to a terminal-side device, such as the second processing device 600 described above.

[0292] To further achieve the above Figures 3 to 5 The present application also provides a training device for a recommendation model, such as Fig.10 As shown, Fig.10 Schematic diagram of the structure of the training device for the recommended model provided in this application Figure 2 The training device 900 of the recommendation model also includes an enhanced training module 930.

[0293] Among them, the enhanced training module 930 is used to obtain the second position specified by the user in the scene construction page, and use the recommendation model to infer at least one element to be built in the second position according to the second position. Obtain the user's feedback on the at least one element to be built in the second position, the feedback includes positive feedback or negative feedback, and train the recommendation model based on the positive feedback or negative feedback.

[0294] Combination of the above Figure 3 , Figures 6 to 8 The scene construction method provided by this embodiment is described in detail. Fig.11A , describing the scene building device provided according to this embodiment.

[0295] Fig.11A Structural diagram of the scene building device provided for this application Figure 1 The scene building device can be used to implement the functions of the execution device or the second processing device 600 in the above method embodiment, and thus can also achieve the beneficial effects of the above method embodiment. In this embodiment, the scene building device can be as follows Figure 3 The execution device 310 shown, Figure 6 The second processing device 600 in the embodiment may also be a module (such as a chip) applied to a server.

[0296] The scene construction device 1100 includes a providing module 1110, an acquiring module 1120, a first responding module 1130, and a second responding module 1140. The scene construction device 1100 is used to implement the above Figures 6 to 8The method embodiments shown in FIG. 1 are functions of a computing device.

[0297] A module 1110 is provided, which is used to provide a scene construction page, where the scene in the scene construction page includes at least one constructed element.

[0298] The acquisition module 1120 is used to acquire the first position specified by the user in the scene building page.

[0299] The first response module 1130 is used to display N elements to be built in the scene building page in response to the specified first position and information of at least one built element, the information of the built elements includes one or more of the second position of the built element on the scene building page, the category of the built element and the name of the built element, and N is a positive integer greater than or equal to 1.

[0300] The second response module 1140 is used to deploy the at least one element at a first position in the scene building page in response to the user's operation on at least one element among the N elements to be built in the scene building page.

[0301] In a possible scenario, the scene construction device 1100 may send the obtained elements to be constructed to the terminal device, or display the elements to be constructed on a display device corresponding to the computing device.

[0302] To further achieve the above Figure 3 , Figures 6 to 8 The present application also provides a scene building device, such as Fig. 11B As shown, Fig. 11B Structural diagram of the scene building device provided for this application Figure 2 The scene construction device 1100 further includes: a receiving module 1150 , a training module 1160 and a reinforcement learning module 1170 .

[0303] The receiving module 1150 is used to receive a function activation instruction sent by a user, where the function activation instruction is used to activate the function of recommending elements to be constructed.

[0304] The training module 1160 is used to train the recommendation model using training data, where the training data includes one or more of the categories of multiple training elements, the distance between two elements of the multiple training elements in the scene, and the order in which the multiple training elements are added to the scene, and the training elements include at least one built element and / or at least one element of N elements to be built.

[0305] The reinforcement learning module 1170 is used to obtain the second position specified by the user in the scene construction page, and use the recommendation model to infer at least one element to be built in the second position based on the second position; obtain the user's feedback on the at least one element to be built in the second position, the feedback includes positive feedback or negative feedback; and train the recommendation model based on the positive feedback or negative feedback.

[0306] It is worth noting that the providing module 1110, the acquiring module 1120, the first response module 1130, and the second response module 1140 can all be implemented by software or by hardware. Exemplarily, the implementation of the providing module 1110 is described below by taking the providing module 1110 as an example. Similarly, the implementation of the acquiring module 1120, the first response module 1130, and the second response module 1140 can refer to the implementation of the providing module 1110.

[0307] As an example of a software functional unit, the module 1110 provided may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the module 1110 provided may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with close geographical locations. Among them, usually a region may include multiple AZs.

[0308] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0309] As an example of a hardware functional unit, the providing module 1110 may include at least one computing device, such as a server, etc. Alternatively, the providing module 1110 may also be a device implemented by using an ASIC or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), an FPGA, a generic array logic (GAL), or any combination thereof.

[0310] The multiple computing devices included in the providing module 1110 may be distributed in the same region or in different regions. The multiple computing devices included in the providing module 1110 may be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the providing module 1110 may be distributed in the same VPC or in multiple VPCs. The multiple computing devices may be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0311] It should be noted that, in other embodiments, the providing module 1110 can be used to execute any step in the scene building method, the acquiring module 1120 can be used to execute any step in the scene building method, the first response module 1130 can be used to execute any step in the scene building method, and the second response module 1140 can be used to execute any step in the scene building method. The steps that the providing module 1110, the acquiring module 1120, the first response module 1130, and the second response module 1140 are responsible for implementing can be specified as needed, and the full functions of the scene building device are realized by respectively implementing different steps in the scene building method through the providing module 1110, the acquiring module 1120, the first response module 1130, and the second response module 1140.

[0312] for Fig. 9 The software or hardware implementation of each module in the training device 900 of the recommendation model shown can refer to the description of module 1110 provided in the above-mentioned scene building device 1100, and will not be repeated here.

[0313] The embodiment of the present application also provides a chip system, which includes a control circuit and an interface circuit. The interface circuit is used to provide a scene building page, and the interface circuit and the control circuit cooperate to implement the function of the execution device 320 or the second processing device 600 in the above method. And / or, the interface circuit is used to obtain training data, and the interface circuit and the control circuit cooperate to implement the function of the training device 310 or the first processing device 400 in the above method.

[0314] In a possible design, the chip system further includes a memory for storing program instructions and / or data. The chip system may be composed of a chip, or may include a chip and other discrete devices.

[0315] The present application also provides a computing device, please refer to Fig.12 , Fig.12 A schematic diagram of the structure of a computing device provided in the present application. The computing device 1200 includes a bus 1202, a processor 1204, a memory 1206, and a communication interface 1208. The processor 1204, the memory 1206, and the communication interface 1208 are connected to each other through the bus 1202. The computing device 1200 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 1200.

[0316] The bus 1202 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.12 The bus 1204 may include a path for transmitting information between various components of the computing device 1200 (eg, the processor 1204, the memory 1206, and the communication interface 1208).

[0317] The processor 1204 may include any one or more processors such as a CPU, a GPU, a microprocessor (MP) or a DSP.

[0318] The memory 1206 may include a volatile memory, such as a random access memory (RAM). The processor 1204 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0319] The memory 1206 stores executable program codes, and the processor 104 executes the executable program codes to respectively implement the functions of the providing module 1110, the acquiring module 1120, the first responding module 1130, and the second responding module 1140, thereby implementing the scene building method. That is, the memory 106 stores instructions for executing the scene building method.

[0320] Alternatively, the memory 1206 stores executable program codes, and the processor 104 executes the executable program codes to respectively implement the functions of the acquisition module 910 and the training module 920, thereby implementing the training method of the recommendation model. That is, the memory 106 stores instructions for executing the training method of the recommendation model.

[0321] The communication interface 1208 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1200 and other devices or communication networks. The computing device 1200 may be a computer (e.g., a server) in a cloud data center, or a computer in an edge data center, or a terminal.

[0322] For the functions of the training device 320 and the functions of the execution device 310 deployed in the same computing device 1200 , the training device 320 can communicate with the execution device 312 through the bus 1202 .

[0323] For the functions of the training device 320 and the functions of the execution device 310 deployed in different computing devices 1200, the training device 320 can communicate with the execution device 310 through a communication network.

[0324] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device, which can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0325] like Fig.13 As shown, Fig.13 A schematic diagram of the structure of a computing device cluster provided in the present application. The computing device cluster includes at least one computing device 1200. The memory 1206 in one or more computing devices 1200 in the computing device cluster may store the same instructions for executing the scene building method.

[0326] In some possible implementations, the memory 1206 of one or more computing devices 1200 in the computing device cluster may also store partial instructions for executing the scene building method. In other words, the combination of one or more computing devices 1200 can jointly execute instructions for executing the scene building method.

[0327] It should be noted that the memory 1206 in different computing devices 1200 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the scene building system. That is, the instructions stored in the memory 1206 in different computing devices 1200 can implement the functions of one or more modules of the providing module 1110, the obtaining module 1120, the first response module 1130, and the second response module 1140.

[0328] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.14 A possible implementation is shown. Fig.14 As shown, Fig.14 A schematic diagram of a connection between computing devices provided in the present application, wherein two computing devices 1200A and 1200B are connected via a network. Specifically, the connection is made to the network via a communication interface in each computing device. In this type of possible implementation, the memory 1206 in the computing device 1200A stores instructions for executing the functions of the providing module 1110 and the acquiring module 1120. At the same time, the memory 1206 in the computing device 1200B stores instructions for executing the functions of the first responding module 1130 and the second responding module 1140.

[0329] It should be understood that Fig.14 The functions of the computing device 1200A shown in FIG. 1200A may also be completed by multiple computing devices 1200. Similarly, the functions of the computing device 1200B may also be completed by multiple computing devices 1200.

[0330] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be a software or program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the above-mentioned scenario building method or recommendation model training method.

[0331] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute a scene building method or a training method for a recommendation model.

[0332] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc (DVD); it may also be a semiconductor medium, for example, a solid state drive (SSD).

[0333] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A scene building method, characterized in that: The method is applied to a scene building device, and the method comprises: Providing a scene building page, wherein the scene in the scene building page includes at least one built element; Obtaining the first position specified by the user in the scene building page; In response to the designated first position and the information of the at least one constructed element, N elements to be constructed are displayed in the scene construction page, the information of the constructed elements including one or more of the second position of the constructed element in the scene construction page, the category of the constructed element and the name of the constructed element, where N is a positive integer greater than or equal to 1; In response to the user's operation on at least one of the N elements to be built in the scene building page, the at least one element to be built is deployed in the first position in the scene building page.

2. The method according to claim 1, characterized in that: In response to the designated first position and the information of the at least one constructed element, displaying N elements to be constructed in the scene construction page includes: Calculating the distance between the second position of the constructed element and the first position, and determining, from the scene construction page, some of the constructed elements whose distance to the first position is less than or equal to the target distance; Determining the N elements to be constructed according to the information of the part of the constructed elements; The N elements to be constructed are displayed in the scene construction page.

3. The method according to claim 1, characterized in that The scene construction device includes an element library, the element library includes a plurality of elements, and in response to the specified first position and the information of the at least one constructed element, N elements to be constructed are displayed in the scene construction page, including: Determining target information of the element to be built according to the information of the built element, wherein the target information of the element to be built includes a target category of the element to be built and / or a target name of the element to be built; Calculating a first similarity between the categories of the multiple elements in the element library and the target categories of the elements to be built, or calculating a second similarity between the names of the multiple elements in the element library and the target names of the elements to be built, and determining the N elements to be built that are ranked higher in the first similarity or the second similarity; The N elements to be constructed are displayed.

4. The method according to any one of claims 1 to 3, characterized in that The information of the constructed elements also includes the order in which the constructed elements are added to the scene construction page.

5. The method according to any one of claims 1 to 4, characterized in that: Before obtaining the first position specified by the user in the scene building page, the method further includes: A function activation instruction sent by a user is received, where the function activation instruction is used to activate a function of recommending the element to be constructed.

6. The method according to any one of claims 1 to 5, characterized in that: The built elements and the elements to be built include one or more of objects, maps, materials, lights or textures.

7. The method according to any one of claims 1 to 6, characterized in that: In response to the designated first position and the information of the at least one constructed element, displaying N elements to be constructed in the scene construction page includes: In response to the specified first position and the information of the at least one constructed element, determining N elements to be constructed using a recommendation model; Displaying N elements to be built in the scene building page; The method further comprises: The recommendation model is trained using training data, wherein the training data includes one or more of categories of multiple training elements, distances between two training elements of the multiple training elements in a scene, and an order in which the multiple training elements are added to the scene, and the training elements include the at least one built element and / or the at least one element of the N elements to be built.

8. The method according to claim 7, characterized in that The method further comprises: Obtaining a second position specified by the user in the scene building page; According to the second position, using the recommendation model to infer at least one element to be built in the second position; Obtaining user feedback on at least one element to be constructed at the second position, wherein the feedback includes positive feedback or negative feedback; The recommendation model is trained based on the positive feedback or the negative feedback.

9. A scene building device, characterized in that: The device comprises: A module is provided, for providing a scene building page, wherein the scene in the scene building page includes at least one built element; An acquisition module, used to acquire a first position specified by a user in the scene building page; A first response module, configured to display N elements to be constructed in the scene construction page in response to the specified first position and the information of the at least one constructed element, wherein the information of the constructed element includes one or more of the second position of the constructed element in the scene construction page, the category of the constructed element, and the name of the constructed element, where N is a positive integer greater than or equal to 1; The second response module is used to deploy the at least one to-be-built element in the first position in the scene building page in response to the user's operation on at least one to-be-built element among the N to-be-built elements in the scene building page.

10. The device according to claim 9, characterized in that The first response module is specifically used to calculate the distance between the second position of the built element and the first position, determine from the scene construction page the part of the built elements whose distance to the first position is less than the target distance, and determine the N elements to be built based on the information of the part of the built elements, and display the N elements to be built in the scene construction page.

11. The device according to claim 9, characterized in that The scene building device includes an element library, which includes multiple elements; the first response module is specifically used to determine the target information of the element to be built based on the information of the built elements, the target information of the element to be built includes the target category of the element to be built and / or the target name of the element to be built, and calculate the first similarity between the category of the multiple elements in the element library and the target category of the element to be built, or calculate the second similarity between the name of the multiple elements in the element library and the target name of the element to be built, determine the N elements to be built that are ranked high in the first similarity or the second similarity; and display the N elements to be built.

12. The device according to any one of claims 9 to 11, characterized in that The information of the constructed elements also includes the order in which the constructed elements are added to the scene construction page.

13. The device according to any one of claims 9 to 12, characterized in that The scene building device further includes: The receiving module is used to receive a function activation instruction sent by a user, where the function activation instruction is used to activate the function of recommending the element to be built.

14. The device according to any one of claims 9 to 13, characterized in that The built elements and the elements to be built include one or more of objects, maps, materials, lights or textures.

15. The device according to any one of claims 9 to 14, characterized in that The first response module is further specifically configured to determine N elements to be constructed using a recommendation model in response to the specified first position and the information of the at least one constructed element; Displaying N elements to be built in the scene building page; The scene building device further includes: A training module is used to train the recommendation model using training data, wherein the training data includes one or more of categories of multiple training elements, a distance between two elements of the multiple training elements in a scene, and an order in which the multiple training elements are added to the scene, and the training elements include the at least one constructed element and / or the at least one element of the N elements to be constructed.

16. The device according to claim 15, characterized in that The scene building device further includes: A reinforcement learning module is used to obtain a second position specified by a user in the scene construction page, and based on the second position, use the recommendation model to infer at least one element to be built in the second position; obtain user feedback on the at least one element to be built in the second position, wherein the feedback includes positive feedback or negative feedback; and train the recommendation model based on the positive feedback or the negative feedback.

17. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 8.

18. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 8.

19. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 8.

Images