A furniture layout method, device and medium based on proximal strategy optimization

Abstract

The present invention discloses a furniture layout method, device and medium based on proximal strategy optimization, including: collecting historical design data, traversing the association information between each furniture object from the historical design data; then traversing the object feature vector of each furniture in the historical design data, and constructing a solution database for furniture layout; then constructing a deep reinforcement network; traversing the solution database again, and training the deep reinforcement network according to the solution database; finally, calculating the apartment feature vector according to the structure of the apartment to be laid out in the target scene, matching candidate similar apartment types from the solution database, and then iterating the furniture layout according to the deep reinforcement network to obtain the final furniture layout solution. The present invention can improve the efficiency of furniture layout design and increase the diversity of furniture layout, and can be widely used in the field of computer technology.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a furniture layout method, device and medium based on proximal strategy optimization. Background Art

[0002] Three-dimensional interior furniture layout is a task that requires placing appropriate furniture in reasonable positions within a given room structure (walls, beams, columns, doors, windows, etc.) and room type. It is widely used in computer-aided design, especially in three-dimensional interior design scenarios.

[0003] The mainstream 3D indoor furniture layout methods are mainly based on three categories. The first category is mainly based on artificially defined spatial design rules. This type of method mostly adopts an iterative optimization method. First, the furniture is initialized to the appropriate room and position according to the set furniture placement rules, and then the furniture position is adjusted to obtain a better furniture layout and display the furniture layout results; however, formulating clearly defined rules is very time-consuming and requires a lot of effort from skilled and experienced designers. At the same time, the predefined design rules can only show simple and intuitive scene composition patterns and cannot express all possible scenes.

[0004] In order to capture more complex scene arrangement patterns for synthesizing different scenes, the second type of methods resort to deep generative models to learn scene priors from large-scale datasets. This type of method leverages the capabilities of generative models such as GAN and VAE to provide better pattern coverage for generating diversity, but the results produced are often unsatisfactory.

[0005] The above methods still use the results of the dataset as experience and cannot truly capture more complex scene combinations. Reinforcement learning-based methods attempt to solve this problem. The third type of method combines reinforcement learning methods to replace the supervised interaction of human designers with the interaction of agents in the designed simulation environment.

[0006] The methods of the related prior art have the following disadvantages:

[0007] 1. The rule-based layout method seriously violates the artificial spatial design rules, which is time-consuming and labor-intensive;

[0008] 2. Existing deep learning-based layout methods rely heavily on the placement of data sets and lack diversity;

[0009] 3. The furniture layout method based on reinforcement learning requires artificial layout rules to provide state rewards and penalties, which has the same problem as disadvantage 1. Summary of the invention

[0010] In view of this, the embodiments of the present invention provide an efficient furniture layout method, device and medium based on proximal strategy optimization, which can improve the diversity of furniture layout.

[0011] An aspect of an embodiment of the present invention provides a furniture layout method based on proximal strategy optimization, comprising:

[0012] Collecting historical design data, and traversing association information between furniture objects from the historical design data;

[0013] Traversing the object feature vectors of each piece of furniture in the historical design data to construct a furniture layout solution database;

[0014] Construct a deep reinforcement network that can simultaneously output discrete selection actions, continuous image processing intensity actions, and evaluation value functions;

[0015] Traversing the solution database, and training the deep reinforcement network according to the solution database, wherein a reward function of the deep reinforcement network is constructed according to the association information;

[0016] The apartment feature vector is calculated according to the structure of the apartment to be arranged in the target scenario, candidate similar apartment types are matched from the solution database, and then the furniture layout is iterated according to the deep reinforcement network to obtain the final furniture layout solution.

[0017] Optionally, the collecting of historical design data and traversing association information between furniture objects from the historical design data include:

[0018] Collect historical design proposals from multiple designers;

[0019] Each historical design scheme is traversed one by one. For all spaces in each historical design scheme, the following steps are performed to calculate the association probability of each historical design scheme:

[0020] Rotate the polygons of the apartment structure in the same direction as the entrance door, and all other objects are rotated using the same rotation matrix;

[0021] Taking the minimum 3D point value of all objects in the entire space as the starting point, the object position is positively processed to obtain the size of the 3D bounding box of the object including the wall and furniture, the center point position, and the object category;

[0022] All numerical data are processed in centimeters and are denoted as O i =(W i ,D i ,H i ,X i ,Y i ,Z i ,Ci ); where O i Represents an object; W i Represents the width of the object; D i Represents the depth of the object; H i Represents the height of the object; X i Represents the starting X-axis coordinate of the object; Y i Represents the starting Y-axis coordinate of the object; Z i Represents the starting Z-axis coordinate of the object; C i Represents the object category;

[0023] Define the sticking threshold constant θ1 = 10, and traverse all the pairwise combinations of objects (O i ,O j ), make a close judgment, and count (C i ,C j ) The probability of sticking to each other, record The sticking judgment formula is:

[0024] (|min(X i +W i ,X j +W j )-max(X i ,X j )|≤θ1)||(|min(Y i +D i ,Y j +D j )-max(Y i ,Y j )|≤θ1)

[0025] Define the alignment threshold constant θ2 = 10, and traverse all the pairwise combinations of objects (O i ,O j ), make alignment judgment, and count (C i ,C j ) is the alignment probability of the pair, The alignment judgment formula is:

[0026]

[0027] Define a symmetric threshold constant θ3 = 10, and traverse all three-by-three combinations of objects (O i ,O j ,O k ), make symmetry judgment, and count (C i ,C j ,C k ) is the symmetric probability of The symmetry judgment formula is:

[0028] (||Xi -X k |-|X j -X k ||≤θ3)||(||Y i -Y k |-|Y j -Y k ||≤θ3)

[0029] Define a symmetric threshold constant θ3 = 10, and traverse all three-by-three combinations of objects (O i ,O j ,O k ), make symmetry judgment, and count (C i ,C j ,C k ) is the symmetric probability of The symmetry judgment formula is:

[0030] (||X i -X k |-|X j -X k ||≤θ2)||(||Y i -Y k |-|Y j -Y k ||≤θ3)

[0031] After completing the calculation of the association probability of each historical design solution, traverse and retain all results with a probability of abutting greater than or equal to 30%, traverse and retain all results with a probability of alignment greater than or equal to 30%, and traverse and retain all results with a probability of symmetry greater than or equal to 30%;

[0032] According to the sticking probability, the alignment probability and the symmetry probability, define For category c i ,c j The probability of sticking between For category c i ,c j The alignment probability between For category c i ,c j ,c k The symmetric probability between .

[0033] Optionally, traversing the object feature vectors of each piece of furniture in the historical design data to construct a furniture layout solution database includes:

[0034] Traverse each historical design scheme, and for all spaces in each historical design scheme, perform the following steps:

[0035] Taking the minimum 3D point value of all objects in the entire space as the starting point, the position of the object is positively processed;

[0036] Obtain the apartment polygon point set according to the apartment structure, and calculate the apartment feature vector with a length of 128 according to the turning function algorithm;

[0037] Record the space type of the space;

[0038] Record the furniture layout data of the space, obtain the size of the three-dimensional bounding box of the object including the wall and furniture, the center point position, the object category, and the rotation angle. All numerical data are processed in centimeters and recorded as

[0039] The solution database (TF, G, A) is obtained, where TF is the apartment feature vector of all spaces, G is the space type of all spaces, and A is the furniture layout feature corresponding to the space.

[0040] Optionally, in the step of constructing a deep reinforcement network that can simultaneously output discrete selection actions, continuous image processing intensity actions, and evaluation value functions:

[0041] The discrete action selection network includes three MLP perception layers, TANH activation is used between the perception layers, and SIGMOID activation is used for the final output;

[0042] The continuous parameter action network includes three MLP perception layers, TANH activation is used between the perception layers, and TANH activation is used for the final output;

[0043] The evaluation value function network includes three MLP perception layers, TANH activation is selected between the perception layers, and the final output does not select the activation function.

[0044] Optionally, in the discrete selection action network, the first MLP perception layer inputs a vector dimension of 1024 and outputs a dimension of 64; the second MLP perception layer inputs a vector dimension of 64 and outputs a dimension of 64; the third MLP perception layer inputs a vector dimension of 64 and outputs a dimension of 61; wherein 60 refers to the movement of 20 pieces of furniture in 3 directions, and the additional 1 dimension represents a "stop" action;

[0045] In the continuous parameter action network, the first MLP perception layer inputs a vector dimension of 1024 and outputs a dimension of 64; the second MLP perception layer inputs a vector dimension of 64 and outputs a dimension of 64; the third MLP perception layer inputs a vector dimension of 64 and outputs a dimension of 60; where 60 refers to the moving distances of the 20 pieces of furniture in three directions, and positive and negative values ​​represent the moving distances in the positive and reverse directions of the corresponding coordinate axes, respectively;

[0046] In the evaluation value function network, the input vector dimension of the first MLP perception layer is 1024, and the output dimension is 64; the input vector dimension of the second MLP perception layer is 64, and the output dimension is 64; the input vector dimension of the third MLP perception layer is 64, and the output dimension is 1; wherein the final output 1 dimension represents the estimated value of the current furniture layout.

[0047] Optionally, traversing the solution database and training the deep reinforcement network according to the solution database includes:

[0048] The solution database is grouped according to space type;

[0049] In one round of training, we traverse 10 space types, and for each space type, we randomly repeat 8 times, and each time we use 8 furniture layout feature vectors as the input of the model;

[0050] According to the network input, obtain the output result of the deep reinforcement network;

[0051] The furniture layout features are input into the discrete selection action network, and the estimated selection action probability is obtained according to the multi-classification selection action probability sampling;

[0052] The furniture layout features are input into the continuous parameter action network, and the estimated parameter action mean is obtained by sampling the normal distribution constructed by the action mean and action variance;

[0053] Inputting the furniture layout features into the evaluation value function network, and obtaining an estimated value evaluation score after normalizing the value evaluation score;

[0054] For the reward score of the deep reinforcement network: calculate the collision penalty, out-of-bounds penalty, door and window penalty, association reward, and total reward score in turn;

[0055] Based on the actions, value scores, and reward scores that have been constructed, the training of the deep reinforcement network is completed in combination with proximal strategy optimization;

[0056] The collision penalty formula is:

[0057]

[0058] Where S is the area calculation, |O i ∩O l | is the intersection between two objects, N is the number of objects in a certain spatial furniture layout;

[0059] The calculation formula for the out-of-bounds penalty is:

[0060]

[0061] Where S is the area calculation, |O i ∩O l | is the intersection between two objects, N is the number of objects in a certain spatial furniture layout, ρ 2 (O i ,O l ) is the center point distance between the two objects, c 2 (O i ,O l ) is the maximum diagonal distance between two objects;

[0062] The calculation formula for door and window penalties is:

[0063]

[0064] Where S is the area calculation, is the intersection of two objects on the y-axis and z-axis, N is the number of objects in a certain spatial furniture layout, c 2 (O i ,O l ) is the maximum diagonal distance between two objects;

[0065] The calculation formula for association rewards is:

[0066]

[0067] The total reward score is calculated as follows:

[0068] Optionally, the step of calculating a housing type feature vector according to the structure of the housing type to be arranged in the target scenario, matching candidate similar housing types from the solution database, and then iterating the furniture layout according to the deep reinforcement network to obtain a final furniture layout solution includes:

[0069] Take the apartment type that needs to complete the furniture layout as input;

[0070] Rotate the polygons of the apartment structure in the same direction as the entrance door, and all other objects are rotated using the same rotation matrix;

[0071] Taking the minimum 3D point value of all objects in the entire space as the starting point, the position of the object is positively processed;

[0072] The unit feature vector with a length of 128 is calculated according to the turning function algorithm, and is denoted by TF l ;

[0073] TF l Calculate the cosine similarity with the solution library TF and get the most similar solution A l;

[0074] A l As the input of the trained deep reinforcement network, perform network reasoning until the discrete selection action is "stop", end the reasoning, and get the final furniture layout state Complete the furniture layout.

[0075] Another aspect of the embodiment of the present invention further provides a furniture layout device based on proximal strategy optimization, comprising:

[0076] A collection module, used to collect historical design data, and traverse the association information between various furniture objects from the historical design data;

[0077] A database construction module, used for traversing the object feature vectors of each piece of furniture in the historical design data to construct a furniture layout solution database;

[0078] A network building module for building deep reinforcement networks that can simultaneously output discrete selection actions, continuous image processing intensity actions, and evaluation value functions;

[0079] A training module, configured to traverse the solution database and train the deep reinforcement network according to the solution database, wherein a reward function of the deep reinforcement network is constructed according to the association information;

[0080] The layout design module is used to calculate the apartment feature vector according to the structure of the apartment to be laid out in the target scenario, match candidate similar apartment types from the solution database, and then iterate the furniture layout according to the deep reinforcement network to obtain the final furniture layout solution.

[0081] Another aspect of an embodiment of the present invention further provides an electronic device, including a processor and a memory;

[0082] The memory is used to store programs;

[0083] The processor executes the program to implement the method described above.

[0084] Another aspect of the embodiments of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method described above.

[0085] The embodiment of the present invention also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.

[0086] The embodiment of the present invention first collects historical design data, and traverses the association information between each furniture object from the historical design data; then traverses the object feature vectors of each furniture in the historical design data to construct a solution database for furniture layout; then constructs a deep reinforcement network that can simultaneously output discrete selection actions, continuous image processing intensity actions, and evaluation value functions; then traverses the solution database, and trains the deep reinforcement network according to the solution database, wherein the reward function of the deep reinforcement network is constructed according to the association information; finally, the apartment feature vector is calculated according to the structure of the apartment to be laid out in the target scenario, and candidate similar apartment types are matched from the solution database, and then the furniture layout is iterated according to the deep reinforcement network to obtain the final furniture layout solution. The present invention can improve the efficiency of furniture layout design and increase the diversity of furniture layout. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0088] Figure 1 An overall step flow chart provided for an embodiment of the present invention;

[0089] Figure 2 A schematic diagram of data acquisition of a furniture layout method provided by an embodiment of the present invention;

[0090] Figure 3 A schematic diagram of the training process of a deep reinforcement network provided by an embodiment of the present invention;

[0091] Figure 4 A schematic diagram of the reasoning process of the furniture layout method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0092] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0093] In response to the problems existing in existing calculations, the present invention adopts an advanced proximal strategy to optimize the reinforcement learning method. On a deep reinforcement network, it processes both discrete actions and continuous actions. In terms of rewards, it provides placement rewards with the help of a data-driven method.

[0094] Specifically, Figure 1As shown, the method of the present invention comprises the following steps:

[0095] 1. Collect historical design data, traverse the design data, traverse the associations between furniture objects including walls, including sticking, alignment, and symmetry, and count the probabilities of various associations between each type of object.

[0096] 2. Traverse the historical design plan data, rotate the polygons of the apartment structure according to the unified entrance door direction, and solve the equal-length apartment feature vectors of each apartment structure polygon. For each wall and furniture object, encode it into an equal-length object feature vector according to the three-dimensional size, three-dimensional positioning center point, three-dimensional rotation angle, and object category; form a plan database that associates space types, space feature vectors, and furniture layout feature vectors.

[0097] 3. Construct a deep reinforcement network that can simultaneously output discrete image processing selection actions, continuous image processing intensity actions, and evaluation value functions.

[0098] 4. Traverse the solution database and input the object feature vector into the network constructed in step 3. According to the proximal strategy optimization, complete the training of the deep reinforcement network. The association reward, object collision penalty, furniture out-of-bounds (house type) penalty, and door and window occlusion penalty solved by the association probability and object position relationship in step 1 are used as the reward function in reinforcement learning training.

[0099] 5. In downstream applications, the apartment structure polygon to be laid out is used to calculate the apartment feature vector, and the similarity is solved with the apartment database in step 2 to find similar apartments. The corresponding furniture layout plan is used as the initial input, and the object feature vector is input into the deep reinforcement network in step 3. The furniture layout is iteratively adjusted until the deep reinforcement learning network outputs a "stop" action to obtain the final furniture layout plan.

[0100] The first step is to collect historical design data, traverse the design data, and traverse the associations between furniture objects including walls. The associations include sticking, alignment, and symmetry. The schematic diagrams of each association are as follows: Figure 2 As shown, the embodiment of the present invention counts the probabilities of various association relationships between each type of objects:

[0101] 1) First, collect the historical design data of all designers.

[0102] 2) Then, traverse all solutions, traverse all spaces in the solution at the second level, and perform the following steps:

[0103] a. Rotate the polygons of the apartment structure in the direction of the entrance door, and rotate all other objects according to the same rotation matrix;

[0104] b. Taking the minimum three-dimensional point value of all objects in the entire space as the starting point, the position of the object is positively processed;

[0105] c. Get the size of the three-dimensional bounding box, the center point position, and the object category of the object including the wall (excluding doors and windows) and furniture. All numerical data are processed in centimeters. Discrete column data is not processed here and is recorded as O i =(W i ,D i ,H i ,X i ,Y i ,Z i ,C i );

[0106] d. Then, define the sticking threshold constant θ1 = 10, and traverse all the pairwise combinations of objects (O i ,O j ), make a close judgment, and count (C i ,C j ) The probability of sticking to each other, record The sticking judgment formula is:

[0107] (|min(X i +W i ,X j +W j )-max(X i ,X j )|≤θ1)||(|min(Y i +D i ,Y j +D j )-max(Y i ,Y j )|≤θ1)

[0108] e. Then, define the alignment threshold constant θ2 = 10 and traverse all the pairwise combinations of objects (O i ,O j ), make alignment judgment, and count (C i ,C j ) is the alignment probability of the pair, The alignment judgment formula is:

[0109]

[0110] f. Then, define the symmetric threshold constant θ3 = 10, and traverse all the three-by-three combinations of objects (O i ,O j ,O k ), make symmetry judgment, and count (C i ,C j ,Ck ) is the symmetric probability of The symmetry judgment formula is:

[0111] (||X i -X k |-|X j -X k ||≤θ3)||(||Y i -Y k |-|Y j -Y k ||≤θ3)

[0112] 3) Then, traverse Keep all the results with probability greater than or equal to 30%, and traverse Keep all the results with probability greater than or equal to 30%, and traverse Keep all results with a probability greater than or equal to 30%.

[0113] 4) Finally, we get three association probabilities and define For category c i ,c j The probability of sticking between For category c i ,c j The alignment probability between For category c i ,c j ,c k The symmetric probability between .

[0114] The second step is to traverse the historical design plan data, rotate the polygons of the apartment structure according to the unified entrance door direction, and solve the equal-length apartment feature vectors of each apartment structure polygon. For each wall and furniture object, encode it into an equal-length object feature vector according to the three-dimensional size, three-dimensional positioning center point, three-dimensional rotation angle, and object category; form a plan database that associates space type, space feature vector, and furniture layout feature vector:

[0115] 1) First, collect the historical design data of all designers;

[0116] 2) Then, traverse all solutions, traverse all spaces in the solution at the second level, and perform the following steps:

[0117] a. Take the minimum 3D point value of all objects in the entire space as the starting point and perform positive processing on the object position;

[0118] b. According to the apartment structure (walls, doors, windows, columns), obtain the apartment polygon point set, and calculate the apartment feature vector with a length of 128 according to the turning function algorithm;

[0119] c. Record the space type of the space;

[0120] d. Record the furniture layout data of the space, and obtain the size of the three-dimensional bounding box of the object including the wall (excluding doors and windows) and furniture, the center point position, object category, and rotation angle. All numerical data are processed in centimeters. Discrete column data is not processed here and is recorded as Among them, C i Encode in one-hot mode, and set the length to be the maximum number of furniture types in the dataset, 16. i The vector length of is fixed to 25;

[0121] 3) Finally, we get the solution database (TF, G, A), where TF is the apartment feature vector of all spaces, G is the space type of all spaces, A is the furniture layout feature corresponding to the space, and the maximum number of furniture in the space is 20, that is, S is a feature vector with a fixed length of 500.

[0122] A k =[F1,F2,…,F 20 ]

[0123] The third step is to build a deep reinforcement network that can simultaneously output discrete image processing selection actions, continuous image processing intensity actions, and evaluation value functions:

[0124] The discrete action selection network consists of three MLP perception layers, with TANH activation used between the perception layers and SIGMOID activation used for the final output:

[0125] 1) The input vector dimension of the first MLP perception layer is 1024, and the output dimension is 64;

[0126] 2) The input vector dimension of the second MLP perception layer is 64, and the output dimension is 64;

[0127] 3) The input vector dimension of the third MLP perception layer is 64, and the output dimension is 61;

[0128] Among them, 60 refers to the movement of 20 pieces of furniture in 3 directions, and the additional 1 dimension represents the "stop" action.

[0129] The continuous parameter action network includes three MLP perception layers, TANH activation is used between the perception layers, and TANH activation is used for the final output:

[0130] 1) The input vector dimension of the first MLP perception layer is 1024, and the output dimension is 64;

[0131] 2) The input vector dimension of the second MLP perception layer is 64, and the output dimension is 64;

[0132] 3) The input vector dimension of the third MLP perception layer is 64, and the output dimension is 60;

[0133] Among them, 60 refers to the moving distance of 20 pieces of furniture in three directions, and positive and negative values ​​represent the moving distance in the positive and negative directions of the corresponding coordinate axis, respectively.

[0134] The evaluation value function network includes three MLP perception layers, TANH activation is selected between the perception layers, and the final output does not select the activation function:

[0135] 1) The input vector dimension of the first MLP perception layer is 1024, and the output dimension is 64;

[0136] 2) The input vector dimension of the second MLP perception layer is 64, and the output dimension is 64;

[0137] 3) The input vector dimension of the third MLP perception layer is 64, and the output dimension is 1;

[0138] The final output dimension 1 represents the estimated value of the current furniture layout.

[0139] The fourth step is to traverse the solution database, input the object feature vector into the network constructed in step 3, and complete the training of the deep reinforcement network according to the proximal strategy optimization, wherein the association reward, object collision penalty, furniture crossing boundary (house type) penalty, and door and window occlusion penalty obtained by the association probability and object position relationship in step 1 are used as the reward function in the reinforcement learning training. The training process of the embodiment of the present invention is as follows: Figure 3 Shown:

[0140] 1) Group the solution database according to the spatial type;

[0141] 2) In one round of training, the space types are traversed 10 times. For each space type, it is randomly repeated 8 times, and 8 furniture layout feature vectors are used as the input of the model each time;

[0142] 3) Obtain the output of the deep reinforcement network based on the network input;

[0143] a. Then, the furniture layout features are input into the discrete selection action network described in the third step to obtain an estimated selection action probability;

[0144] b. At the same time, the furniture layout features are input into the continuous parameter action network described in the third step to obtain the estimated parameter action mean;

[0145] c. At the same time, the furniture layout features are input into the evaluation value function network described in the third step to obtain an estimated value evaluation score;

[0146] d. Then, the selected action is obtained by sampling the selected action probabilities of multiple categories;

[0147] e. At the same time, the parameter action is sampled from the normal distribution constructed by the action mean and action variance;

[0148] f. At the same time, the value assessment score is normalized to obtain:

[0149]

[0150] 4) For reward scores:

[0151] a. First, calculate the collision penalty, the formula is:

[0152]

[0153] Where S is the area calculation, |O i ∩O l | is the intersection between two objects, N is the number of objects in a certain spatial furniture layout;

[0154] b. Then, calculate the out-of-bounds penalty, the formula is:

[0155]

[0156] Where S is the area calculation, |O i ∩O l | is the intersection between two objects, N is the number of objects in a certain spatial furniture layout, ρ 2 (O i ,O l ) is the center point distance between the two objects, c 2 (O i ,O l ) is the maximum diagonal distance between two objects;

[0157] Then, the door and window penalty is calculated, and the formula is:

[0158]

[0159] Where S is the area calculation, is the intersection of two objects on the y-axis and z-axis, N is the number of objects in a certain spatial furniture layout, c 2 (O i ,O l ) is the maximum diagonal distance between two objects;

[0160] Then, calculate the associated reward, the formula is:

[0161]

[0162] The definition of each p is shown in the first step;

[0163] Finally, the total reward score is obtained, and the formula is:

[0164]

[0165] 5) Finally, based on the constructed actions, value scores, and reward scores, the training of the deep reinforcement network is completed in combination with proximal strategy optimization;

[0166] In the fifth step, in the downstream application, the apartment structure polygon to be arranged is calculated to obtain the apartment feature vector, and the similarity is solved with the apartment database in step 2 to find similar apartments, and the corresponding furniture layout plan is used as the initial input. The object feature vector is input into the deep reinforcement network in step 3, and the furniture layout is iteratively adjusted until the deep reinforcement learning network outputs a "stop" action to obtain the final furniture layout plan. The furniture layout reasoning process of the present invention is as follows: Figure 4 As shown:

[0167] 1) First, take the apartment type that needs to be furnished as input;

[0168] 2) Rotate the polygons of the apartment structure in the direction of the entrance door, and all other objects are rotated according to the same rotation matrix;

[0169] 3) Taking the minimum three-dimensional point value of all objects in the entire space as the starting point, the object position is positively processed;

[0170] 4) Calculate the apartment type feature vector with a length of 128 according to the turning function algorithm, and record it as TF l ;

[0171] 5) TF l Calculate the cosine similarity with the solution library TF and get the most similar solution A l ;

[0172] 6) A l As the input of the trained deep reinforcement network, perform network reasoning until the discrete selection action is "stop", end the reasoning, and get the final furniture layout state

[0173] 7) Complete the furniture layout.

[0174] In summary, the present invention has the following advantages:

[0175] By introducing reinforcement learning content including Monte Carlo concepts into the furniture layout process, taking the existing layout of all furniture as the state in reinforcement learning, and taking the fine-tuning of the furniture position as the action in reinforcement learning, an overall furniture layout adjustment method is formed; at the same time, the deep reinforcement learning reward function with collision penalty, out-of-bounds penalty, door and window penalty, and associated probability reward is combined, so that the network can learn the designer's historical design experience and eliminate the need for artificial rule acquisition, providing more possible results for furniture layout.

[0176] Another aspect of the embodiment of the present invention further provides a furniture layout device based on proximal strategy optimization, comprising:

[0177] A collection module, used to collect historical design data, and traverse the association information between various furniture objects from the historical design data;

[0178] A database construction module, used for traversing the object feature vectors of each piece of furniture in the historical design data to construct a furniture layout solution database;

[0179] A network building module for building deep reinforcement networks that can simultaneously output discrete selection actions, continuous image processing intensity actions, and evaluation value functions;

[0180] A training module, configured to traverse the solution database and train the deep reinforcement network according to the solution database, wherein a reward function of the deep reinforcement network is constructed according to the association information;

[0181] The layout design module is used to calculate the apartment feature vector according to the structure of the apartment to be laid out in the target scenario, match candidate similar apartment types from the solution database, and then iterate the furniture layout according to the deep reinforcement network to obtain the final furniture layout solution.

[0182] Another aspect of an embodiment of the present invention further provides an electronic device, including a processor and a memory;

[0183] The memory is used to store programs;

[0184] The processor executes the program to implement the method described above.

[0185] Another aspect of the embodiments of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method described above.

[0186] The embodiment of the present invention also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.

[0187] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0188] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0189] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0190] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0191] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0192] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0193] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0194] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0195] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A furniture layout method based on proximal strategy optimization, characterized in that: include: Collecting historical design data, and traversing association information between furniture objects from the historical design data; Traversing the object feature vectors of each piece of furniture in the historical design data to construct a furniture layout solution database; Construct a deep reinforcement network that can simultaneously output discrete selection actions, continuous image processing intensity actions, and evaluation value functions; Traversing the solution database, and training the deep reinforcement network according to the solution database, wherein a reward function of the deep reinforcement network is constructed according to the association information; Calculate the apartment feature vector according to the structure of the apartment to be arranged in the target scenario, match candidate similar apartment types from the solution database, and then iterate the furniture layout according to the deep reinforcement network to obtain the final furniture layout solution; The step of traversing the object feature vectors of each piece of furniture in the historical design data and constructing a furniture layout solution database includes: Traverse each historical design scheme, and for all spaces in each historical design scheme, perform the following steps: Taking the minimum 3D point value of all objects in the entire space as the starting point, the position of the object is positively processed; Obtain the apartment polygon point set according to the apartment structure, and calculate the apartment feature vector with a length of 128 according to the turning function algorithm; Record the space type of the space; Record the furniture layout data of the space, obtain the size of the three-dimensional bounding box of the object including the wall and furniture, the center point position, the object category, and the rotation angle. All numerical data are processed in centimeters and recorded as Among them, W i Represents the width of the object; D i Represents the depth of the object; H i Represents the height of the object; X i Represents the starting X-axis coordinate of the object; Y i Represents the starting Y-axis coordinate of the object; Z i Represents the starting Z-axis coordinate of the object; C i Represents the object category; Represents the three-dimensional rotation angle; Get the solution database (TF, G, A), where TF is the apartment feature vector of all spaces, G is the space type of all spaces, and A is the furniture layout feature corresponding to the space; In the step of constructing a deep reinforcement network that can simultaneously output discrete selection actions, continuous image processing intensity actions, and evaluation value functions: The discrete action selection network includes three MLP perception layers, TANH activation is used between the perception layers, and SIGMOID activation is used for the final output; The continuous parameter action network includes three MLP perception layers, TANH activation is used between the perception layers, and TANH activation is used for the final output; The evaluation value function network includes three MLP perception layers, TANH activation is selected between the perception layers, and the final output does not select the activation function.

2. The furniture layout method based on proximal strategy optimization according to claim 1, characterized in that: The collecting of historical design data and traversing association information between furniture objects from the historical design data include: Collect historical design proposals from multiple designers; Each historical design scheme is traversed one by one. For all spaces in each historical design scheme, the following steps are performed to calculate the association probability of each historical design scheme: Rotate the polygons of the apartment structure in the same direction as the entrance door, and all other objects are rotated using the same rotation matrix; Taking the minimum 3D point value of all objects in the entire space as the starting point, the object position is positively processed to obtain the size of the 3D bounding box of the object including the wall and furniture, the center point position, and the object category; All numerical data are processed in centimeters and are denoted as O i =(W i ,D i ,H ii ,X i ,Y i ,Z i ,C i ); where O i Represents an object; W i Represents the width of the object; D i Represents the depth of the object; H i Represents the height of the object; X i Represents the starting X-axis coordinate of the object; Y i Represents the starting Y-axis coordinate of the object; Z i Represents the starting Z-axis coordinate of the object; C i Represents the object category; Define the sticking threshold constant θ1 = 10, and traverse all the pairwise combinations of objects (O i ,O j ), make a close judgment, and count (C i ,C j ) The probability of sticking to each other, record The sticking judgment formula is: (|min(X i +W i ,X j +W j )-max(X i ,X j )|≤θ1)||(|min(Y i +D i ,Y j +D j )-max(Y i ,Y j )|≤θ1) Define the alignment threshold constant θ2 = 10, and traverse all the pairwise combinations of objects (O i ,O j ), make alignment judgment, and count (C i ,C j ) is the alignment probability of the pair, The alignment judgment formula is: Define a symmetric threshold constant θ3 = 10, and traverse all three-by-three combinations of objects (O i ,O j ,O k ), make symmetry judgment, and count (C i ,C j ,C k ) is the symmetric probability of The symmetry judgment formula is: (||X i -X k |-|X j -X k ||≤θ3)||(||And i -AND k |-|And j -AND k ||≤θ3) After completing the calculation of the association probability of each historical design solution, traverse and retain all results with a probability of abutting greater than or equal to 30%, traverse and retain all results with a probability of alignment greater than or equal to 30%, and traverse and retain all results with a probability of symmetry greater than or equal to 30%; According to the sticking probability, the alignment probability and the symmetry probability, define For category c i ,c j The probability of sticking between For category c i ,c j The alignment probability between For category c i ,c j ,c k The symmetric probability between .

3. The furniture layout method based on proximal strategy optimization according to claim 1, characterized in that: In the discrete selection action network, the first MLP perception layer inputs a vector dimension of 1024 and outputs a dimension of 64; the second MLP perception layer inputs a vector dimension of 64 and outputs a dimension of 64; the third MLP perception layer inputs a vector dimension of 64 and outputs a dimension of 61; where 60 refers to the movement of 20 pieces of furniture in 3 directions, and the additional 1 dimension represents the "stop" action; In the continuous parameter action network, the first MLP perception layer inputs a vector dimension of 1024 and outputs a dimension of 64; the second MLP perception layer inputs a vector dimension of 64 and outputs a dimension of 64; the third MLP perception layer inputs a vector dimension of 64 and outputs a dimension of 60; where 60 refers to the moving distances of the 20 pieces of furniture in three directions, and positive and negative values ​​represent the moving distances in the positive and reverse directions of the corresponding coordinate axes, respectively; In the evaluation value function network, the input vector dimension of the first MLP perception layer is 1024, and the output dimension is 64; the input vector dimension of the second MLP perception layer is 64, and the output dimension is 64; the input vector dimension of the third MLP perception layer is 64, and the output dimension is 1; wherein the final output 1 dimension represents the estimated value of the current furniture layout.

4. The furniture layout method based on proximal strategy optimization according to claim 2 is characterized in that: The traversing the solution database and training the deep reinforcement network according to the solution database includes: The solution database is grouped according to space type; In one round of training, we traverse 10 space types, and for each space type, we randomly repeat 8 times, and each time we use 8 furniture layout feature vectors as the input of the model; According to the network input, obtain the output result of the deep reinforcement network; The furniture layout features are input into the discrete selection action network, and the estimated selection action probability is obtained according to the multi-classification selection action probability sampling; The furniture layout features are input into the continuous parameter action network, and the estimated parameter action mean is obtained by sampling the normal distribution constructed by the action mean and action variance; Inputting the furniture layout features into the evaluation value function network, and obtaining an estimated value evaluation score after normalizing the value evaluation score; For the reward score of the deep reinforcement network: calculate the collision penalty, out-of-bounds penalty, door and window penalty, association reward, and total reward score in turn; Based on the actions, value scores, and reward scores that have been constructed, the training of the deep reinforcement network is completed in combination with proximal strategy optimization; Among them, the collision penalty The formula is: Where S is the area calculation, |O i ∩O l | is the intersection between two objects, N is the number of objects in a certain spatial furniture layout; Cross-border punishment The calculation formula is: Where S is the area calculation, |O i ∩O l | is the intersection between two objects, N is the number of objects in a certain spatial furniture layout, ρ 2 (O i ,O l ) is the center point distance between the two objects, c 2 (O i ,O l ) is the maximum diagonal distance between two objects; Door and Window Penalty The calculation formula is: Where S is the area calculation, is the intersection of two objects on the y-axis and z-axis, N is the number of objects in a certain spatial furniture layout, c 2 (O i ,O l ) is the maximum diagonal distance between two objects; The calculation formula for association rewards is: in, For category c i ,c l The probability of sticking between For category c i ,c l The alignment probability between For category c i ,c j ,c l The symmetric probability between The total reward score is calculated as follows:

5. The furniture layout method based on proximal strategy optimization according to claim 4 is characterized in that: The method calculates the apartment feature vector according to the apartment structure to be arranged in the target scenario, matches candidate similar apartment types from the solution database, and then iterates the furniture layout according to the deep reinforcement network to obtain the final furniture layout solution, including: Take the apartment type that needs to complete the furniture layout as input; Rotate the polygons of the apartment structure in the same direction as the entrance door, and all other objects are rotated using the same rotation matrix; Taking the minimum 3D point value of all objects in the entire space as the starting point, the position of the object is positively processed; The unit feature vector with a length of 128 is calculated according to the turning function algorithm, and is denoted by TF l ; TF l Calculate the cosine similarity with the solution library TF and get the most similar solution A l ; A l As the input of the trained deep reinforcement network, perform network reasoning until the discrete selection action is "stop", end the reasoning, and get the final furniture layout state Complete the furniture layout.

6. A furniture layout device based on proximal strategy optimization, characterized in that: include: A collection module, used to collect historical design data, and traverse the association information between various furniture objects from the historical design data; A database construction module, used for traversing the object feature vectors of each piece of furniture in the historical design data to construct a furniture layout solution database; A network building module for building deep reinforcement networks that can simultaneously output discrete selection actions, continuous image processing intensity actions, and evaluation value functions; A training module, configured to traverse the solution database and train the deep reinforcement network according to the solution database, wherein a reward function of the deep reinforcement network is constructed according to the association information; A layout design module is used to calculate a housing type feature vector according to the structure of the housing type to be arranged in the target scenario, match candidate similar housing types from the solution database, and then iterate the furniture layout according to the deep reinforcement network to obtain a final furniture layout solution; Wherein, the database construction module is specifically used for: Traverse each historical design scheme, and for all spaces in each historical design scheme, perform the following steps: Taking the minimum 3D point value of all objects in the entire space as the starting point, the position of the object is positively processed; Obtain the apartment polygon point set according to the apartment structure, and calculate the apartment feature vector with a length of 128 according to the turning function algorithm; Record the space type of the space; Record the furniture layout data of the space, obtain the size of the three-dimensional bounding box of the object including the wall and furniture, the center point position, the object category, and the rotation angle. All numerical data are processed in centimeters and recorded as Among them, W i Represents the width of the object; D i Represents the depth of the object; H i Represents the height of the object; X i Represents the starting X-axis coordinate of the object; Y i Represents the starting Y-axis coordinate of the object; Z i Represents the starting Z-axis coordinate of the object; C i Represents the object category; Represents the three-dimensional rotation angle; Get the solution database (TF, G, A), where TF is the apartment feature vector of all spaces, G is the space type of all spaces, and A is the furniture layout feature corresponding to the space; In this module, the network building module: The discrete action selection network includes three MLP perception layers, TANH activation is used between the perception layers, and SIGMOID activation is used for the final output; The continuous parameter action network includes three MLP perception layers, TANH activation is used between the perception layers, and TANH activation is used for the final output; The evaluation value function network includes three MLP perception layers, TANH activation is selected between the perception layers, and the final output does not select the activation function.

7. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The storage medium stores a program, and the program is executed by a processor to implement the method according to any one of claims 1 to 5.

Images