Layout rationality evaluation method and system based on graph attention network

Abstract

The present invention provides a layout rationality evaluation method based on a graph attention network. On the basis of an initialized 3D-SPAN network, a small-sample three-dimensional scene evaluation network based on a 3D-SPAN-F framework is trained. Then, a three-dimensional scene graph is constructed on the obtained indoor layout graph based on the trained small-sample three-dimensional scene evaluation network based on a three-dimensional scene layout rationality evaluation algorithm to generate a three-dimensional scene graph. The three-dimensional scene graph is evaluated to generate a binary evaluation result and a subjective logic evaluation result. The whole process fully taps the value of the constructed 3D-SPAD dataset, and can accurately explore the relationship between human subjective evaluation and scene complexity, scene category and object category, as well as the relationship between the overall scene evaluation and the evaluation of objects in the scene, thereby improving the accuracy of the indoor three-dimensional scene layout rationality evaluation algorithm.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and more particularly to a method and system for evaluating the rationality of indoor layouts, and more particularly to a method and system for evaluating the rationality of indoor layouts based on a graph attention network. Background Art

[0002] As part of the field of 3D scene understanding, the research tasks of indoor 3D scene understanding are still expanding with the deepening of research, such as indoor 3D scene risk assessment, identification of functional relationships between objects in indoor 3D scenes, and target-driven visual navigation in indoor 3D scenes.

[0003] Existing work evaluating the rationality of indoor 3D scene layouts mostly relies on direct subjective evaluation methods to provide solutions. However, this subjective evaluation method consumes a significant amount of human resources, is not reusable due to differences in experimental environments, and presents reliability issues. As for objective evaluation methods, traditional expert experience can be used. However, this method has limitations in application and expert opinions may not reflect objective reality. Another approach is to measure rationality by calculating the norm difference between the bounding boxes of the real scene and the generated scene. However, this method ignores the semantic properties of the scene and consumes a large amount of computational resources for traversal operations. Alternatively, ergonomic priors and statistical models can be used to define an energy function for human interaction with the indoor 3D scene environment and objects, using an energy field approach for quantitative evaluation. However, this method relies on probabilistic prior information and does not fully reflect human subjective cognition.

[0004] From the above analysis, we can see that the current indoor 3D scene layout rationality evaluation task still has much room for improvement in terms of practicality and reliability. The main difficulties are as follows:

[0005] Lack of datasets: According to research, there is currently a lack of public 3D scene datasets suitable for indoor 3D scene layout rationality evaluation tasks;

[0006] Difficulties in feature extraction and representation: Given the massive amount of 3D scene data and diverse raw features, effectively extracting layout information and uniformly representing it is a key issue.

[0007] Therefore, there is an urgent need for a method to improve the breadth and complexity of the dataset, utilize the fact that the computing mechanism of the graph attention network is similar to the human subjective evaluation logic, combine local enhancement with global construction, and construct a large number of three-dimensional scene datasets suitable for the evaluation of the rationality of indoor three-dimensional scene layouts, and improve the evaluation accuracy and effectiveness of the layout rationality evaluation method and system based on the graph attention network based on this dataset. Summary of the Invention

[0008] In view of the above problems, the purpose of the present invention is to provide a layout rationality evaluation method and system based on graph attention network to solve the problems of lack of data sets and inaccurate feature extraction and representation in the field of indoor three-dimensional scene layout rationality evaluation in the existing technology.

[0009] The present invention provides a layout rationality evaluation method based on a graph attention network, which includes:

[0010] Constructing a 3D network model dataset and an indoor 3D scene layout evaluation dataset based on the SUNCG dataset, and evaluating and annotating the 3D network model dataset and the indoor 3D scene layout evaluation dataset based on a preset Unity3D engine-released 3D scene annotation program to form a layout rationality evaluation dataset;

[0011] A preset 3D-SPAN network is trained using base class scene samples in the layout rationality evaluation dataset to initialize the 3D-SPAN network, and a sub-network in the initialized 3D-SPAN network is trained for binary classification using the 3D network model dataset and the indoor 3D scene layout evaluation dataset to form a small sample 3D scene evaluation network based on the 3D-SPAN-F framework;

[0012] The small sample three-dimensional scene evaluation network is used to construct a three-dimensional scene graph of the obtained indoor layout diagram based on the three-dimensional scene layout rationality evaluation algorithm to generate a three-dimensional scene graph, and the three-dimensional scene graph is evaluated to generate a binary evaluation result and a subjective logic evaluation result.

[0013] Preferably, the step of constructing a three-dimensional network model dataset based on the SUNCG dataset includes:

[0014] Collect a preset number of categories of scenes in the SUNCG dataset as 3D network model data samples;

[0015] Generate three-dimensional network model pending data according to the three-dimensional network model data sample;

[0016] The data of ambiguous three-dimensional object models are eliminated from all pending three-dimensional network model data to form a three-dimensional network model data set; wherein the ambiguous three-dimensional object models include: three-dimensional object models with collision ambiguity, position ambiguity, morphological ambiguity and visual ambiguity.

[0017] Preferably, the step of constructing an indoor three-dimensional scene layout evaluation dataset includes:

[0018] A preset number of categories of scenes are collected in the SUNCG dataset as 3D scene layout samples;

[0019] A 3D scene annotation program is released based on the Unity3D engine, and the 3D scene annotation program is configured to design a human subjective evaluation annotation process for the 3D network model dataset as a whole and for individual objects using a single stimulus continuous quality grading method, so as to generate subjective evaluation data on the 3D scene layout sample during the evaluation annotation stage;

[0020] Summarizing the subjective evaluation data to label binary evaluation data for the three-dimensional scene layout samples;

[0021] All 3D scene layout samples with binary evaluation data and subjective evaluation data are used as indoor 3D scene layout evaluation dataset.

[0022] Preferably, the step of constructing a three-dimensional scene graph for the acquired indoor layout diagram based on a three-dimensional scene layout rationality evaluation algorithm to generate a three-dimensional scene graph includes:

[0023] The obtained scene graph is parsed by the three-dimensional scene graph construction module in the three-dimensional scene layout rationality evaluation algorithm to obtain three-dimensional nodes; wherein the three-dimensional nodes include semantic information, spatial information and physical information;

[0024] Extracting the edge relationship of the object based on the semantic information, the spatial information, and the physical information, and representing the edge relationship as an adjacency vector; abstracting the semantic information into a word vector, abstracting the spatial information and the physical information into an attribute vector, and concatenating the word vector and the attribute vector to obtain a fixed-dimensional object feature vector;

[0025] The object feature vectors are used as nodes, and the adjacent vectors are used as edges to represent the indoor scene graph to form a three-dimensional scene graph.

[0026] Preferably, the step of performing a three-dimensional scene graph evaluation on the three-dimensional scene graph to generate a binary classification evaluation result and a subjective logic evaluation result includes:

[0027] Performing a local layout evaluation on the three-dimensional scene graph using a three-dimensional scene object evaluation network in the small sample three-dimensional scene evaluation network to obtain local evaluation prediction data, and performing a global layout evaluation on the three-dimensional scene graph using a three-dimensional scene overall evaluation network in the small sample three-dimensional scene overall evaluation network to obtain global evaluation prediction data;

[0028] The small sample network in the small sample three-dimensional scene evaluation network performs a binary evaluation on the three-dimensional scene graph based on the local evaluation prediction data and the global evaluation prediction data to obtain a binary evaluation result, and the global evaluation prediction data and the local evaluation prediction data are integrated to form a logical evaluation result.

[0029] Preferably, the 3D scene object evaluation network performs local layout evaluation on the 3D scene graph to obtain local evaluation prediction data, including:

[0030] Embedding nodes in the three-dimensional scene graph into respective local feature spaces to form local feature space representations through a single-layer perception mechanism of trainable parameters in the three-dimensional scene object evaluation network;

[0031] The single-layer multi-head graph attention layer and the double-layer graph attention layer in the graph attention module of the three-dimensional scene object evaluation network are used to perform local rationality prediction of the object on the feature space representation to obtain local evaluation prediction data.

[0032] Preferably, the process of performing a global layout evaluation on the three-dimensional scene graph by the three-dimensional scene overall evaluation network to obtain global evaluation prediction data includes:

[0033] Obtaining an original scene graph channel of the three-dimensional scene graph, and concatenating the original scene graph channel with the local feature space representation to obtain an enhanced global feature space representation;

[0034] Embedding the enhanced global feature space representation into a unified feature space based on a single-layer perception mechanism with trainable parameters in a three-dimensional scene overall evaluation network to form a global feature space representation;

[0035] Performing a global rationality evaluation on the global feature space representation through the dual-layer graph attention layer and dual-layer perception mechanism of the graph classification module in the 3D scene overall evaluation network to obtain a global rationality evaluation feature;

[0036] Data conversion is performed on the global rationality evaluation features to obtain global evaluation prediction data.

[0037] Preferably, the step of performing a binary classification evaluation on the three-dimensional scene graph based on the local evaluation prediction data and the global evaluation prediction result by the small sample network to obtain a binary classification evaluation result includes:

[0038] The small sample network extracts a hidden layer feature vector based on the local evaluation prediction data and the global evaluation prediction, and enhances the hidden layer feature vector to obtain a hidden layer vector;

[0039] Embedding the hidden layer vector into a high-discriminative feature space based on a prototype feature embedder with learnable parameters in the small sample network to form a high-discriminative feature representation;

[0040] The high-discrimination feature representation is subjected to a binary classification judgment by the multivariate learning module in the small sample network, so as to discriminate a binary classification evaluation result of "good" or "bad" for the three-dimensional scene graph based on the high-discrimination feature representation.

[0041] The present invention also provides a layout rationality evaluation system based on a graph attention network, wherein the layout rationality evaluation method based on a graph attention network as described above is implemented, and the system includes:

[0042] A sample data acquisition unit is used to construct a three-dimensional network model dataset and an indoor three-dimensional scene layout evaluation dataset based on the SUNCG dataset, and to evaluate and annotate the three-dimensional network model dataset and the indoor three-dimensional scene layout evaluation dataset based on a preset Unity3D engine-released three-dimensional scene annotation program to form a layout rationality evaluation dataset;

[0043] a model training unit, configured to train a preset 3D-SPAN network using base class scene samples in the layout rationality evaluation dataset to initialize the 3D-SPAN network, and perform binary classification training on a sub-network in the initialized 3D-SPAN network using the 3D network model dataset and the indoor 3D scene layout evaluation dataset to form a small sample 3D scene evaluation network based on the 3D-SPAN-F framework;

[0044] A rationality evaluation unit is used to construct a three-dimensional scene graph of the acquired indoor layout diagram based on the three-dimensional scene layout rationality evaluation algorithm through the small sample three-dimensional scene evaluation network to generate a three-dimensional scene graph, and to perform three-dimensional scene graph evaluation on the three-dimensional scene graph to generate a binary evaluation result and a subjective logic evaluation result.

[0045] Preferably, the small sample 3D scene evaluation network includes a 3D scene object evaluation network, a 3D scene overall evaluation network and a small sample network; wherein,

[0046] The three-dimensional scene object evaluation network is used to perform local layout evaluation on the three-dimensional scene graph to obtain local evaluation prediction data;

[0047] The three-dimensional scene overall evaluation network is used to perform global layout evaluation on the three-dimensional scene graph to obtain global evaluation prediction data;

[0048] The small sample network is used to perform a binary evaluation on the three-dimensional scene graph based on the local evaluation prediction data and the global evaluation prediction data to obtain a binary evaluation result, and integrate the global evaluation prediction data and the local evaluation prediction data to form a logical evaluation result.

[0049] From the above technical solution, it can be seen that the present invention provides a layout rationality evaluation method based on a graph attention network. First, a three-dimensional network model dataset and an indoor three-dimensional scene layout evaluation dataset are constructed based on the SUNCG dataset to form a layout rationality evaluation dataset. Then, a preset 3D-SPAN network is trained with base class scene samples in the layout rationality evaluation dataset to initialize the 3D-SPAN network. A small sample three-dimensional scene evaluation network based on the 3D-SPAN-F framework is trained on the basis of the initialized 3D-SPAN network. Then, a three-dimensional scene graph is constructed for the obtained indoor layout graph based on the three-dimensional scene layout rationality evaluation algorithm through the trained small sample three-dimensional scene evaluation network to generate a three-dimensional scene graph. , and performs 3D scene graph evaluation on the 3D scene graph to generate binary evaluation results and subjective logic evaluation results. The whole process utilizes the characteristic that the computational mechanism of the graph attention network is similar to the human subjective evaluation logic, combined with local enhanced global construction, to construct a 3D scene dataset suitable for the rationality evaluation of indoor 3D scene layout, and proposes a small sample 3D scene evaluation network for the rationality evaluation algorithm of indoor 3D scene layout based on the graph attention network. By fully mining the value of the constructed 3D-SPAD dataset, it can explore the relationship between human subjective evaluation and scene complexity, scene category and object category, as well as the relationship between the overall scene evaluation and the evaluation of objects in the scene with high precision, thereby improving the accuracy of the indoor 3D scene layout rationality evaluation algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] By referring to the following description in conjunction with the accompanying drawings, and with a more complete understanding of the present invention, other objects and results of the present invention will become more apparent and easier to understand. In the accompanying drawings:

[0051] Figure 1 Flowchart of a layout rationality evaluation method based on a graph attention network according to an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of data for constructing an indoor three-dimensional scene layout evaluation dataset in a layout rationality evaluation method based on a graph attention network according to an embodiment of the present invention;

[0053] Figure 3 Schematic diagram of a 3D scene object evaluation network and a 3D scene overall evaluation network in a small sample 3D scene evaluation network in a layout rationality evaluation method based on a graph attention network according to an embodiment of the present invention;

[0054] Figure 4 2 is a system block diagram of a layout rationality evaluation system based on a graph attention network according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The current task of evaluating the rationality of indoor three-dimensional scene layout still has much room for improvement in terms of practicality and reliability. The main difficulties are the following two aspects: lack of datasets: According to research, there is currently a lack of public three-dimensional scene datasets suitable for the task of evaluating the rationality of indoor three-dimensional scene layout; feature extraction and representation difficulties: The amount of three-dimensional scene data is huge and the original features are diverse. How to effectively extract layout information and represent it in a unified way is a key issue.

[0056] In response to the above problems, the present invention provides a layout rationality evaluation method and system based on a graph attention network. The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0057] In order to illustrate the layout rationality evaluation method and system based on graph attention network provided by the present invention, Figures 1-4 The embodiments of the present invention are exemplarily indicated.

[0058] The following description of exemplary embodiments is merely illustrative in nature and is in no way intended to limit the present invention, its application, or uses. Techniques and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques and devices should be considered part of the specification.

[0059] like Figure 1 As shown, the present invention provides a layout rationality evaluation method based on a graph attention network, comprising:

[0060] S1: constructing a 3D network model dataset and an indoor 3D scene layout evaluation dataset based on the SUNCG dataset, and evaluating and annotating the 3D network model dataset and the indoor 3D scene layout evaluation dataset based on a preset Unity3D engine-released 3D scene annotation program to form a layout rationality evaluation dataset;

[0061] S2: training a preset 3D-SPAN network using base class scene samples in the layout rationality evaluation dataset to initialize the 3D-SPAN network, and performing binary classification training on the sub-network in the initialized 3D-SPAN network using the 3D network model dataset and the indoor 3D scene layout evaluation dataset to form a small sample 3D scene evaluation network based on the 3D-SPAN-F framework;

[0062] S3: constructing a three-dimensional scene graph of the obtained indoor layout diagram based on the three-dimensional scene layout rationality evaluation algorithm through the small sample three-dimensional scene evaluation network to generate a three-dimensional scene graph, and performing a three-dimensional scene graph evaluation on the three-dimensional scene graph to generate a binary evaluation result and a subjective logic evaluation result.

[0063] exist Figure 1 、 Figure 2In the embodiment shown in the figure, step S1 is a step of constructing a 3D network model dataset and an indoor 3D scene layout evaluation dataset based on the SUNCG dataset, and evaluating and annotating the 3D network model dataset and the indoor 3D scene layout evaluation dataset based on a preset Unity3D engine-released 3D scene annotation program to form a layout rationality evaluation dataset; wherein, the step of constructing a 3D network model dataset based on the SUNCG dataset includes:

[0064] S111; collecting a preset number of categories of scenes in the SUNCG dataset as three-dimensional network model data samples;

[0065] S112: generating pending three-dimensional network model data according to the three-dimensional network model data sample;

[0066] S113: Eliminate ambiguous three-dimensional object model data from all pending three-dimensional network model data to form a three-dimensional network model data set; wherein the ambiguous three-dimensional object models include: three-dimensional object models with collision ambiguity, position ambiguity, morphology ambiguity and visual ambiguity.

[0067] The steps to construct an indoor 3D scene layout evaluation dataset include:

[0068] S121: Collecting a preset number of categories of scenes in the SUNCG dataset as three-dimensional scene layout samples;

[0069] S122: publishing a 3D scene annotation program based on the Unity3D engine, and causing the 3D scene annotation program to design a human subjective evaluation annotation process for the 3D network model dataset as a whole and for individual objects using a single stimulus continuous quality grading method, so as to generate subjective evaluation data on the 3D scene layout sample in an evaluation annotation phase;

[0070] S123: Summarizing the subjective evaluation data to label binary evaluation data for the three-dimensional scene layout sample;

[0071] S124: All three-dimensional scene layout samples with binary evaluation data and subjective evaluation data are used as an indoor three-dimensional scene layout evaluation dataset.

[0072] More specifically, in a specific embodiment, regarding the 3D network model dataset and indoor 3D scene layout evaluation dataset, a total of 2,100 indoor 3D scene samples from nine scene categories were collected from the SUNCG dataset to construct the 3D-SPAD dataset as 3D network model data samples. For example, the three scene categories of bedroom, bathroom, and living room, which appear frequently within the indoor scene types, were selected as base scene categories. The relatively low-occurrence scenes of nursery, dining room, gym, kitchen, office, and closet were selected as subsample scenes. Based on this data collection, a total of 1,500 scenes were collected for the bedroom, bathroom, and living room categories, with an average of 500 samples per category. Furthermore, a total of 600 scenes were collected for the nursery, dining room, gym, kitchen, office, and closet categories, with an average of 100 samples per category. Furthermore, due to the relatively small number of scenes collected, in order to prioritize subjective evaluation on scene layout and thus ensure data quality, a strategy based on object bounding box collision detection and filtering combined with manual filtering was used to filter out potentially ambiguous 3D object models during dataset construction. Ambiguous 3D object models can be roughly divided into four categories: collision ambiguity, position ambiguity, morphological ambiguity, and visual ambiguity:

[0073] Collision ambiguity: A 3D model that has regular collision and penetration with other 3D models on the bounding box.

[0074] Position ambiguity: A three-dimensional model that is hung without reasonable support.

[0075] Morphological ambiguity: Three-dimensional models with varying sizes and shapes, such as doors and irregular partitions.

[0076] Visual ambiguity: A 3D model that appears different from the real world, such as an air conditioner (with the air outlet facing upwards).

[0077] When constructing an indoor 3D scene layout evaluation dataset, we released 3D scene annotation software based on the Unity3D engine and used a single stimulus continuous quality grading method to design human subjective evaluation annotations for the collected 3D scene, both as a whole and on individual objects, to collect subjective evaluation data and obtain a basic evaluation dataset. The steps are as follows:

[0078] 1) Conduct evaluation software design:

[0079] A labeling software application for evaluating three-dimensional scenes was released based on the Unity3D engine. Through data conversion, the three-dimensional mesh model selected from the SUNCG dataset is first converted into the FBX file format, and an interface with the SUNCG data table is established. Then, rendering is generated based on the Unity3D engine. Evaluators can freely change the perspective to observe the indoor three-dimensional scene and evaluate the objects and the overall scene. The evaluation environment is an office environment with standard indoor lighting levels, providing standard lighting from top to bottom. In addition, the labeling software also provides a variety of functions that facilitate human-computer interaction, such as annotation guidance of the green bounding box, wall cancellation, global perspective, etc., to facilitate evaluators to conduct standardized evaluations.

[0080] 2) Evaluator training:

[0081] Evaluators can participate in the collection of subjective evaluation data. The age range of evaluators is 22-24 years old, with a male-to-female ratio of 11:25. After communication, it is understood that each evaluator has basic aesthetic appreciation for indoor three-dimensional scene layout. Before the evaluation begins, each evaluator will be briefly informed of the purpose of the evaluation and given a demonstration of the evaluation process to ensure the accuracy of the evaluation results. Since this article focuses on the rationality of the three-dimensional scene layout, the scene layout evaluation factors will be emphasized in the introduction. No other interference will be made to ensure the validity of the subjective data collection.

[0082] In one specific embodiment, the evaluation data consisted of 2,100 indoor 3D scenes. The scene data and annotation software were distributed to 36 evaluators through manual assignment. To ensure that each scene was evaluated by three different evaluators, the 36 evaluators were divided into three groups, each of which fully annotated 2,100 indoor 3D scenes, thereby ensuring evaluation accuracy. The evaluation method in this specific embodiment employed a single stimulus continuous quality grading method. This method involves presenting a single scene to the evaluators for observation and evaluation according to a set rating scale. Evaluators were required to subjectively evaluate the objects and the overall scene in the indoor 3D scene, with the evaluation options categorized as "good" or "poor." When evaluating objects in the scene, since walls and floors are also included in the rationality calculation, the evaluation labels for these objects are set to "empty" by default. Finally, the scene annotation data was aggregated, and a voting method was used to derive the final evaluation results, forming a basic evaluation dataset. The constructed basic evaluation dataset includes 2,100 indoor 3D scene samples, each with annotation results for overall rationality and the layout of objects within it.

[0083] Because the scene layouts in the SUNCG dataset are manually created by planners on a home design platform, the 3D scene samples collected from the SUNCG dataset have an overall superior layout. Furthermore, due to the small amount of collected data, the diversity of scene layouts is limited, which in turn limits the distribution range at the data level, affecting the learning performance of the neural network. Therefore, this embodiment adopts a data augmentation strategy that adds noise to address the data aggregation and insufficient data volume issues in the basic evaluation dataset.

[0084] That is, scene samples are randomly extracted from the basic evaluation dataset, and their layout is randomly shuffled. By default, the labels of objects and the overall scene are all "poor", so as to expand the diversity of samples. Among them, the 1500 base scene samples are expanded to 3000 by complete replication, and the 600 small sample scenes are randomly extracted and expanded to 900 after the layout is shuffled. In this embodiment, the scene samples after data enhancement are merged with the basic evaluation dataset to finally construct the indoor three-dimensional scene layout rationality evaluation dataset 3D-SPAD, which includes a total of 3900 scene samples, including 9 categories of scenes and 98 categories of objects. Among them, the three categories of scenes, bedroom, bathroom, and living room, have an average of 1000 examples per category, which are used as base category data to support graphic representation and small sample evaluation; the six categories of scenes, including nursery room, dining room, gym, kitchen, office, and cabinet room, have an average of 150 examples per category, which are used as small sample data to support small sample evaluation.

[0085] This improves the diversity of scene sample layouts in the dataset and expands the distribution range of the data, which helps the neural network better learn the decision boundary.

[0086] like Figure 1 、 Figure 3 As shown together, in this embodiment, step S2 is a training process, that is, a preset 3D-SPAN network is trained using base class scene samples in the layout rationality evaluation dataset to initialize the 3D-SPAN network, and a sub-network in the initialized 3D-SPAN network is trained for binary classification using the 3D network model dataset and the indoor 3D scene layout evaluation dataset to form a small sample 3D scene evaluation network based on the 3D-SPAN-F framework;

[0087] Specifically, in this embodiment, the 3D-SPAN network includes the 3DScene Plausibility Assessment Network-Object (3D-SPAN-O), which is trained as a 3D scene object evaluation network; the 3D Scene Plausibility Assessment Network-Scene (3D-SPAN-S), which is trained as a 3D scene overall evaluation network; and the 3D-SPAN-F framework, which is trained as a small-shot network. 3D-SPAN-F is a neural network based on 3D-SPAN, designed to improve the generalization of indoor 3D scene layout rationality evaluation, thereby expanding the algorithm's applicability. It uses a scene graph as input and outputs a 3D scene overall layout rationality evaluation result. In this embodiment, the small-shot network evaluation is defined as a "2-class k-shot" problem. That is, after learning samples of a base class of 3D scenes, only a small number of samples of a new class of 3D scenes are needed to acquire the ability to evaluate the layout rationality of a new class of 3D scenes. Specifically, it uses the binary classification of "good" and "bad" to evaluate scene samples each time, uses k samples from each type of scene to train the network or serve as comparative data support, and then tests the network and query results on samples of the new type of scene.

[0088] Specifically, during the training process, the 3D-SPAN network is used to learn the generalized layout information of base class scene samples in the 3D-SPAD dataset through pre-training, thereby achieving knowledge transfer and providing good initialization.

[0089] Regarding 3D-SPAN-O pre-training, the subnetwork 3D-SPAN-O consists of a single fully connected layer, a single multi-head graph attention layer, and a dual-layer graph attention layer. It is trained for node classification tasks by evaluating and labeling the layout rationality of objects within a three-dimensional scene. Using the graph attention mechanism, 3D-SPAN-O calculates weights based on the semantic, spatial, and physical features of objects, thereby performing asymmetric linear weighting when aggregating features. This process is similar to human subjective evaluation and reasoning of objects. Furthermore, at a logical level, the layout rationality of objects and the overall layout rationality of the scene form a local-global relationship. Therefore, as a local feature extractor, the subnetwork 3D-SPAN-O extracts local features E from a given scene graph G and enhances node features by aligning the scene graph, resulting in an enhanced scene graph G′ with richer semantic features.

[0090] Regarding D-SPAN-S pre-training, the subnetwork 3D-SPAN-S consists of a feature embedding module VE, a graph attention layer module GAT, and an output module Readout. The feature embedding module VE consists of a single fully connected layer, whose purpose is to embed the node features of the enhanced scene graph into a unified feature space. In the GAT part, a two-layer graph attention layer updates the features of the graph structure. Since the scene layout rationality evaluation classification is used as the training goal, the parameter update of the GAT part tends to learn the mapping relationship between the local and the global, that is, the influence of the objects represented by the nodes on the rationality of the global scene layout. The output module Readout consists of a two-layer fully connected layer, which aggregates the features of the graph structure and embeds them into a three-dimensional scene layout rationality feature vector for classification.

[0091] And in this embodiment, the training process adopts a meta-learning strategy to train the network, so that it can learn more general layout rationality features from the base class indoor three-dimensional scene samples of the 3D-SPAD dataset, thereby achieving better results in the new category indoor three-dimensional scene layout rationality evaluation task. In addition, in the sub-network 3D-SPAN-S, all the front layers except the last layer of the Readout part are used as general layout feature extractors to extract the hidden feature representation of the three-dimensional scene layout information. The Readout part of 3D-SPAN-S consists of two layers of fully connected layers, of which the first fully connected layer is responsible for aggregating the node embedding features output by the GAT part into the layout feature representation vector of the three-dimensional scene, and the second layer is used as a classifier to reduce the dimension of the layout feature representation vector of the three-dimensional scene. As a result, the hidden vector representing the three-dimensional scene layout information between the two fully connected layers of the Readout is extracted as the original feature of the prototype representation metric learning stage.

[0092] In addition, in this embodiment, graph fusion (Graph Mixup) is used to enhance the hidden layer feature vectors extracted by the sub-network 3D-SPAN-S to enrich the feature diversity, so that the network can better learn the generalization features in the layout features.

[0093] Given a set of hidden vectors Z, randomly select pairs of hidden vectors from them and aggregate them in proportion to obtain new hidden vectors. This process can be expressed as:

[0094] Z new =Z G α+Z′ G (1-α), Z G ,Z′ G ∈Z

[0095] where Z G and Z′ Grepresents different hidden layer vectors randomly selected from the hidden layer vector set Z, α is the scaling factor, and the experimental verification shows that α is 0.6, Z new As the hidden vector after graph fusion amplification, the label transfer strategy is used to obtain the Z G Consistent labeling.

[0096] Given a set of hidden vectors Z′ after feature enhancement, we use a set of hidden vectors with learnable parameters θ PE The prototype feature embedder PE embeds it into the feature space P with high discrimination. This process can be expressed as:

[0097] P=PE(Z′,θ PE )

[0098] In the prototype representation learning phase, the base class scenes in the 3D-SPAD dataset are first divided into two sets of "good" and "bad" according to the scene labels. Subsequently, a random sampling strategy is used to further divide the support set and query set based on the two sets of "good" and "bad". The support set contains 3 / 4 of the samples in the "good" and "bad" sets, and the remaining 1 / 4 of the samples are used as the query set. Two types of sample sets P are extracted from the support set scene representation vector set. good and P bad Aggregate the prototype center C of different categories of scenes in two evaluations good and C bad , this process can be expressed as:

[0099] C good ,C bad =∑(P good ,P bad )

[0100] In the metric learning phase, the scene representation vector set P in the query set is query With Prototype Center C good and C bad The measurement is performed to predict the evaluation classification of the query set scene samples. Here, the distance function between the measured scene representation vector and the corresponding prototype center is used as the loss function. This process can be expressed as:

[0101] C query =Metric((C bad ||C good ),P query ); where Metric represents the distance function. This section uses the L1 norm distance, || represents the concatenation operation, and C query It represents the distance between the query set sample and the prototype center, which can also be understood as the distance between the scene samples in the query set and different evaluation classification standards.

[0102] This embodiment uses a multi-task meta-learning strategy to update network parameters. The learning tasks can be divided into graph classification tasks and metric-based prototype representation learning tasks. These two tasks adopt a shared parameter fine-tuning training strategy. [96,102] With learnable parameters θ PS A common feature extractor is trained in the sub-network 3D-SPAN-S. And in the process of learning the graph classification task, the last layer of Readout will cause the three-dimensional scene hidden features extracted by the 3D-SPAN-S network to lose information during the update process due to the limitations of the low-dimensional space, which is equivalent to the graph representation vector output by the final classifier only containing part of the information of the hidden features. The purpose of prototype representation learning is to make the sample feature vector close to the prototype representation vector of the real evaluation type. Therefore, it is equivalent to constraining the feature distribution of the hidden features during the network training process, so that the universal feature extractor with shared parameters can achieve better performance in fine-tuning training. While maintaining the dimensional space, this information is used to aggregate the prototype representation to maximize the use of information in the hidden features for reasoning. Finally, the cross entropy loss is used for these two tasks to obtain the total loss function, where Y scene is the label of the scene, β is the hyperparameter ratio between the two losses, which is experimentally verified to be set to 1.6 to improve the training accuracy of the network.

[0103] exist Figure 1 、 Figure 3 In the embodiment shown together, step S3 is a process of constructing a three-dimensional scene graph of the obtained indoor layout diagram based on the three-dimensional scene layout rationality evaluation algorithm through the small sample three-dimensional scene evaluation network to generate a three-dimensional scene graph, and performing a three-dimensional scene graph evaluation on the three-dimensional scene graph to generate a binary evaluation result and a subjective logic evaluation result; wherein,

[0104] S311: The step of constructing a three-dimensional scene graph for the obtained indoor layout diagram based on a three-dimensional scene layout rationality evaluation algorithm to generate a three-dimensional scene graph includes:

[0105] S312: parsing the acquired scene graph using the 3D scene graph construction module in the 3D scene layout rationality evaluation algorithm to obtain 3D nodes; wherein the 3D nodes include semantic information, spatial information, and physical information;

[0106] S313: Extracting the edge relationship of the object based on the semantic information, the spatial information, and the physical information, and representing the edge relationship as an adjacency vector; abstracting the semantic information into a word vector, and abstracting the spatial information and the physical information into an attribute vector, and concatenating the word vector and the attribute vector to obtain a fixed-dimensional object feature vector;

[0107] S314: Using the object feature vectors as nodes and the adjacent vectors as edges to represent the indoor scene graph to form a three-dimensional scene graph.

[0108] In one specific embodiment, a 3D scene graph construction method based on a Gaussian mixture model is used to rapidly construct a 3D scene graph and efficiently and uniformly extract and organize indoor 3D scene layout information into a graph representation. During node construction, a zero-padding strategy is used for training to fix the number of nodes in the scene graph, and the nodes are represented as feature vectors. The feature vector is composed of feature vectors representing the semantic, spatial, and physical attributes of an object. Each node contains information about the object it represents. Specifically, the information contained in the node is as follows: Semantic information: Feature vectors are generated based on the object's category information using word vector embedding. For multi-word categories, vector alignment is achieved through average feature aggregation. Spatial information: A coordinate system centered on the scene ground is established, and the relative position coordinate parameters of the object, the initial orientation parameters of the object, and the rotation parameters of the object are extracted to generate a feature vector by splicing. Physical information: The coordinate parameters of the diagonal vertices are used to represent the object's bounding box features, and a feature vector is generated by splicing the vectorized object bounding box feature vector with the object's scale feature vector.

[0109] In the edge relationship construction part, the relationship between pairs of objects is represented in the form of an adjacency matrix. For each pair of relationships, if the relationship is judged to exist, it is represented as "1", otherwise it is represented as "0". Compared with the type of relationship between objects, this paper focuses on the possibility of the existence of relationships between objects. In terms of edge relationship extraction, an object-centered reference frame is used to extract the relationship between pairs of objects, and edge relationship extraction is divided into three types: object-supported, wall-supported, and non-supported. The object-supported and wall-supported types are based on manual definitions, while the non-supported type is based on a Gaussian mixture model, which is detailed as follows: Object-supported: The bottom of the bounding box of one object touches the top of the bounding box of another object. Wall-supported: By default, every object in the three-dimensional scene establishes a relationship with the wall and the ground, and the wall provides a constraint relationship for the object. Non-supported: If the Gaussian mixture distribution probability between the object pair exceeds the spatial distribution threshold of 0.05, it is defined as the existence of a non-supported relationship.

[0110] First, the input 3D scene data is parsed to obtain the semantic, spatial, and physical properties of each object in the 3D scene. The semantic properties are embedded as word vectors, and the spatial and physical properties are processed into vector form. A fixed-dimensional object feature vector is obtained by concatenating the vectors. Next, three types of object-to-edge relationships are extracted based on the semantic, spatial, and physical properties of the objects. All relationships are uniformly represented as "0" or "1" and output as an adjacency matrix. Finally, the object feature vectors are represented as nodes, and the adjacency matrix as edges, representing the indoor 3D scene layout information as a scene graph.

[0111] The step of performing a three-dimensional scene graph evaluation on the three-dimensional scene graph to generate a binary classification evaluation result and a subjective logic evaluation result includes:

[0112] S321: performing a local layout evaluation on the 3D scene graph using a 3D scene object evaluation network in the small sample 3D scene evaluation network to obtain local evaluation prediction data, and performing a global layout evaluation on the 3D scene graph using a 3D scene overall evaluation network in the small sample 3D scene overall evaluation network to obtain global evaluation prediction data;

[0113] S322: Perform a binary evaluation on the three-dimensional scene graph based on the local evaluation prediction data and the global evaluation prediction data through the small sample network in the small sample three-dimensional scene evaluation network to obtain a binary evaluation result, and integrate the global evaluation prediction data and the local evaluation prediction data to form a logical evaluation result.

[0114] Specifically, the 3D scene object evaluation network performs local layout evaluation on the 3D scene graph to obtain local evaluation prediction data, including:

[0115] Embedding nodes in the three-dimensional scene graph into respective local feature spaces to form local feature space representations through a single-layer perception mechanism of trainable parameters in the three-dimensional scene object evaluation network;

[0116] The single-layer multi-head graph attention layer and the double-layer graph attention layer in the graph attention module of the three-dimensional scene object evaluation network are used to perform local rationality prediction of the object on the feature space representation to obtain local evaluation prediction data.

[0117] The process of the 3D scene overall evaluation network performing global layout evaluation on the 3D scene graph to obtain global evaluation prediction data includes:

[0118] Obtaining an original scene graph channel of the three-dimensional scene graph, and concatenating the original scene graph channel with the local feature space representation to obtain an enhanced global feature space representation;

[0119] Embedding the enhanced global feature space representation into a unified feature space based on a single-layer perception mechanism with trainable parameters in a three-dimensional scene overall evaluation network to form a global feature space representation;

[0120] Performing a global rationality evaluation on the global feature space representation through the dual-layer graph attention layer and dual-layer perception mechanism of the graph classification module in the 3D scene overall evaluation network to obtain a global rationality evaluation feature;

[0121] Data conversion is performed on the global rationality evaluation features to obtain global evaluation prediction data.

[0122] More specifically, in this embodiment, 3D-SPAN consists of two sub-networks: a 3D Scene Plausibility Assessment Network-Object (trained by 3D-SPAN-O) and a 3D Scene Plausibility Assessment Network-Scene (trained by 3D-SPAN-S). Each sub-network is trained separately using a single-task learning strategy. Sub-network 3D-SPAN-O evaluates the layout rationality of objects in a 3D scene based on a scene graph, while sub-network 3D-SPAN-S evaluates the layout rationality of the entire 3D scene based on a feature-enhanced scene graph. 3D-SPAN's structural design simulates human subjective evaluation thinking. On the one hand, a local-enhanced-global structure is used to simulate the human thinking logic of evaluating the overall layout rationality of a 3D scene by referring to the layout information of objects within the scene. On the other hand, a graph attention mechanism is introduced to simulate the differentiated attention characteristics of humans when evaluating the rationality of scene and object layouts.

[0123] The superiority of the 3D-SPAN network is reflected in three parts: graph attention mechanism, local layout rationality evaluation, and global layout rationality evaluation:

[0124] 1. Graph Attention Mechanism; The superiority of the 3D-SPAN network is reflected in the graph attention mechanism it adopts. The graph attention network

[73] introduces the attention mechanism into the graph neural network based on the spatial domain. It can learn asymmetric attention weights from neighboring nodes and then update the node features through weighted aggregation calculation. This paper studies the task of combining local and global features to evaluate the entire scene. In a three-dimensional scene, local features come from objects in the three-dimensional scene. The evaluation of the rationality of the layout of objects in a three-dimensional scene is a relative problem. Human judgment of the rationality of the layout of objects is affected by factors such as relative spatial distribution, combination structure, and style differences. The graph attention mechanism can well consider the interaction between objects and calculate the correlation weight between object pairs based on their different semantic, spatial, and physical features. Therefore, this section introduces the graph attention mechanism to calculate the attention coefficients of different object pairs under different spatial distributions.

[0125] Here, the update process of node features can be defined as:

[0126]

[0127] Among them, GAC (v i ,v j ) represents the calculation of the attention weights between the nodes shown in the formula, vi represents the head node, v j Represents the tail node, A ij Indicates v i and v j The adjacency value between them, W represents the weight matrix, V represents the node feature set, represents the updated node feature set, and σ represents the activation function.

[0128] 2. Evaluation of local layout rationality: In the sub-network 3D-SPAN-O, the 3D scene graph G is taken as input, which can be expressed as Where N represents the number of nodes and M represents the length of node features. The 3D-SPAN-O network has trainable parameters θ F The single-layer perceptron VE embeds the nodes in the scene graph into a more distinguishable feature space F, where d represents the length of the node feature after embedding (in this paper, N=14, M=33, d=33). This process can be expressed as: F=VE(G,θ F );

[0129] The graph attention module GAM consists of a single-layer multi-head graph attention layer (head number is 4, output is ) and a two-layer graph attention layer (the first layer output is The output of the second layer is ) are connected in series, with trainable parameters θ A. GAM is used to update node features and generate object layout rationality evaluation features O, where (C=3), represents the predicted values ​​of N objects in the three layout rationality evaluation categories of "good", "bad", and "empty". This process can be expressed as:

[0130] O=GAM(F,θ A )

[0131] Specifically, the graph attention module (GAM) first processes the scene graph using a multi-head graph attention layer. Four parallel graph attention kernels are used to obtain four sets of scene graph features under different attention conditions. These four sets of graph features are then aggregated into a single graph embedding representation through feature concatenation. This set of graph embedding features is then processed using a two-layer graph attention layer to generate layout rationality features O. After softmax calculation and classification, O can be represented as an evaluation of object layout rationality. Furthermore, because O fully encodes a specific 3D scene through its corresponding set of nodes, it represents a set of latent vectors representing the local representation of the scene.

[0132] Finally, given a three-dimensional scene L = {l1,l2,l3,l n The rationality evaluation of object layout is as follows: The sub-network 3D-SPAN-O uses the cross entropy loss function for subjective evaluation of object layout rationality to train and update the network parameters. This process can be expressed as:

[0133]

[0134] 3. Global layout rationality evaluation: that is, before the scene graph is input into the sub-network 3D-SPAN-S, the scene graph needs to be enhanced. First, according to the channel dimension of the original scene graph, the original node feature vector V is spliced ​​with the 3D scene local representation O output by the sub-network 3D-SPAN-O, thereby obtaining a 3D scene graph G′ with enhanced features based on local information.

[0135] In the sub-network 3D-SPAN-S, there are trainable parameters θ F′ The single-layer perceptron VE′ embeds the original features and local features in the 3D scene graph into a unified feature space F′, where d′ is the length of the feature vector after embedding (this paper sets d′=36). This process can be expressed as:

[0136] F′=VE′(G′,θ F′ )

[0137] The graph classification module GCM consists of a two-layer graph attention layer (the first layer output is The output of the second layer is ) and a two-layer perceptron (the first layer output is The output of the second layer is ) are connected in series, which has a trainable parameter θ B The graph embedding features are processed through GCM, and the rationality evaluation feature S of the overall layout of the three-dimensional scene is generated. (C′=2), represents the predicted value of the scene in the two layout rationality evaluation categories of "good" and "bad". This process can be expressed as:

[0138] S=GCM(F′,θ B );

[0139] The subnetwork 3D-SPAN-S expands the 3D scene layout information globally by combining local representations with original scene graph features to infer the rationality of the 3D scene layout. The two-layer graph attention layer updates the unified features that combine the rationality information of object layouts and the object's own information to obtain the object layout latent vector, which represents the weight and influence trend of the object in the scene on the rationality evaluation of the 3D scene layout. Finally, a two-layer perceptron performs weighted aggregation of the object layout latent vector features of each object in the scene to obtain the overall scene layout rationality evaluation feature S. The main difference between GCM and GAM lies in the different features extracted by the graph attention layer. GCM focuses on learning global features in the 3D scene representation to calculate and output the overall layout rationality evaluation.

[0140] More importantly, in this embodiment, the step of performing a binary classification evaluation on the three-dimensional scene graph based on the local evaluation prediction data and the global evaluation prediction result by the small sample network to obtain a binary classification evaluation result includes:

[0141] The small sample network extracts a hidden layer feature vector based on the local evaluation prediction data and the global evaluation prediction, and enhances the hidden layer feature vector to obtain a hidden layer vector;

[0142] Embedding the hidden layer vector into a high-discriminative feature space based on a prototype feature embedder with learnable parameters in the small sample network to form a high-discriminative feature representation;

[0143] The high-discrimination feature representation is subjected to a binary classification judgment by the multivariate learning module in the small sample network, so as to discriminate a binary classification evaluation result of "good" or "bad" for the three-dimensional scene graph based on the high-discrimination feature representation.

[0144] That is, the small sample network is a unique network structure in the small sample 3D scene evaluation network. The framework of the small sample 3D scene evaluation network 3D-SPAN-F is as follows:

[0145] 3D-SPAN-F is a neural network proposed based on 3D-SPAN. Its purpose is to improve the generalization of indoor 3D scene layout rationality evaluation, thereby expanding the algorithm's applicability. It uses a scene graph as input and outputs an evaluation result of the overall 3D scene layout rationality. Small-sample scene evaluation is defined as a "2-class k-shot" problem. That is, after learning samples of base-class 3D scenes, only a small number of new-class 3D scene samples are needed to acquire the ability to evaluate the layout rationality of new-class 3D scenes. Specifically, scene samples are evaluated using a binary classification of "good" or "bad." K samples from each class are used to train the network or serve as comparison data. The network and query results are then tested on samples from the new-class scene.

[0146] As described above, the layout rationality evaluation method based on the graph attention network provided by the present invention first constructs a three-dimensional network model dataset and an indoor three-dimensional scene layout evaluation dataset based on the SUNCG dataset to form a layout rationality evaluation dataset, and then trains a preset 3D-SPAN network through the base class scene samples in the layout rationality evaluation dataset to initialize the 3D-SPAN network, and trains a small sample three-dimensional scene evaluation network based on the 3D-SPAN-F framework on the basis of the initialized 3D-SPAN network, and then constructs a three-dimensional scene graph for the obtained indoor layout graph based on the three-dimensional scene layout rationality evaluation algorithm through the trained small sample three-dimensional scene evaluation network to generate a three-dimensional scene graph, and performs three-dimensional scene evaluation on the three-dimensional scene layout rationality evaluation algorithm. The 3D scene graph is evaluated based on the 3D-SPAD scene graph to generate binary evaluation results and subjective logic evaluation results. The whole process utilizes the characteristic that the computational mechanism of the graph attention network is similar to the human subjective evaluation logic, combined with local enhancement of global construction, to construct a 3D scene dataset suitable for the rationality evaluation of indoor 3D scene layout. A small sample 3D scene evaluation network of the indoor 3D scene layout rationality evaluation algorithm based on the graph attention network is proposed. By fully mining the value of the constructed 3D-SPAD dataset, it can explore the relationship between human subjective evaluation and scene complexity, scene category and object category, as well as the relationship between the overall scene evaluation and the evaluation of objects in the scene with high precision, thereby improving the accuracy of the indoor 3D scene layout rationality evaluation algorithm.

[0147] like Figure 4 As shown, the present invention also provides a layout rationality evaluation system 100 based on a graph attention network, which implements the layout rationality evaluation method based on a graph attention network as described above, including:

[0148] The sample data collection unit 101 is used to construct a 3D network model dataset and an indoor 3D scene layout evaluation dataset based on the SUNCG dataset, and evaluate and annotate the 3D network model dataset and the indoor 3D scene layout evaluation dataset based on a preset Unity3D engine-released 3D scene annotation program to form a layout rationality evaluation dataset;

[0149] A model training unit 102 is configured to train a preset 3D-SPAN network using base class scene samples in the layout rationality evaluation dataset to initialize the 3D-SPAN network, and perform binary classification training on a sub-network in the initialized 3D-SPAN network using the 3D network model dataset and the indoor 3D scene layout evaluation dataset to form a small sample 3D scene evaluation network 000 based on the 3D-SPAN-F framework;

[0150] The rationality evaluation unit 103 is used to construct a three-dimensional scene graph of the obtained indoor layout diagram based on the three-dimensional scene layout rationality evaluation algorithm through the small sample three-dimensional scene evaluation network 000 to generate a three-dimensional scene graph, and to perform three-dimensional scene graph evaluation on the three-dimensional scene graph to generate a binary evaluation result and a subjective logic evaluation result.

[0151] The small sample 3D scene evaluation network 000 includes a 3D scene object evaluation network 001, a 3D scene overall evaluation network 002 and a small sample network 003; wherein,

[0152] The three-dimensional scene object evaluation network 001 is used to perform local layout evaluation on the three-dimensional scene graph to obtain local evaluation prediction data;

[0153] The three-dimensional scene overall evaluation network 002 is used to perform global layout evaluation on the three-dimensional scene graph to obtain global evaluation prediction data;

[0154] The small sample network 003 is used to perform a binary evaluation on the three-dimensional scene graph based on the local evaluation prediction data and the global evaluation prediction data to obtain a binary evaluation result, and integrate the global evaluation prediction data and the local evaluation prediction data to form a logical evaluation result.

[0155] The specific implementation method refers to the above-mentioned embodiment of the layout rationality evaluation method based on the graph attention network, which will not be described in detail here.

[0156] As described above, the layout rationality evaluation system based on the graph attention network provided by the present invention first constructs a three-dimensional network model dataset and an indoor three-dimensional scene layout evaluation dataset based on the SUNCG dataset through the data acquisition unit 101 to form a layout rationality evaluation dataset, and then trains a preset 3D-SPAN network based on the base class scene samples in the layout rationality evaluation dataset based on the model training unit 102 to initialize the 3D-SPAN network, and trains a small sample three-dimensional scene evaluation network 000 based on the 3D-SPAN-F framework on the basis of the initialized 3D-SPAN network, and then in the rationality evaluation unit 103, the obtained indoor layout graph is evaluated based on the three-dimensional scene layout rationality evaluation algorithm through the trained small sample three-dimensional scene evaluation network 000. A three-dimensional scene graph is constructed to generate a three-dimensional scene graph, and a three-dimensional scene graph evaluation is performed on the three-dimensional scene graph to generate a binary evaluation result and a subjective logic evaluation result. The whole process utilizes the characteristic that the computational mechanism of the graph attention network is similar to the human subjective evaluation logic, combined with local enhancement of global construction, to construct a three-dimensional scene dataset suitable for the rationality evaluation of indoor three-dimensional scene layout, and proposes a small sample three-dimensional scene evaluation network for the rationality evaluation algorithm of indoor three-dimensional scene layout based on the graph attention network. By fully mining the value of the constructed 3D-SPAD dataset, it is possible to explore with high precision the relationship between human subjective evaluation and scene complexity, scene category and object category, as well as the relationship between the overall scene evaluation and the evaluation of objects in the scene, thereby improving the accuracy of the indoor three-dimensional scene layout rationality evaluation algorithm.

[0157] The above description uses the accompanying drawings to illustrate the layout rationality evaluation method and system based on a graph attention network proposed in accordance with the present invention. However, those skilled in the art will appreciate that various improvements can be made to the layout rationality evaluation method and system based on a graph attention network proposed in accordance with the present invention without departing from the scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the contents of the appended claims.

Claims

1. A layout rationality evaluation method based on graph attention network, characterized by: include: Based on the SUNCG dataset, a 3D network model dataset and an indoor 3D scene layout evaluation dataset are constructed. The 3D network model dataset and the indoor 3D scene layout evaluation dataset are evaluated and annotated based on a preset Unity3D engine-released 3D scene annotation program to form a layout rationality evaluation dataset. All 3D scene layout samples with binary evaluation data and subjective evaluation data are used as the indoor 3D scene layout evaluation dataset. A preset 3D-SPAN network is trained using base-class scene samples in the layout rationality evaluation dataset to initialize the 3D-SPAN network, and a sub-network in the initialized 3D-SPAN network is trained for binary classification using the 3D network model dataset and the indoor 3D scene layout evaluation dataset to form a small-sample 3D scene evaluation network based on the 3D-SPAN-F framework. In the process of training the small-sample 3D scene evaluation network, the small-sample 3D scene evaluation network is defined as a "2-class k-shot problem." Based on learning base-class 3D scene samples, a small number of new-class 3D scene samples are learned, and each time a binary classification of "good" and "bad" is used to evaluate scene samples. K samples of each class of scenes are used to train the network or serve as comparison data support, and the network and query results are tested on samples of the new class of scenes. The small sample three-dimensional scene evaluation network is used to construct a three-dimensional scene graph for the obtained indoor layout diagram based on the three-dimensional scene layout rationality evaluation algorithm to generate a three-dimensional scene graph, and the three-dimensional scene graph is evaluated to generate a binary evaluation result and a subjective logic evaluation result; wherein, The step of performing a three-dimensional scene graph evaluation on the three-dimensional scene graph to generate a binary classification evaluation result and a subjective logic evaluation result includes: Performing a local layout evaluation on the three-dimensional scene graph using a three-dimensional scene object evaluation network in the small sample three-dimensional scene evaluation network to obtain local evaluation prediction data, and performing a global layout evaluation on the three-dimensional scene graph using a three-dimensional scene overall evaluation network in the small sample three-dimensional scene overall evaluation network to obtain global evaluation prediction data; The method further comprises: performing a binary evaluation on the three-dimensional scene graph based on the local evaluation prediction data and the global evaluation prediction data by a small sample network in the small sample three-dimensional scene evaluation network to obtain a binary evaluation result, and integrating the global evaluation prediction data and the local evaluation prediction data to form a logical evaluation result; wherein the method further comprises: The small sample network extracts a hidden layer feature vector based on the local evaluation prediction data and the global evaluation prediction, and enhances the hidden layer feature vector to obtain a hidden layer vector; Embedding the hidden layer vector into a high-discriminative feature space based on a prototype feature embedder with learnable parameters in the small sample network to form a high-discriminative feature representation; The high-discrimination feature representation is subjected to a binary classification judgment by the multivariate learning module in the small sample network, so as to determine a binary classification evaluation result of "good" or "bad" for the three-dimensional scene graph based on the high-discrimination feature representation.

2. The layout rationality evaluation method based on graph attention network according to claim 1 is characterized in that: The steps to construct a 3D network model dataset based on the SUNCG dataset include: Collect a preset number of categories of scenes in the SUNCG dataset as 3D network model data samples; Generate three-dimensional network model pending data according to the three-dimensional network model data sample; The data of ambiguous three-dimensional object models are eliminated from all pending three-dimensional network model data to form a three-dimensional network model data set; wherein the ambiguous three-dimensional object models include: three-dimensional object models with collision ambiguity, position ambiguity, morphological ambiguity and visual ambiguity.

3. The layout rationality evaluation method based on graph attention network according to claim 2 is characterized in that: The steps for obtaining subjective evaluation data and binary evaluation data include: A preset number of categories of scenes are collected in the SUNCG dataset as 3D scene layout samples; A 3D scene annotation program is released based on the Unity3D engine, and the 3D scene annotation program is configured to design a human subjective evaluation annotation process for the 3D network model dataset as a whole and for individual objects using a single stimulus continuous quality grading method, so as to generate subjective evaluation data on the 3D scene layout sample during the evaluation annotation stage; The subjective evaluation data are summarized to simultaneously label binary evaluation data for the three-dimensional scene layout samples.

4. The layout rationality evaluation method based on graph attention network according to claim 3 is characterized in that: The step of constructing a three-dimensional scene graph based on the obtained indoor layout diagram based on the three-dimensional scene layout rationality evaluation algorithm to generate a three-dimensional scene graph includes: The obtained scene graph is parsed by the three-dimensional scene graph construction module in the three-dimensional scene layout rationality evaluation algorithm to obtain three-dimensional nodes; wherein the three-dimensional nodes include semantic information, spatial information and physical information; Extracting the edge relationship of the object based on the semantic information, the spatial information, and the physical information, and representing the edge relationship as an adjacency vector; abstracting the semantic information into a word vector, abstracting the spatial information and the physical information into an attribute vector, and concatenating the word vector and the attribute vector to obtain a fixed-dimensional object feature vector; The object feature vectors are used as nodes, and the adjacent vectors are used as edges to represent the indoor scene graph to form a three-dimensional scene graph.

5. The layout rationality evaluation method based on graph attention network according to claim 4 is characterized in that: The 3D scene object evaluation network performs local layout evaluation on the 3D scene graph to obtain local evaluation prediction data, including: Embedding nodes in the three-dimensional scene graph into respective local feature spaces to form local feature space representations through a single-layer perception mechanism of trainable parameters in the three-dimensional scene object evaluation network; The single-layer multi-head graph attention layer and the double-layer graph attention layer in the graph attention module of the three-dimensional scene object evaluation network are used to perform local rationality prediction of the object on the feature space representation to obtain local evaluation prediction data.

6. The layout rationality evaluation method based on graph attention network according to claim 5 is characterized in that: The process of the 3D scene overall evaluation network performing global layout evaluation on the 3D scene graph to obtain global evaluation prediction data includes: Obtaining an original scene graph channel of the three-dimensional scene graph, and concatenating the original scene graph channel with the local feature space representation to obtain an enhanced global feature space representation; Embedding the enhanced global feature space representation into a unified feature space based on a single-layer perception mechanism with trainable parameters in a three-dimensional scene overall evaluation network to form a global feature space representation; Performing a global rationality evaluation on the global feature space representation through the dual-layer graph attention layer and dual-layer perception mechanism of the graph classification module in the 3D scene overall evaluation network to obtain a global rationality evaluation feature; Data conversion is performed on the global rationality evaluation features to obtain global evaluation prediction data.

7. A layout rationality evaluation system based on graph attention network, characterized by: A method for evaluating layout rationality based on a graph attention network according to any one of claims 1 to 6 is implemented, wherein the system comprises: A sample data acquisition unit is configured to construct a 3D network model dataset and an indoor 3D scene layout evaluation dataset based on the SUNCG dataset, and to evaluate and annotate the 3D network model dataset and the indoor 3D scene layout evaluation dataset based on a preset Unity3D engine-released 3D scene annotation program to form a layout rationality evaluation dataset; wherein the indoor 3D scene layout evaluation dataset is composed of all 3D scene layout samples with binary evaluation data and subjective evaluation data attached; a model training unit, configured to train a preset 3D-SPAN network using base-class scene samples in the layout rationality evaluation dataset to initialize the 3D-SPAN network, and perform binary classification training on a subnetwork in the initialized 3D-SPAN network using the 3D network model dataset and the indoor 3D scene layout evaluation dataset to form a small-sample 3D scene evaluation network based on the 3D-SPAN-F framework; wherein, during the training of the small-sample 3D scene evaluation network, the small-sample 3D scene evaluation network is defined as a "2-class k-shot problem"; based on learning base-class 3D scene samples, a small number of new-class 3D scene samples are learned, and each time a binary classification of "good" and "bad" is used to evaluate scene samples, with k samples of each class of scenes used to train the network or as comparison data support, and the network and query results are tested on samples of the new class of scenes; A rationality evaluation unit is configured to construct a three-dimensional scene graph for the acquired indoor layout diagram based on the three-dimensional scene layout rationality evaluation algorithm through the small sample three-dimensional scene evaluation network to generate a three-dimensional scene graph, and to perform three-dimensional scene graph evaluation on the three-dimensional scene graph to generate a binary evaluation result and a subjective logic evaluation result; wherein, The three-dimensional scene object evaluation network is used to perform local layout evaluation on the three-dimensional scene graph to obtain local evaluation prediction data; The three-dimensional scene overall evaluation network is used to perform global layout evaluation on the three-dimensional scene graph to obtain global evaluation prediction data The small sample network is used to perform a binary evaluation on the three-dimensional scene graph based on the local evaluation prediction data and the global evaluation prediction data to obtain a binary evaluation result, and integrate the global evaluation prediction data and the local evaluation prediction data to form a logical evaluation result; wherein, it includes: The small sample network extracts a hidden layer feature vector based on the local evaluation prediction data and the global evaluation prediction, and enhances the hidden layer feature vector to obtain a hidden layer vector; Embedding the hidden layer vector into a high-discriminative feature space based on a prototype feature embedder with learnable parameters in the small sample network to form a high-discriminative feature representation; The high-discrimination feature representation is subjected to a binary classification judgment by the multivariate learning module in the small sample network, so as to determine a binary classification evaluation result of "good" or "bad" for the three-dimensional scene graph based on the high-discrimination feature representation.

Images