Text-driven three-dimensional scene matching method based on structure alignment

Abstract

The invention discloses a text-driven three-dimensional scene matching method based on structure alignment, and the method comprises the steps: obtaining instance feature vectors and relation feature vectors of a query text and a to-be-matched three-dimensional scene, and carrying out the statistics of the number of instances of the query text and the number of instances of the to-be-matched three-dimensional scene; judging the number of instances of the to-be-matched three-dimensional scene and the number of instances of the query text; matching of the query instance and the scene instance is completed, and an instance matching degree score is obtained; extracting a relation feature vector of a corresponding instance in the three-dimensional scene, and calculating a relation matching degree score; calculating a consistency score of the instance and the relationship; and performing weighted summation on the instance matching degree score, the relationship matching degree score and the consistency score to obtain a final matching score of the query text and the three-dimensional scene, and completing matching of the three-dimensional scene and the query text. The method solves the problems that a traditional method is weak in structure perception, low in matching precision and insufficient in efficiency, and is suitable for large-scale text-driven three-dimensional scene retrieval and scene matching.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of 3D scene retrieval and natural language processing, specifically a text-driven 3D scene matching method based on structural alignment. Background Technology

[0002] Current text-to-3D scene matching methods primarily focus on instance-level single-level matching, which has several core shortcomings: First, global matching is coarse. Most methods emphasize global similarity comparison, making it difficult to accurately characterize the local semantic constraints in the query text. Existing text-to-3D scene matching methods typically perform overall similarity calculations between the query text and the entire scene representation, focusing on coarse-grained correspondences in global semantics, while lacking fine-grained modeling of key objects and their local structural constraints in the query. For example, for descriptions like "apple on the table," if only global similarity comparison is performed, although both "table" and "apple" may exist in the scene, it is impossible to accurately distinguish whether they satisfy the spatial relationship of "apple is on the table," thus causing the matching result to deviate from the user's true semantic intent. Secondly, there is a lack of query-driven local structure localization. Faced with sparse natural language queries and dense 3D scene representations, there is a lack of targeted structure alignment mechanisms, making it impossible to accurately locate query-related local structures from the global scene. This easily leads to the inclusion of objects in the scene that are irrelevant to the query in the matching range, causing matching redundancy and significantly reducing matching accuracy. Thirdly, there is a lack of "instance-relationship" consistency constraints. In existing methods, instance matching and relationship matching are mostly calculated and independently merged, without modeling the structural consistency between the two. It is difficult to ensure the synergy between instance matching and relationship matching, and fragmented matching problems such as correct semantic matching of instances but inconsistent spatial relationships are prone to occur, which seriously affects the reliability and accuracy of the overall matching results.

[0003] Furthermore, existing methods lack standardized feature processing and matching procedures, failing to achieve standardized alignment between query text and 3D scene features. This results in poor repeatability of matching results and makes them unsuitable for engineering applications in large-scale 3D scene retrieval systems. In summary, there is an urgent need for a text-driven 3D scene matching method that can achieve two-dimensional "instance-relationship" matching, possesses efficient structural alignment capabilities, and balances matching accuracy with computational efficiency, in order to address the shortcomings of existing technologies. Summary of the Invention

[0004] The purpose of this invention is to provide a text-driven 3D scene matching method based on structural alignment to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A text-driven 3D scene matching method based on structure alignment includes the following steps: Step 1: Obtain the instance feature vector and relation feature vector of the query text and the 3D scene to be matched, and count the number of instances n of the query text and the number of instances m of the 3D scene to be matched; Step 2: Determine the size of the number of instances m of the 3D scene to be matched and the number of instances n of the query text. If m < n, the matching degree between the 3D scene and the query text is determined to be 0, and the subsequent matching steps for the 3D scene are skipped; if m ≥ n, the subsequent matching steps are executed. Step 3: Calculate the instance similarity matrix based on the instance feature vectors of the query and the scene, and use a greedy matching algorithm to match the query instance with the scene instance to obtain the instance matching score; Step 4: Based on the scene instance index obtained by greedy matching, extract the relation feature vector of the corresponding instance in the 3D scene, and combine it with the relation feature vector of the query text to calculate the relation matching score. Step 5: Combine the instance similarity matrix, relation similarity matrix, and greedy matching results to calculate the consistency score between instances and relations; Step 6: Perform a weighted sum of the instance matching score, relation matching score, and consistency score to obtain the final matching score between the query text and the 3D scene, thus completing the matching between the 3D scene and the query text.

[0006] Further, in step 1, the feature vectors of the query text and the 3D scene are both extracted through a pre-trained model, stored as torch format files, and loaded using the torch.load() function; wherein, the dimension of the instance feature vector Query_Feats_Obj of the query text is (n,4,feat_dim), the dimension of the relational feature vector Query_Feats_Rel of the query text is (n,n,feat_dim), the dimension of the instance feature vector Scene_Feats_Obj of the 3D scene is (m,4,feat_dim), and the dimension of the relational feature vector Scene_Feats_Rel of the 3D scene is (m,m,feat_dim), where feat_dim is the feature dimension.

[0007] Furthermore, in step 3, the calculation process of the instance similarity matrix S_obj_mat is as follows: Step 3.1: Using the L2 normalization method, normalize Query_Feats_Obj and Scene_Feats_Obj respectively to obtain the normalized feature vectors A_norm and B_norm; Step 3.2: Calculate the temporary instance similarity matrix S_obj_mat_tmp using tensor einsum operation to obtain a temporary matrix with dimensions (n,m,4); Step 3.3: Generate the mask S_obj_mat_tmp_mask for the temporary matrix; Step 3.4, set the weight vector; The temporary similarity matrix is ​​weighted by matrix multiplication to obtain the final instance similarity matrix S_obj_mat. The calculation formula is as follows: ; The final result is an instance similarity matrix with dimensions (n, m).

[0008] Furthermore, the normalization formula in step 3.1 is: ; Where x_norm is the feature vector to be normalized, F is torch.nn.functional, torch.nn.functional is a set of functions in PyTorch, normalize(・) represents L2 normalization operation, p is the norm type to be used, and dim is the specified dimension for normalization. The calculation formula for step 3.2 is as follows: ; Wherein, nkd represents The dimension, mkd represents The dimension, nmk represents The dimension is torch.einsum() represents the torch.einsum function.

[0009] Further, in step 3.3, the mask S_obj_mat_tmp_mask is determined by the formula... The calculation yielded the result; In step 3.4, the formula for setting the weight vector weights is: ; Here, torch.tensor() represents the torch.tensor function, and w_cat, w_col, w_mat, and w_oth represent the weights of instance type, color, material, and other attributes, respectively.

[0010] Furthermore, in step 3, the specific process of the greedy matching algorithm is as follows: Step 3.5: Initialize the remaining unmatched query instance index set `remaining_queries`, which contains all query instance indexes, with index values ​​ranging from 0 to n-1, i.e., `remaining_queries={0,1,...,n-1}`; and the remaining unmatched scene instance index set `remaining_scenes`, which contains all scene instance indexes, with index values ​​ranging from 0 to m-1, i.e., `remaining_scenes={0,1,...,m-1}`; simultaneously, initialize the list `matched_scores` for storing match similarity, and the lists `row_ind` and `col_ind` for storing match indices. Step 3.6: Loop n times. Each time, iterate through all instance pairs from remaining_queries and remaining_scenes, find the instance pair with the highest similarity in the instance similarity matrix S_obj_mat, and record the similarity value of the instance pair, the query instance index best_q, and the scene instance index best_s. Step 3.7: Add the current highest similarity value to the matched_scores list, add best_q and best_s to the row_ind and col_ind lists respectively, and remove the corresponding instance index from remaining_queries and remaining_scenes; Step 3.8: After the loop ends, calculate the average of all similarity values ​​in the matched_scores list as the instance matching score S_obj. The calculation formula is: ; Here, matched_scores is a list storing the similarity values ​​between each pair of query instances and scene instances, and mean() represents the mean function.

[0011] Furthermore, in step 4, the calculation process for the relationship matching score S_rel is as follows: Step 4.1: Use the torch.meshgrid() function to generate the grid indices ix and iy corresponding to the greedy matching index col_ind, with the indexing mode being indexing='ij'; Step 4.2: Based on the grid indices ix and iy, extract the relation features of the corresponding instances from the 3D scene relation feature vector Scene_Feats_Rel to obtain the feature vector Scene_Feats_Rel_ with dimensions (n, n, feat_dim); Step 4.3: Using the L2 normalization method, normalize Query_Feats_Rel and Scene_Feats_Rel to obtain the normalized feature vectors A_norm and B_norm. Step 4.4: Calculate the sum of the dot products of the two normalized feature vectors to obtain the relation similarity matrix S_rel_mat. The calculation formula is as follows: ; This yields a relation similarity matrix of dimension (n,n); where sum() represents the summation function. Step 4.5, generate the relation feature mask, the formula is: ; In the formula, ne(0) means checking whether each element is not equal to 0, any(dim=-1) means checking for the existence of any element in the last dimension, and int() means the int function. Step 4.6: Based on the relationship similarity matrix and mask, calculate the relationship matching score S_rel. The calculation formula is as follows: .

[0012] Furthermore, in step 5, the calculation process for the consistency score C is as follows: Step 5.1, generate the off-diagonal mask non_diag_mask, the formula is: ; Used to filter diagonal elements in the relation similarity matrix; where torch.eye(n) is a PyTorch function used to generate an n×n identity matrix; Step 5.2: Multiply the relation feature mask (mask) with the off-diagonal mask (non_diag_mask) to obtain the combined mask (combined_mask); Step 5.3: Based on the combined mask, calculate the sum of each row of the relation similarity matrix S_rel_mat (row_sum) and the number of valid elements in each row (row_count); Step 5.4, calculate the average similarity row_mean for each row. The formula is: ; 1e-8 is used to avoid division by zero; Step 5.5: Based on the row index list `row_ind` and column index list `col_ind` matched by the greedy algorithm, extract the corresponding similarity values ​​from the instance similarity matrix `S_obj_mat`, multiply them element-wise with `row_mean`, and take the average to obtain the consistency score `C`. The calculation formula is as follows: ; Where K is the total number of matching pairs, I r (k) represents the row_ind corresponding to the k-th matching pair, I c (k) represents the col_ind corresponding to the k-th matching pair, and r represents the row_mean vector.

[0013] Furthermore, in step 6, the formula for calculating the final matching score SS is as follows: ; ; Where alpha is the instance matching weight; beta is the consistency score weight; and S is the median score of instance-relationship matching.

[0014] Furthermore, in step 6, completing the matching of the 3D scene and the query text includes: sorting the final matching scores SS of all 3D scenes to be matched, selecting the scene with the highest score as the 3D scene that best matches the query text, thereby realizing text-driven 3D scene retrieval and matching.

[0015] Compared with the prior art, the beneficial effects of the present invention are: 1) Breaking through the limitations of existing methods that only focus on instance matching, we achieve accurate alignment between query instances and scene instances through greedy matching. Then, we extract the scene relation substructure based on the matching results to complete the relation dimension matching. At the same time, we introduce a consistency score to penalize the fragmented situation of "accurate instance matching but incorrect relation matching", ensuring that the matching results are highly consistent with the structural semantics of the query text, and achieving "instance-relationship" two-dimensional structural alignment.

[0016] 2) The greedy matching algorithm is adopted, which greatly reduces the computational complexity while ensuring matching accuracy, and adapts to the real-time requirements of large-scale 3D scene retrieval. At the same time, the features are stored in a standardized torch format, and the matching process is standardized and modularized, which facilitates patent implementation and engineering integration. It can be directly applied to various 3D content retrieval platforms, balancing matching efficiency and engineering adaptability.

[0017] 3) By pre-screening by the number of instances, scenarios where the number of instances is less than the query are directly excluded, reducing invalid calculations; a masking mechanism is used to filter instance attributes not mentioned in the query and invalid relationship features, avoiding interference from irrelevant information on the matching results and significantly improving matching accuracy.

[0018] 4) The weights of instance attributes and matching weights (alpha, beta) can be flexibly adjusted according to the actual application scenario. For example, in furniture design retrieval, the material weight can be increased, and in scene layout retrieval, the relationship matching weight can be increased, with extremely high flexibility and adaptability. Brief Description of the Drawings

[0019] Figure 1 This is the flowchart of the present invention. Detailed Implementation Manner

[0020] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0021] This embodiment discloses a text-driven 3D scene matching method based on structural alignment, as Figure 1 shown, mainly including the following steps: Step 1, obtain the instance feature vector Query_Feats_Obj and the relationship feature vector Query_Feats_Rel corresponding to the query text, obtain the instance feature vector Scene_Feats_Obj and the relationship feature vector Scene_Feats_Rel corresponding to the 3D scene to be matched, and respectively count the instance number n of the query text and the instance number m of the 3D scene to be matched.

[0022] Among them, the dimension of Query_Feats_Obj is (n, 4, feat_dim), the dimension of Query_Feats_Rel is (n, n, feat_dim), the dimension of Scene_Feats_Obj is (m, 4, feat_dim), and the dimension of Scene_Feats_Rel is (m, m, feat_dim); n is the instance number of the query text (in this embodiment, n = 5 is taken), m is the instance number of the scene (in this embodiment, m = 8 is taken), and feat_dim is the feature dimension (in this embodiment, feat_dim = 1280 is taken); the feature vectors are all extracted through a pre-trained vision-language model and stored in a tensor format.

[0023] Step 2, compare the scene instance number m with the query text instance number n. If m < n (for example, if m = 3 and n = 5 in this embodiment), it is determined that the scene matching degree is 0, and the subsequent matching is skipped; if m ≥ n (for example, m = 8 ≥ n = 5 in this embodiment), continue to execute.

[0024] Step 3: Based on the query text instance feature vector Query_Feats_Obj and the 3D scene instance feature vector Scene_Feats_Obj, calculate the instance similarity matrix S_obj_mat. Then, use a greedy matching algorithm to match the query text instances with the 3D scene instances to obtain the instance matching score S_obj. The calculation process of the instance similarity matrix S_obj_mat is as follows: Step 3.1: Using L2 normalization, normalize Query_Feats_Obj and Scene_Feats_Obj respectively to obtain the normalized feature vectors A_norm and B_norm. The normalization formula is: (1) In equation (1), x_norm is the feature vector to be normalized (in this embodiment, x is Query_Feats_Obj with dimensions (5,4,1280) or Scene_Feats_Obj with dimensions (8,4,1280)). normalize(・) represents the L2 normalization operation, and the normalized feature vector has a magnitude of 1. p specifies the norm type to use, and dim specifies the dimension to perform the normalization.

[0025] Step 3.2: Calculate the temporary instance similarity matrix S_obj_mat_tmp using the tensor einsum operation. The calculation formula is as follows: (2) We obtain a temporary matrix with dimensions (5,8,4).

[0026] Wherein, nkd represents The dimension, mkd represents The dimension, nmk represents The dimension is torch.einsum() represents the torch.einsum function.

[0027] Step 3.3: Generate the mask S_obj_mat_tmp_mask for the temporary matrix, which is obtained by the following formula and used to filter invalid eigenvalues: (3) In the formula, ! = indicates not equal to, and float() represents the float function; Step 3.4, set the weight vector `weights`, using the following formula: (4) In the formula, torch.tensor() represents the torch.tensor function. In this embodiment, w_cat=0.85 is the instance type weight, w_col=0.05 is the color weight, w_mat=0.05 is the material weight, and w_oth=0.05 is the weight of other attributes, such as shape, texture, size, etc.

[0028] The temporary similarity matrix is ​​weighted by matrix multiplication to obtain the final instance similarity matrix S_obj_mat. The calculation formula is as follows: (5) In the formula, torch.matmul represents the torch.matmul function, which ultimately yields an instance similarity matrix with dimensions (5,8).

[0029] The specific process of the greedy matching algorithm is as follows: Step 3.5: Initialize the remaining unmatched query instance index set `remaining_queries`, which contains the indexes of all query instances, with a total of 5 query instances and index values ​​ranging from 0 to 4 (i.e., `remaining_queries={0,1,2,3,4}`); and the remaining unmatched scene instance index set `remaining_scenes`, which contains the indexes of all scene instances, with a total of 8 scene instances and index values ​​ranging from 0 to 7 (i.e., `remaining_scenes={0,1,2,3,4,5,6,7}`). Simultaneously, initialize the list `matched_scores` for storing match similarity scores, and the lists `row_ind` and `col_ind` for storing match indices. Step 3.6: Loop 5 times. Each time, iterate through all instance pairs from remaining_queries and remaining_scenes, find the instance pair with the highest similarity in the instance similarity matrix S_obj_mat, and record the similarity value of the instance pair, query the instance index best_q and the scene instance index best_s. Step 3.7: Add the current highest similarity value to the matched_scores list, add best_q and best_s to the row_ind and col_ind lists respectively, and remove the corresponding instance index from remaining_queries and remaining_scenes; Step 3.8, after the loop ends (the content after 5 iterations in step 3.6), calculate the average of all similarity values ​​in the matched_scores list, as the instance matching score S_obj. The calculation formula is: (6) In the formula, matched_scores is a list storing the similarity values ​​between each pair of query instances and scene instances, and mean() represents the mean function. In this embodiment, matched_scores=[0.92,0.88,0.90,0.85,0.89], and the average value S_obj=0.888.

[0030] Step 4: Based on the scene instance index col_ind obtained through greedy matching, extract the relation feature vector Scene_Feats_Rel_ of the corresponding instance in the 3D scene. Combine this with the query text relation feature vector Query_Feats_Rel to calculate the relation matching score S_rel. The calculation process for the relation matching score S_rel is as follows: Step 4.1: Use the torch.meshgrid() function to generate the grid indices ix and iy corresponding to the greedy matching index col_ind, with the indexing mode being indexing='ij'; Step 4.2: Based on the grid indices ix and iy, extract the relation features of the corresponding instances from the 3D scene relation feature vector Scene_Feats_Rel to obtain the feature vector Scene_Feats_Rel_ with dimensions (5, 5, 1280); Step 4.3: Using the L2 normalization method, normalize Query_Feats_Rel and Scene_Feats_Rel to obtain the normalized feature vectors A_norm and B_norm. Step 4.4: Calculate the sum of the dot products of the two normalized feature vectors to obtain the relation similarity matrix S_rel_mat. The calculation formula is as follows: (7) In the formula, sum() represents the summation function, and in this embodiment, the dimension of S_rel_mat is (5,5); Step 4.5: Generate a relation feature mask to filter invalid relation features. The formula is: (8) In the formula, ne(0) means checking whether each element is not equal to 0, any(dim=-1) means checking for the existence of any element in the last dimension, and int() means the int function.

[0031] Step 4.6: Based on the relationship similarity matrix and mask, calculate the relationship matching score S_rel. The calculation formula is as follows: (9) By combining the weighted average of the mask matrix (the effective number of masks in this embodiment is 20), we get S_rel=0.85.

[0032] Step 5: Based on the instance similarity matrix S_obj_mat, the relation similarity matrix S_rel_mat, and the greedy matching results, calculate the consistency score C between instances and relations. The calculation process for the consistency score C is as follows: Step 5.1, generate the off-diagonal mask non_diag_mask, the formula is: (10) Used to filter diagonal elements in the relation similarity matrix; where torch.eye(n) is a PyTorch function used to generate an n×n identity matrix.

[0033] Step 5.2: Multiply the relation feature mask (mask) with the off-diagonal mask (non_diag_mask) to obtain the combined mask (combined_mask); Step 5.3: Based on the combined mask, calculate the sum of each row of the relation similarity matrix S_rel_mat (row_sum) and the number of valid elements in each row (row_count); Step 5.4, calculate the average similarity row_mean for each row. The formula is: (11) 1e-8 is used to avoid division by zero; Step 5.5: Based on the row index list `row_ind` and column index list `col_ind` matched by the greedy algorithm, extract the corresponding similarity values ​​from the instance similarity matrix `S_obj_mat`, multiply them element-wise with `row_mean`, and take the average to obtain the consistency score `C`. The calculation formula is as follows: (12) Where K is the total number of matching pairs (K=5 in this embodiment), I r (k) is the query instance matching index for the k-th matching pair (e.g., [0,1,2,3,4]), I c (k) is the scene instance matching index of the k-th matching pair (e.g., [2,5,1,7,4]), and r is the row average similarity vector (r=[0.86,0.89,0.87,0.84,0.88] in this embodiment); calculated, C=0.87 in this embodiment.

[0034] Step 6: Based on the instance matching score S_obj, the relation matching score S_rel, and the consistency score C, the final matching score SS between the query text and the 3D scene to be matched is calculated by weighted summation, thus completing the matching between the 3D scene and the query text.

[0035] The formula for calculating the final match score SS is: (13) (14) Where alpha is the instance matching weight (alpha=0.6 in this embodiment); beta is the consistency score weight (beta=0.2 in this embodiment); S is the intermediate score of instance-relationship matching, and SS is the final matching score. Substituting the values, we get S=0.8728, SS=(1-0.2)×0.8728+0.2×0.87=0.8722.

[0036] In this embodiment, the final matching score between the query text and the 3D scene is 0.8722.

[0037] The process of matching 3D scenes with query text involves sorting the final matching scores (SS) of all 3D scenes to be matched (e.g., scene 1 gets 0.8722, scene 2 gets 0.75, scene 3 gets 0.91, and so on), selecting the scene with the highest score as the 3D scene that best matches the query text, thus achieving text-driven 3D scene retrieval and matching.

[0038] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A text-driven 3D scene matching method based on structural alignment, characterized in that, Includes the following steps: Step 1: Obtain the instance feature vector and relation feature vector of the query text and the 3D scene to be matched, and count the number of instances n of the query text and the number of instances m of the 3D scene to be matched; Step 2: Determine the size of the number of instances m of the 3D scene to be matched and the number of instances n of the query text. If m < n, the matching degree between the 3D scene and the query text is determined to be 0, and the subsequent matching steps for the 3D scene are skipped; if m ≥ n, the subsequent matching steps are executed. Step 3: Calculate the instance similarity matrix based on the instance feature vectors of the query and the scene, and use a greedy matching algorithm to match the query instance with the scene instance to obtain the instance matching score; Step 4: Based on the scene instance index obtained by greedy matching, extract the relation feature vector of the corresponding instance in the 3D scene, and combine it with the relation feature vector of the query text to calculate the relation matching score. Step 5: Combine the instance similarity matrix, relation similarity matrix, and greedy matching results to calculate the consistency score between instances and relations; Step 6: Perform a weighted sum of the instance matching score, relation matching score, and consistency score to obtain the final matching score between the query text and the 3D scene, thus completing the matching between the 3D scene and the query text.

2. The text-driven 3D scene matching method based on structure alignment according to claim 1, characterized in that, In step 1, the feature vectors of the query text and the 3D scene are extracted by a pre-trained model, stored as torch format files, and loaded using the torch.load() function. The query text instance feature vector Query_Feats_Obj has dimensions (n, 4, feat_dim), the query text relational feature vector Query_Feats_Rel has dimensions (n, n, feat_dim), the 3D scene instance feature vector Scene_Feats_Obj has dimensions (m, 4, feat_dim), and the 3D scene relational feature vector Scene_Feats_Rel has dimensions (m, m, feat_dim), where feat_dim is the feature dimension.

3. The text-driven 3D scene matching method based on structural alignment according to claim 2, characterized in that, In step 3, the calculation process of the instance similarity matrix S_obj_mat is as follows: Step 3.1: Using the L2 normalization method, normalize Query_Feats_Obj and Scene_Feats_Obj respectively to obtain the normalized feature vectors A_norm and B_norm; Step 3.2: Calculate the temporary instance similarity matrix S_obj_mat_tmp using tensor einsum operation to obtain a temporary matrix with dimensions (n,m,4); Step 3.3: Generate the mask S_obj_mat_tmp_mask for the temporary matrix; Step 3.4, set the weight vector; The temporary similarity matrix is ​​weighted by matrix multiplication to obtain the final instance similarity matrix S_obj_mat. The calculation formula is as follows: ； The final result is an instance similarity matrix with dimensions (n, m).

4. The text-driven 3D scene matching method based on structural alignment according to claim 3, characterized in that, The normalization formula in step 3.1 is: ； Where x_norm is the feature vector to be normalized, F is torch.nn.functional, torch.nn.functional is a set of functions in PyTorch, normalize(・) represents L2 normalization operation, p is the norm type to be used, and dim is the specified dimension for normalization. The calculation formula for step 3.2 is as follows: ； Wherein, nkd represents The dimension, mkd represents The dimension, nmk represents The dimension is torch.einsum() represents the torch.einsum function.

5. The text-driven 3D scene matching method based on structure alignment according to claim 4, characterized in that, In step 3.3, the mask S_obj_mat_tmp_mask is determined by the formula... The calculation yielded the result; In step 3.4, the formula for setting the weight vector weights is: ； Here, torch.tensor() represents the torch.tensor function, and w_cat, w_col, w_mat, and w_oth represent the weights of instance type, color, material, and other attributes, respectively.

6. The text-driven 3D scene matching method based on structural alignment according to claim 5, characterized in that, In step 3, the specific process of the greedy matching algorithm is as follows: Step 3.5: Initialize the remaining unmatched query instance index set `remaining_queries`, which contains all query instance indexes, with index values ​​ranging from 0 to n-1, i.e., `remaining_queries={0,1,...,n-1}`; and the remaining unmatched scene instance index set `remaining_scenes`, which contains all scene instance indexes, with index values ​​ranging from 0 to m-1, i.e., `remaining_scenes={0,1,...,m-1}`; simultaneously, initialize the list `matched_scores` for storing match similarity, and the lists `row_ind` and `col_ind` for storing match indices. Step 3.6: Loop n times. Each time, iterate through all instance pairs from remaining_queries and remaining_scenes, find the instance pair with the highest similarity in the instance similarity matrix S_obj_mat, and record the similarity value of the instance pair, the query instance index best_q, and the scene instance index best_s. Step 3.7: Add the current highest similarity value to the matched_scores list, add best_q and best_s to the row_ind and col_ind lists respectively, and remove the corresponding instance index from remaining_queries and remaining_scenes; Step 3.8: After the loop ends, calculate the average of all similarity values ​​in the matched_scores list as the instance matching score S_obj. The calculation formula is: ； Here, matched_scores is a list storing the similarity values ​​between each pair of query instances and scene instances, and mean() represents the mean function.

7. The text-driven 3D scene matching method based on structural alignment according to claim 1, characterized in that, In step 4, the calculation process for the relationship matching score S_rel is as follows: Step 4.1: Use the torch.meshgrid() function to generate the grid indices ix and iy corresponding to the greedy matching index col_ind, with the indexing mode being indexing='ij'; Step 4.2: Based on the grid indices ix and iy, extract the relation features of the corresponding instances from the 3D scene relation feature vector Scene_Feats_Rel to obtain the feature vector Scene_Feats_Rel_ with dimensions (n, n, feat_dim); Step 4.3: Using the L2 normalization method, normalize Query_Feats_Rel and Scene_Feats_Rel to obtain the normalized feature vectors A_norm and B_norm. Step 4.4: Calculate the sum of the dot products of the two normalized feature vectors to obtain the relation similarity matrix S_rel_mat. The calculation formula is as follows: ； This yields a relation similarity matrix of dimension (n,n); where sum() represents the summation function. Step 4.5, generate the relation feature mask, the formula is: ； In the formula, ne(0) means checking whether each element is not equal to 0, any(dim=-1) means checking for the existence of any element in the last dimension, and int() means the int function. Step 4.6: Based on the relationship similarity matrix and mask, calculate the relationship matching score S_rel. The calculation formula is as follows: 。 8. The text-driven 3D scene matching method based on structural alignment according to claim 1, characterized in that, In step 5, the consistency score C is calculated as follows: Step 5.1, generate the off-diagonal mask non_diag_mask, the formula is: ； Used to filter diagonal elements in the relation similarity matrix; where torch.eye(n) is a PyTorch function used to generate an n×n identity matrix; Step 5.2: Multiply the relation feature mask (mask) with the off-diagonal mask (non_diag_mask) to obtain the combined mask (combined_mask); Step 5.3: Based on the combined mask, calculate the sum of each row of the relation similarity matrix S_rel_mat (row_sum) and the number of valid elements in each row (row_count); Step 5.4, calculate the average similarity row_mean for each row. The formula is: ； 1e-8 is used to avoid division by zero; Step 5.5: Based on the row index list `row_ind` and column index list `col_ind` matched by the greedy algorithm, extract the corresponding similarity values ​​from the instance similarity matrix `S_obj_mat`, multiply them element-wise with `row_mean`, and take the average to obtain the consistency score `C`. The calculation formula is as follows: ； Where K is the total number of matching pairs, I r (k) represents the row_ind corresponding to the k-th matching pair, I c (k) represents the col_ind corresponding to the k-th matching pair, and r represents the row_mean vector.

9. A text-driven 3D scene matching method based on structural alignment according to claim 1, characterized in that, In step 6, the formula for calculating the final matching score SS is: ； ； Where alpha is the instance matching weight; beta is the consistency score weight; and S is the median score of instance-relationship matching.

10. A text-driven 3D scene matching method based on structural alignment according to claim 1, characterized in that, In step 6, completing the matching of the 3D scene and the query text includes: sorting the final matching scores SS of all the 3D scenes to be matched, selecting the scene with the highest score as the 3D scene that best matches the query text, and realizing text-driven 3D scene retrieval and matching.

Images