A three-dimensional scene generation method based on entity spatial relationship reasoning

Abstract

The present application belongs to the field of computer three-dimensional scene generation, and more specifically, relates to a three-dimensional scene generation method based on entity space relationship reasoning. This method combines the reasoning ability of a large language model with the learning ability of a specific task to reason and summarize the entity and space position relationship in the text description. In this way, the scene described in the text can be converted into a specific three-dimensional scene and rendered and displayed in a three-dimensional engine. This method can be widely used in virtual reality, game development, film and television production and other fields, and has important value for improving work efficiency and innovation ability.

Description

Technical Field

[0001] This invention belongs to the field of computer 3D scene generation, and more specifically, relates to a 3D scene generation method based on entity spatial relationship reasoning. Background Technology

[0002] 3D scene construction technology integrates digital image processing, computer graphics, computer vision, human-computer interaction, and other technologies. Leveraging the powerful computing and graphics rendering capabilities of computers, it creates a realistic virtual world. Currently, 3D scene construction technology is widely used in military, medical, educational, entertainment, and cultural relic preservation fields. In terms of construction methods, it can be broadly divided into manual modeling and automatic modeling. Automatic scene modeling technology can be further divided into constraint-based scene construction, rule-based scene construction, and learning-based scene construction. Currently, manually constructing 3D virtual scenes requires significant human and material resources; the scene production is complex and requires a large number of skilled CG professionals. Automatic scene construction technology is still in its early stages, remaining at a simple text-to-model mapping level. Object relationships within the scene are often vague, resulting in low richness and flexibility in the generated scenes, and the generated scene effects still fall short of users' realistic needs. Summary of the Invention

[0003] To address the problems of monotonous scene content and disorganized model placement, this invention proposes a 3D scene generation method based on entity spatial relationship reasoning. This method uses a Large Language Model (LLM) to capture information from the scene description text and infers the entities, implicit entities, and existing spatial relationships, which helps identify discontinuous entities in the text. A spatial layout algorithm based on prior probability and fuzzy distance calculation between entities is proposed to realize the placement of scene objects, ultimately generating richer and more rationally laid-out virtual scenes. This improves modeling efficiency, shortens project cycles, reduces communication costs, and provides a new solution for virtual scene construction.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] A method for generating 3D scenes based on entity spatial relationship reasoning, comprising the following steps:

[0006] S1 uses a large language model to receive the scene text input by the user and completes the task of information extraction and information reasoning from the input text. This text includes one or more entities and their spatial relationships. The large language model processes the scene text and outputs the entities in the scene and their spatial relationships.

[0007] S2, Spatial Relationship Optimization: Utilizes prior probability algorithms and fuzzy distance algorithms to optimize the spatial relationships of entities returned by the large language model, forming a new scene description file;

[0008] S3 generates a 3D scene based on the scene description file.

[0009] In a further optimization of this technical solution, step S1 defines the spatial relationship library as 12 basic three-dimensional spatial position constraints {"up", "down", "left", "right", "front", "back", "middle", "edge", "inner", "outer", "near", "far"}, and uses triples of <α,r,β> to formally describe the spatial relationships of entities, where α and β represent entities within the scene, and r is the spatial relationship type.

[0010] In a further optimization of this technical solution, the fuzzy distance algorithm in step S2 calculates the entity positional relationship as follows:

[0011] Distance calculation between scene entities generally falls into two categories: one is distance calculation between two scene entities on the same support, and the other is distance calculation between two scene entities on different supports. When the distance between scene entities is not explicitly given, and the scene description text contains words indicating distance but not explicitly stated, d>0 is used to represent this. If words indicate no distance but are difficult for the computer to understand, d=0 is used to represent this. Assuming the 3D coordinates of scene entity A are represented as A(x1,y1,z1) and the 3D coordinates of scene entity B are represented as B(x2,y2,z2), fuzzy distance calculation can be further divided into the following three cases:

[0012] (1) If d>0, the distance calculation for scene entities on different supports can be performed by inference based on the distances of other known scene entities. Assuming the coordinates of scene entity A are A(x1,y1,z1) and the coordinates of scene entity B are B(x2,y2,z2), the specific formula is as follows:

[0013]

[0014] (2) When one scene entity supports another scene entity, the distance between them is represented by d = 0. The formula for calculating the distance between entities A(x1,y1) and B(x2,y2) on the same supporting surface is as follows:

[0015]

[0016] (3) When the distance cannot be estimated by existing scene entities, the approximate position of the object is estimated by using spatial relationships, and the spatial layout of the scene entities is precisely arranged by combining the support relationship of the object. Finally, according to the principle of d=0, the object is placed on its own support to determine the fuzzy distance between different objects in order to achieve the best effect.

[0017] In a further optimization of this technical solution, step S2 defines the set of all entity categories in the scene as I = {l1, l2, l3, ..., l...} n},l1,l2,l3,…,l n Let there be n distinct entities l in the scene. The set of positional relationships between the objects is defined as R = {r1, r2, r3, ..., r...} p}, r1, r2, r3, ..., r p Represent p distinct positional relations r, and construct a set of spatial relation triples. A semantic scene graph is constructed based on the set RT. The directional word R is used to represent the spatial relationship between element X and element Y, where X is the main component in the spatial relationship and Y is the reference object for the orientation of scene entity X.

[0018] When multiple spatial relationships exist in a scene, the following steps are used to traverse the spatial relationships between different entities and create a spatial relationship connectivity graph:

[0019] 201. Before traversing, mark each vertex as visited, initially setting it to an unvisited state;

[0020] 202. For the current vertex "v", mark it as visited;

[0021] 203. Find the first neighboring vertex "w" of vertex "v";

[0022] 204. If vertex "w" has not been visited, then recursively execute the algorithm starting from vertex "w";

[0023] 205. Update the current vertex "w" to the next neighbor of vertex "v";

[0024] 206. Repeat steps 204 and 205 until all adjacent vertices of vertex "v" have been visited;

[0025] 207. Repeat steps 202 to 206 until all vertices have been visited;

[0026] By traversing the spatial relationships between different entities and using the spatial relationship propagation method, a spatial relationship connectivity graph is formed, which contains the set of spatial relationships RT in the entire scene.

[0027] This technical solution is further optimized by generating a semantic scene graph using the following steps based on the input spatial relation triplet set RT={(X,Y,R)}:

[0028] 211. Preprocess the input spatial relation triples RT;

[0029] 212. Extract candidate spatial relation triples from the preprocessed spatial relation triples;

[0030] 213. Sort and remove duplicates from candidate spatial relation triples using sorting and filtering functions;

[0031] 214. Iterate through each preprocessed triplet. For each triplet "T"_i, perform the following operations:

[0032] a. Call the "Expand" function to expand the current triple to achieve spatial relation propagation.

[0033] b. Based on the expanded spatial relationship connectivity graph, call the "ssgGen" function to generate a semantic scene graph.

[0034] c. Store the generated semantic scene graph.

[0035] 215. Output the final generated semantic scene graph.

[0036] This technical solution is further optimized, and the prior probability algorithm is as follows:

[0037] Next, we calculate the prior probability of scene entities appearing in different scenarios, and the frequency P of scene entities appearing in a certain scenario description. prop The specific formula is as follows:

[0038]

[0039] In the formula above, d represents the scene entity, and S i This represents a scenario of a certain type, i. Count(d) represents the number of times the hidden entity appears in that scenario based on prior knowledge. i P represents the expected number of scenarios of a certain type i. prop (d|S i The number ) represents the frequency of occurrence of the hidden entity d in a certain type of scene i;

[0040] The probability P of spatial relationships between entities in a statistical scenario rela The specific formula is as follows:

[0041]

[0042] r represents the relative positional relationship between scene entities. This indicates that, under the observation of m scene examples, object O i With object O j The number of relations r, Count(S) i (,r) represents the number of relations r in scenario i, calculated by P rela (r|Si It can clearly describe the spatial relationships between scene entities, and their relationships in different scenes. i The number of times and frequency of occurrence of the following.

[0043] In a further optimization of this technical solution, step S3 includes searching for the entity 3D model contained in the JSON file in the local model library, converting the scene entity into a specific 3D model, importing it into the 3D engine, reading the scene information to be constructed from the scene description file, and realizing the placement of different entities in the scene.

[0044] The above technical solution has the following advantages compared to existing technologies:

[0045] This invention primarily studies how to visualize scene description text in natural language form as a three-dimensional static scene. It proposes a method using a fine-tuned large language model to extract information and summarize spatial relationships from user-input scene text, inferring implicit entities associated with entities in the user's description text. This expands the model's scope within the three-dimensional scene, addressing the previous issues of limited scene content and variability, and ensuring the richness of scene content.

[0046] This invention proposes a spatial layout algorithm based on prior probability and fuzzy distance calculation between entities to realize the spatial placement of objects in a scene. The construction process is more automated and intelligent, and the generated scene layout is more reasonable and meets actual needs. Attached Figure Description

[0047] Figure 1 This is a framework diagram for a 3D scene method based on entity spatial relationship reasoning. Detailed Implementation

[0048] To explain in detail the technical content, structural features, objectives, and effects of the technical solution, the following description is provided in conjunction with specific embodiments and accompanying drawings.

[0049] This invention proposes a 3D scene generation method based on entity spatial relationship reasoning. Using a large language model, it analyzes the user-input scene description text and extracts scene entity and spatial relationship information, obtaining implicit entities with strong correlation to scene entities, forming spatial relationship triples, thereby enriching the scene. Multiple spatial relationship constraints are defined, and through scene layout reasoning based on prior probability and scene entity distance reasoning based on fuzzy algorithms, the layout of model positions within the 3D scene becomes more reasonable and intelligent, addressing the problems of monotonous scene content and unreasonable entity positional relationships in related methods.

[0050] See Figure 1The diagram shown is a framework diagram of a 3D scene method based on entity spatial relationship reasoning. A preferred embodiment of the present invention provides a 3D scene method based on entity spatial relationship reasoning, which specifically includes the following steps:

[0051] Step S1: Use Large Language Models (LLMs) to receive the user-inputted scene text and perform information extraction and reasoning tasks from the input text. The input text may contain one or more entities and their spatial relationships. This step is the starting point of the entire method, providing the necessary input information for subsequent steps.

[0052] Furthermore, the large language model is fine-tuned using cue word engineering. Through few-shot learning, a specific cue template is set, and the large language model is fine-tuned using a small number of training samples and examples. This guides the large language model to complete a specific task and produce outputs that meet the requirements. The task-specific model needs to complete two tasks: first, extracting information from the input text to extract spatial relationships between scene entities; second, using thought chain cue engineering technology to guide the model in information reasoning, completing the proposed entity reasoning to associate entities with the implicit spatial relationships existing within the scene.

[0053] Furthermore, leveraging the information extraction and summarization capabilities of the large language model, the task of extracting information from scene description text is completed, enabling the extraction and summarization of scene entities and spatial relationships. Ultimately, the entity and spatial relationship information within the scene is represented in a structured manner. For example, a finely tuned large language model prompt template could be: "You are an excellent linguist. Your task is to extract entity information and spatial relationships from a given scene description text. Now let's analyze the entity and spatial position relationships within the scene description text <<There is a sofa in the living room, and to the left of the sofa is a table>> step by step, and return spatial relationship triples in JSON format." Based on the input text, the LLM will generate the corresponding reasoning process and output format, describing the extracted spatial relationships and formalizing them as <α,r,β> triples. Here, α and β represent entities within the scene, and r is the spatial relationship type, providing information for subsequent scene generation.

[0054] Furthermore, utilize the context understanding ability of the large language model to complete the information inference task of the input text, and achieve the inference and prediction of hidden entities and implicit spatial relationships within the scene. Based on the proposed entities and inferred implicit entities, infer the possible spatial relationships between scene entities. Here, scene entities include various types of named entities such as person names, object names, and organization names, and the scene relationships are a set of 12 basic spatial constraints: {"above", "below", "left", "right", "front", "back", "center", "edge", "inside", "outside", "near", "far"}, which are used to limit the types of spatial relationships automatically inferred and predicted by the LLM, and realize the topological relationship position of implicit entities in three-dimensional space. The LLM will automatically select the most similar spatial relationship from the set of spatial constraints and return it. Use the parameters x, y, z to represent the coordinate information of the object in three-dimensional space. The specific 12 spatial constraint relationships can be interpreted as follows: [[ID=z]]

[0055] The coordinate representation form of the spatial relationship constraint "above" is Up(A, B): A(z) > B(z), which means that object A is above object B. <000zz74>

[0056] The coordinate representation form of the spatial relationship constraint "below" is Down(A, B): A(z) < B(z), which means that object A is below object B.

[0057] The coordinate representation form of the spatial relationship constraint "left" is Left(A, B): A(y) < B(y), which means that object A is on the left of object B.

[0058] The coordinate representation form of the spatial relationship constraint "right" is Right(A, B): A(y) > B(y), which means that object A is on the right of object B.

[0059] The coordinate representation form of the spatial relationship constraint "front" is Front(A, B): A(x) < B(x), which means that object A is in front of object B.

[0060] The coordinate representation form of the spatial relationship constraint "back" is Bhind(A, B): A(x) > B(x), which means that object A is behind object B.

[0061] The coordinate representation form of the spatial relationship constraint "center" is Center(A, B): A(x) ∈ B(x) / 2 & A(y) ∈ B(y) / 2 & A(z) ∈ B(z) / 2, which means that object A is in the middle of object B.

[0062] The coordinate representation form of the spatial relationship constraint "edge" is Edge(A, B): A(x) = B(x) or A(y) = B(y) or A(z) = B(Z), which means that object A is on the edge of object B in a certain direction.

[0063] The coordinate representation form of the spatial relationship constraint "inside" is Inside(A, B): B(x max ) > A(x) > B(x min ) & B(y max ) > A(y) > B(y min ) & B(z max ) > A(z) > B(z min ), which means that object A is inside object B.

[0064] The coordinate representation form of the spatial relationship constraint "outside" is Outside(A, B): A(x) < B(x min ) or A(x) > B(x max ) or A(y) < B(y min ) or A(y) > B(y max ) or A(z) < B(z min ) or A(z) > B(z max ), which means that object A is outside object B.

[0065] The coordinate representation form of the spatial relationship constraint "near" is Near(A, B): distance(A, B) < d, which means that object A is near object B, where d is the fuzzy distance relationship threshold.

[0066] The coordinate representation form of the spatial relationship constraint "far" is Far(A, B) = distance(A, B) > d, which means that object A is far from object B, where d is the fuzzy distance relationship threshold.

[0067] For example, for the text "There is a table on the floor of the living room", the LLM will automatically extract <table, on, floor> as the spatial relationship within the text and output it in JSON format. According to the defined three-dimensional spatial position constraint relationship "on", object A is above object B. The spatial orientation in the example text can be expressed as:

[0068] Up(table, floor): table(z) > floor(z)

[0069] The LLM will infer the implicit entity "sofa" based on the scene information "living room" and the implicit relationship <sofa, inside, living room> as the information inference result for return output. According to the defined three-dimensional spatial position constraint relationship "inside", object A is inside object B. The spatial orientation in the example text can be expressed as:

[0070] Inside(sofa, living room): living room(x max ) > sofa(x) > living room(x min ) & living room(ymax Sofa (y) > Living Room (y) min )&Living Room(z max Sofa (z) > Living Room (z) min )

[0071] Step S2: Spatial Relationship Optimization. While most scene entities and spatial relationships can be directly derived from semantics, the implicit spatial relationships of entities inferred by large language models may contradict common sense. To more accurately grasp the spatial relationships described in the scene text, further processing and optimization of the results returned by LLM are necessary.

[0072] Furthermore, a prior probability algorithm for statistically analyzing common entities and spatial relationships is proposed. First, a scene set needs to be selected, and the entity elements and spatial relationships within the scene need to be abstractly described to provide data support for subsequent prior probability calculations. The scene data set is selected, and all entities within the scene are defined as set I = {l1, l2, l3, ..., l...}. n},l1,l2,l3,…,l n Let R represent n distinct entities l within the scene. The set of positional relationships between these objects is defined as R = {r1, r2, r3, ..., r...} p}, r1, r2, r3, ..., r p Represent p distinct positional relations r. Construct a set of spatial relation triples. A semantic scene graph (SSG) is constructed based on the set RT. The locative term R represents the spatial relationship between element X and element Y, where X is the main component of the spatial relationship, and Y is the reference point for the orientation of scene entity X. This embodiment uses a spatial relationship propagation method to form a spatial relationship connectivity graph to describe the spatial relationship set RT in the entire scene, ultimately obtaining the semantic scene graph, which guides the subsequent scene generation.

[0073] When multiple spatial relationships exist in a scene, the following steps are used to traverse the spatial relationships between different entities and create a spatial relationship connectivity graph:

[0074] 1. Before traversing, mark each vertex as unvisited.

[0075] 2. For the current vertex "v", mark it as visited.

[0076] 3. Find the first neighboring vertex "w" of vertex "v".

[0077] 4. If vertex "w" has not been visited, the algorithm is recursively executed starting from vertex "w".

[0078] 5. Update the current vertex "w" to the next neighbor of vertex "v".

[0079] 6. Repeat steps 4 and 5 until all adjacent vertices of vertex "v" have been visited.

[0080] 7. Repeat steps 2 through 6 until all vertices have been visited.

[0081] In this way, by traversing the spatial relationships between different entities and utilizing the propagation of spatial relationships, a spatial relationship connectivity graph is formed, which contains the set RT of spatial relationships throughout the entire scene. The specific code steps are as follows:

[0082] {

[0083] Visit vertex v,visited[v]=1;

[0084] W=the first adjacent point of vertex v;

[0085] While (w exists)

[0086] If(w is not visited)recursively executes the algorithm from vertex w;

[0087] w=the next adjacent point of vertex v;

[0088] }

[0089] Furthermore, based on the input set of spatial relation triples RT = {(X,Y,R)}, the semantic scene graph is generated using the following steps:

[0090] 1. Preprocess the input spatial relation triples RT.

[0091] 2. Extract candidate spatial relation triples from the preprocessed spatial relation triples.

[0092] 3. Sort and remove duplicates from candidate spatial relation triples using sorting and filtering functions.

[0093] 4. Iterate through each preprocessed triplet. For each triplet T... i Perform the following operations:

[0094] a. Call the "Expand" function to expand the current triplet to achieve spatial relation propagation.

[0095] b. Based on the expanded spatial relationship connectivity graph, call the "ssgGen" function to generate a semantic scene graph (SSG).

[0096] c. Store the generated Semantic Scene Graph (SSG).

[0097] 5. Output the final generated Semantic Scene Graph (SSG).

[0098] The specific code steps are as follows:

[0099]

[0100] Next, we calculate the prior probabilities of scene entities and spatial relationships occurring in different scenarios. The frequency P of a certain scene entity occurring in a certain scenario is... prop The specific formula is as follows:

[0101]

[0102] In the formula above, d represents the scene entity, and S i This represents a scenario of a certain type, i. Count(d) represents the number of times an entity appears in that scenario given prior knowledge. i P represents the expected number of scenarios of a certain type i. prop (d|S i ) represents the frequency with which the hidden entity d appears in a certain type of scene i.

[0103] The probability P of spatial relationships between entities in a statistical scenario rela The specific formula is as follows:

[0104]

[0105] r represents the relative positional relationship between scene entities. This indicates that, under the observation of m scene examples, object O i With object O j The number of relations r. Count(S) i (,r) represents the number of relations r in scenario i. This is determined by calculating P... rela (r|S i It can clearly describe the spatial relationships between scene entities, and their relationships in different scenes. i The number of times and frequency of occurrence of the following.

[0106] Furthermore, fuzzy distance calculations are performed on the positional relationships of entities. Appropriate distances between models in a 3D scene are a key element for the naturalness of scene layout, but scene description text typically does not include the accurate distances between models.

[0107] This step uses fuzzy distance calculation to ensure the rationality and naturalness of entity positions within the scene. Distance calculation between scene entities generally falls into two categories: one is the distance calculation between two scene entities on the same support, and the other is the distance calculation between two scene entities on different supports. When the distance between scene entities is not explicitly given, for example, when the scene description text may contain vague terms like "very close" or "not far," d>0 is used. If terms like "placed" or "next to" indicate no distance but are difficult for the computer to understand, d=0 is used. Assuming the 3D coordinates of scene entity A are represented as A(x1,y1,z1) and the 3D coordinates of scene entity B are represented as B(x2,y2,z2), fuzzy distance calculation can be further divided into the following three cases:

[0108] (1) If d>0, the distance calculation of scene entities on different supports can be performed by distance inference based on other known scene entity distances. The specific formula is as follows:

[0109]

[0110] (2) When one scene entity supports another scene entity, the distance between them is represented by d = 0. The formula for calculating the distance between entity A and entity B on the same supporting surface is as follows:

[0111]

[0112] (3) When the distance cannot be estimated by existing scene entities, use spatial relationships (such as up, down, left, right, east, west, south, north, etc.) to estimate the approximate position of the object, and combine the support relationship of the object to accurately arrange the spatial layout of the scene entities. Finally, according to the principle of d=0, place the object on its own support to determine the fuzzy distance between different objects in order to achieve the best effect.

[0113] Finally, the JSON data returned by LLM is read, and the scene description returned by LLM is iteratively updated using the statistical prior probability and fuzzy distance calculation method to generate an optimized scene description file, ensuring the naturalness and realism of the subsequent 3D scene generation.

[0114] S3, 3D scene generation.

[0115] The system searches the local model library for the 3D models corresponding to the entities contained in the JSON file. Once the 3D models are found, the scene entities are converted into concrete 3D models and loaded into the Blender 3D engine for texture mapping. Next, the system reads the data information of the virtual scene to be constructed from the updated scene description file. This information describes the information of different entities in the scene and the relative positional relationships between them. Combined with the S2 fuzzy distance calculation results, the specific position of each entity in the scene is determined. Based on the calculated position information, each entity is placed in the correct position.

[0116] In summary, this invention provides a 3D scene generation method based on entity spatial relationship reasoning. It combines the reasoning capabilities of a large language model with task-specific learning capabilities to infer and summarize the spatial relationships between entities in text descriptions. In this way, we can transform text-described scenes into concrete 3D scenes, which can then be rendered and displayed in a 3D engine. This method can be widely applied in fields such as virtual reality, game development, and film production, and is of significant value in improving work efficiency and innovation capabilities.

[0117] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Unless otherwise specified, an element defined by the phrase "comprising..." or "including..." does not exclude the presence of additional elements in the process, method, article, or terminal device that includes said element. Additionally, in this document, "greater than," "less than," "exceeding," etc., are understood to exclude the stated number; "above," "below," "within," etc., are understood to include the stated number.

[0118] Although the above embodiments have been described, those skilled in the art, once they understand the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the above descriptions are merely embodiments of the present invention and do not limit the scope of patent protection of the present invention. Any equivalent structural or procedural transformations made using the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for generating a 3D scene based on entity spatial relationship reasoning, characterized in that, The method includes the following steps: Step S1: Use the large language model to receive the scene text input by the user, and complete the information extraction and information reasoning tasks of the input text. This text includes one or more entities and their spatial relationships. The large language model processes the scene text and outputs the entities in the scene and their spatial relationships. Step S2, spatial relationship optimization: the spatial relationships of entities returned by the large language model are optimized using the prior probability algorithm and the fuzzy distance algorithm to form a new scene description file; In step S2, the fuzzy distance algorithm calculates the entity positional relationships as follows: Distance calculation between scene entities is generally divided into two types: one is the distance calculation between two scene entities on the same support, and the other is the distance calculation between two scene entities on different supports. When the distance between scene entities is not explicitly given, and the scene description text contains words indicating distance but not explicitly stated, this is represented by d>0. If words indicating no distance appear but are difficult for the computer to understand, d=0 is used. Assume the three-dimensional coordinates of scene entity A are represented as follows: The three-dimensional coordinates of scene entity B are represented as follows: Fuzzy distance calculation can be divided into the following three cases: (1) If d>0, the distance calculation for scene entities on different supports can be based on the distance inference of other known scene entity distances. Assume the coordinates of scene entity A. The coordinates of scene entity B Below is the specific formula for calculation: (2) When one scene entity supports another scene entity, the distance between them is represented by d=0. For entities on the same supporting surface and The formula for calculating the distance between them is as follows: (3) When the distance cannot be estimated by existing scene entities, the approximate position of the object is estimated by using spatial relationships, and the spatial layout of the scene entities is precisely arranged by combining the support relationship of the object. Finally, according to the principle of d=0, the object is placed on its own support to determine the fuzzy distance between different objects in order to achieve the best effect. Step S3: Generate a 3D scene based on the scene description file.

2. The 3D scene generation method based on entity spatial relationship reasoning as described in claim 1, characterized in that, In step S1, the spatial relationship database is defined as 12 basic three-dimensional spatial position constraints {"up", "down", "left", "right", "front", "back", "middle", "edge", "inner", "outer", "near", "far"}, and then... The triplet provides a formal description of the spatial relationships between entities, where, and Represents entities within the scene. This refers to a spatial relationship type.

3. The 3D scene generation method based on entity spatial relationship reasoning as described in claim 2, characterized in that, In step S2, the set of all entity categories within the scene is defined as... Represents n distinct entities within a scene. The set of positional relationships between objects is defined as follows: , express Different positional relationships Construct a set of spatial relation triples and according to the set Construct a semantic scene graph, where the locative word R is used to represent the spatial relationship between element X and element Y, where X is the main component in the spatial relationship, and Y is the reference object for the orientation of scene entity X. When multiple spatial relationships exist in a scene, the following steps are used to traverse the spatial relationships between different entities and create a spatial relationship connectivity graph:

201. Before traversing, mark each vertex as visited, initially setting it to an unvisited state; 202. For the current vertex "v", mark it as visited; 203. Find the first neighboring vertex "w" of vertex "v"; 204. If vertex "w" has not been visited, then recursively execute the algorithm starting from vertex "w"; 205. Update the current vertex "w" to be the next neighbor of vertex "v"; 206. Repeat steps 204 and 205 until all adjacent vertices of vertex "v" have been visited; 207. Repeat steps 202 to 206 until all vertices have been visited; By traversing the spatial relationships between different entities and using the spatial relationship propagation method, a spatial relationship connectivity graph is formed, which contains the set of spatial relationships RT in the entire scene.

4. The 3D scene generation method based on entity spatial relationship reasoning as described in claim 3, characterized in that, Set of spatial relation triples based on input Use the following steps to generate a semantic scene graph:

211. Preprocess the input spatial relation triples RT; 212. Extract candidate spatial relation triples from the preprocessed spatial relation triples; 213. Sort and remove duplicates from candidate spatial relation triples using sorting and filtering functions; 214. Iterate through each preprocessed triplet. For each triplet "T"_i, perform the following operations: a. Call the "Expand" function to expand the current triple to achieve spatial relation propagation. b. Based on the expanded spatial relationship connectivity graph, call the "ssgGen" function to generate a semantic scene graph. c. Store the generated semantic scene graph.

215. Output the final generated semantic scene graph.

5. The 3D scene generation method based on entity spatial relationship reasoning as described in claim 4, characterized in that, The prior probability algorithm is as follows: Next, we will calculate the prior probability of scene entities appearing in different scenarios and the frequency of scene entities appearing in a certain scenario description. The specific formula is as follows: In the formula above, d represents the scene entity. This represents a certain type of scenario i. This indicates the number of times the hidden entity appears in the scene based on prior knowledge. This represents the expected number of scenarios of a certain type i. This represents the frequency with which the hidden entity d appears in a certain type of scenario i; Statistical probability of spatial relationships between entities The specific formula is as follows: r represents the relative positional relationship between scene entities. This indicates that, under the observation of m scene examples, the object With objects The number of relations r. This represents the number of relations r in scenario i, calculated by... It can clearly describe the spatial relationships between scene entities, and their relationships in different scenes. The number of times and frequency of occurrence of the following.

6. The 3D scene generation method based on entity spatial relationship reasoning as described in claim 1, characterized in that, Step S3 includes searching for the entity 3D model contained in the JSON file in the local model library, converting the scene entities into specific 3D models, importing them into the 3D engine, reading the scene information to be constructed in the scene description file, and realizing the placement of different entities in the scene.

Images