Three-dimensional scene generation method and device based on semantic analysis and graph neural network
By using semantic analysis and graph neural networks, 3D scenes are automatically generated, solving the problem of low efficiency in existing technologies and achieving efficient and accurate 3D scene generation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-10
- Publication Date
- 2026-03-20
AI Technical Summary
Existing 3D scene generation methods rely on manual operation, resulting in low generation efficiency and an inability to efficiently meet dynamic and complex spatial decision-making needs.
Using a semantic analysis and graph neural network-based approach, spatial semantic labels are extracted from natural language descriptions through a pre-trained spatial semantic feature extraction model, spatial operation chains are generated, and a spatial computing engine is used to perform data registration, computation, and rendering operations to automatically generate 3D scenes.
It significantly improves the efficiency of 3D scene generation, reduces reliance on professional skills and programming abilities, ensures the accuracy of complex scene generation and automated processes, and reduces manual processing time.
Smart Images
Figure CN121708232A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of 3D image generation technology, and in particular to a method and apparatus for generating 3D scenes based on semantic analysis and graph neural networks. Background Technology
[0002] The automatic generation technology of 3D scenes based on geographic tiling has become the mainstream paradigm for building city-level and regional-level 3D visualization scenes. It preprocesses large-scale geographic data into multi-level, block-based images or elevation tiles and performs efficient scheduling and fusion on the client side. This significantly improves the spatial cognition and decision-making efficiency in fields such as digital twins, smart cities, and emergency command.
[0003] Existing methods typically rely on specialized software and pre-defined rule bases, combining geographic analysis tools with 3D modeling workflows through manual operation to achieve static reconstruction and display of specific geographic phenomena. Such methods have significant limitations when dealing with dynamic and complex spatial decision-making needs. This sequential processing mode, dependent on manual intervention, consumes substantial time and manpower, resulting in low efficiency in 3D scene generation. Summary of the Invention
[0004] In view of this, the present invention provides a method and apparatus for generating three-dimensional scenes based on semantic analysis and graph neural networks, the main purpose of which is to solve the problem of low efficiency in existing three-dimensional scene generation.
[0005] According to one aspect of the present invention, a method for generating a 3D scene based on semantic analysis and graph neural networks is provided, comprising: Obtain the scene intent description information and multimodal geographic data of the scene to be generated; Based on the multimodal geographic data, spatial semantic labels for the scene intent description information are extracted using a pre-trained spatial semantic feature extraction model; A spatial operation chain is generated based on the spatial semantic tags; The spatial operation chain is mapped to the spatial computing engine interface, so that the data registration operation, spatial computing operation and rendering operation in the spatial operation chain are executed sequentially by the spatial computing engine to generate a 3D scene that matches the scene intent description information.
[0006] Furthermore, the pre-trained spatial semantic feature extraction model includes a large language model, a spatial semantic embedding layer, and a geographic logical constraint mechanism; The step of extracting spatial semantic labels for the scene intent description information based on the multimodal geographic data using a pre-trained spatial semantic feature extraction model includes: Spatial keywords and intent keywords are extracted from the scene intent description information using a large language model, wherein the spatial keywords include spatial entity keywords and spatial range keywords; Spatial features are extracted from the multimodal geographic data through the spatial semantic embedding layer to obtain spatial entities and spatial relationships between different spatial entities. The multimodal geographic data includes remote sensing images and sensor data. The spatial semantic embedding layer aligns the spatial keywords with the geospatial features to obtain geographic entities and spatial constraints, and generates decision targets through intent keywords. Spatial semantic labels are generated based on the geographic entities, the spatial constraints, and the decision objectives; In the process of spatial feature extraction and spatial semantic alignment, a geographic logic constraint mechanism is used to constrain the spatial feature reasoning path and the output probability of different spatial semantic labels so that the generated spatial semantic labels meet the geographic logic constraints.
[0007] Furthermore, before extracting the spatial semantic labels of the scene intent description information through the pre-trained spatial semantic feature extraction model, the method further includes: Text training samples were constructed based on geographical knowledge corpus, and multimodal geospatial training samples were constructed based on remote sensing images and sensor data. The spatial feature embedding layer is trained based on the multimodal geospatial training samples, and during the training process, the loss function of spatial feature extraction is constrained by the geographic logic constraint mechanism to obtain the trained spatial feature embedding layer. The trained spatial feature embedding layer is introduced into the large language model to obtain the initial spatial semantic feature extraction model; The initial spatial semantic feature extraction model is incrementally fine-tuned based on the text training samples and the multimodal geospatial training samples. During the training process, the loss function of spatial semantic generation is constrained by the geographic logic constraint mechanism to obtain the pre-trained spatial semantic feature extraction model.
[0008] Further, the step of generating a spatial operation chain based on the spatial semantic tags includes: Identify at least one target task template that matches the spatial semantic label from the predefined spatial analysis task templates; The geographic entities, spatial constraints, and decision objectives are filled into the corresponding parameter bits of the target task template to generate at least one executable operation sequence; A spatial operation chain is generated based on all the executable operation sequences described.
[0009] Further, mapping the spatial operation chain to the spatial computing engine interface, so as to sequentially execute the data registration operation, spatial computing operation, and rendering operation in the spatial operation chain through the spatial computing engine to generate a 3D scene matching the scene intent description information, includes: The data registration operation is called to invoke the data loading interface of the spatial computing engine to load the registration data matching each geographic entity in the spatial semantic tag. Invoke at least one target prediction model in the spatial computing engine that matches the spatial computing operation; The registration data is processed by the target prediction model to obtain the state prediction data of each geographic entity, wherein the data format of the state prediction data is GeoJSON or voxel grid. Based on the geographic entities and the state prediction data, a 3D scene is rendered by a rendering engine that matches the rendering operation.
[0010] Furthermore, when the target prediction model is constructed based on a graph neural network, the registration data includes the attribute data and real-time data of the geographic entities, as well as the connection paths between the geographic entities. The step of performing prediction processing on the registered data using the target prediction model to obtain state prediction data for each geographic entity includes: A graph structure that satisfies spatial constraints is constructed, with each geographic entity as a node, the connection path between geographic entities as an edge, the attribute data and real-time data of geographic entities as node attributes, and the length, capacity and topological direction of the connection path as edge attributes. The target prediction model extracts graph features from the graph structure and predicts the state prediction data of the geographic entities based on the extracted graph features. The state prediction data includes road flooding depth and / or intersection traffic risk.
[0011] Further, the step of rendering a 3D scene based on the geographic entities and the state prediction data by matching the rendering engine of the rendering operation includes: Retrieve the 3D real-world data of the geographic entity, and invoke the real-time neural graphics primitives that match the rendering operation, the neural radiation field network that incorporates the physical model, and the web-based graphics library. A three-dimensional radiation field is generated based on the three-dimensional real-world data using the instantaneous neural graphics primitives. The state prediction data is used as semantic injection into the neural radiation field network. Based on the three-dimensional radiation field, the neural radiation field network with the physical model is used to generate two-dimensional images and depth maps from any viewpoint. Based on the two-dimensional image and depth map from any given viewpoint, volume rendering is performed using a web-based graphics library to obtain a three-dimensional scene.
[0012] According to another aspect of the present invention, a three-dimensional scene generation apparatus based on semantic analysis and graph neural networks is provided, comprising: The acquisition module is used to acquire scene intent description information and multimodal geographic data of the scene to be generated; The extraction module is used to extract spatial semantic labels of the scene intent description information based on the multimodal geographic data using a pre-trained spatial semantic feature extraction model. An operation chain generation module is used to generate a spatial operation chain based on the spatial semantic tags; The scene generation module is used to map the spatial operation chain to the spatial computing engine interface, so that the spatial computing engine sequentially executes the data registration operation, spatial computing operation and rendering operation in the spatial operation chain to generate a 3D scene that matches the scene intent description information.
[0013] Furthermore, the pre-trained spatial semantic feature extraction model includes a large language model, a spatial semantic embedding layer, and a geographic logical constraint mechanism. The extraction module includes: The semantic feature extraction unit is used to extract spatial keywords and intent keywords from the scene intent description information through a large language model, wherein the spatial keywords include spatial entity keywords and spatial range keywords; The spatial feature extraction unit is used to extract spatial features from the multimodal geographic data through the spatial semantic embedding layer to obtain spatial entities and spatial relationships between different spatial entities, wherein the multimodal geographic data includes remote sensing images and sensor data. The spatial semantic alignment unit is used to perform spatial semantic alignment between the spatial keywords and geospatial features through the spatial semantic embedding layer to obtain geographic entities and spatial constraints, and to generate decision targets through intent keywords. The generation unit is used to generate spatial semantic labels based on the geographic entities, the spatial constraints, and the decision objectives; wherein, during the process of spatial feature extraction and spatial semantic alignment, the spatial feature reasoning path and the output probability of different spatial semantic labels are constrained based on the geographic logic constraint mechanism so that the generated spatial semantic labels meet the geographic logic constraints.
[0014] Furthermore, the device also includes: The first construction module is used to construct text training samples based on geographic knowledge corpus and multimodal geospatial training samples based on remote sensing images and sensor data. The first training module is used to train a spatial feature embedding layer based on the multimodal geospatial training samples, and during the training process, constrains the loss function of spatial feature extraction through a geographic logic constraint mechanism to obtain the trained spatial feature embedding layer. The second building module is used to introduce the trained spatial feature embedding layer into the large language model to obtain the initial spatial semantic feature extraction model. The second training module is used to incrementally fine-tune the initial spatial semantic feature extraction model based on the text training samples and the multimodal geospatial training samples. During the training process, the loss function of spatial semantic generation is constrained by the geographic logic constraint mechanism to obtain the pre-trained spatial semantic feature extraction model.
[0015] Furthermore, the generation module includes: The template recognition unit is used to identify at least one target task template that matches the spatial semantic label from predefined spatial analysis task templates; A fill unit is used to fill the geographic entity, the spatial constraints and the decision objective into the corresponding parameter bits of the target task template to generate at least one executable operation sequence; A generation unit is used to generate a spatial operation chain based on all the executable operation sequences.
[0016] Furthermore, the scene generation module includes: The data loading unit is used to call the data loading interface of the spatial computing engine according to the data registration operation, so as to load the registration data matching each geographic entity in the spatial semantic tag through the data loading interface; The calling unit is used to call at least one target prediction model in the spatial computing engine that matches the spatial computing operation; The prediction unit is used to perform prediction processing on the registration data through the target prediction model to obtain the state prediction data of each geographic entity, wherein the data format of the state prediction data is GeoJSON or voxel grid. The rendering unit is used to render a 3D scene based on the geographic entities and the state prediction data, by using a rendering engine that matches the rendering operation.
[0017] Furthermore, when the target prediction model is constructed based on a graph neural network, the registration data includes the attribute data and real-time data of the geographic entities, as well as the connection paths between the geographic entities. In a specific application scenario, the prediction unit is specifically used to construct a graph structure that satisfies spatial constraints, with each geographic entity as a node, the connection paths between geographic entities as edges, the attribute data and real-time data of the geographic entities as node attributes, and the length, capacity, and topological direction of the connection paths as edge attributes. The target prediction model extracts graph features from the graph structure and predicts the state prediction data of the geographic entities based on the extracted graph features, wherein the state prediction data includes road segment flooding depth and / or intersection traffic risk.
[0018] Furthermore, in specific application scenarios, the rendering unit is specifically used to retrieve the three-dimensional real-scene data of the geographic entity and call the real-time neural graphics primitives that match the rendering operation, the neural radiation field network that introduces the physical model, and the web-based graphics library. A three-dimensional radiation field is generated based on the three-dimensional real-world data using the instantaneous neural graphics primitives. The state prediction data is used as semantic injection into the neural radiation field network. Based on the three-dimensional radiation field, the neural radiation field network with the physical model is used to generate two-dimensional images and depth maps from any viewpoint. Based on the two-dimensional image and depth map from any given viewpoint, volume rendering is performed using a web-based graphics library to obtain a three-dimensional scene.
[0019] According to another aspect of the present invention, a storage medium is provided, wherein at least one executable instruction is stored therein, the executable instruction causing a processor to perform operations corresponding to the above-described method for generating a 3D scene based on semantic analysis and graph neural networks.
[0020] According to another aspect of the present invention, a terminal is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus; The memory is used to store at least one executable instruction, which causes the processor to perform the operations corresponding to the above-described 3D scene generation method based on semantic analysis and graph neural networks.
[0021] By employing the above-described technical solutions, the technical solutions provided by the embodiments of the present invention have at least the following advantages: This invention provides a method and apparatus for generating 3D scenes based on semantic analysis and graph neural networks. The embodiments of this invention acquire scene intent description information and multimodal geographic data of the scene to be generated; based on the multimodal geographic data, spatial semantic labels are extracted from the scene intent description information using a pre-trained spatial semantic feature extraction model; a spatial operation chain is generated based on the spatial semantic labels; the spatial operation chain is mapped to a spatial computing engine interface, so that the data registration operation, spatial computing operation, and rendering operation in the spatial operation chain are executed sequentially by the spatial computing engine to generate a 3D scene matching the scene intent description information. By driving the generation of complex 3D scenes through natural language description, the reliance on professional GIS software operation skills and programming capabilities is greatly reduced. The automated workflow significantly reduces the time required for manual data processing, model configuration, and parameter adjustment, while ensuring the accuracy of scene generation under complex scene requirements, thereby improving the efficiency of 3D scene generation.
[0022] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention, it can be implemented according to the contents of the specification. Furthermore, in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description
[0023] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 The flowchart of a three-dimensional scene generation method based on semantic analysis and graph neural network provided by an embodiment of the present invention is shown. Figure 2 The flowchart of another 3D scene generation method based on semantic analysis and graph neural network provided by an embodiment of the present invention is shown; Figure 3 The diagram shows a block diagram of a 3D scene generation device based on semantic analysis and graph neural networks provided in an embodiment of the present invention. Figure 4 A schematic diagram of the structure of a terminal provided in an embodiment of the present invention is shown. Detailed Implementation
[0024] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
[0025] To address the issue of low efficiency in existing 3D scene generation methods, this invention provides a 3D scene generation method based on semantic analysis and graph neural networks, such as... Figure 1 As shown, the method includes: 101. Obtain the scene intent description information and multimodal geographic data of the scene to be generated.
[0026] In this embodiment of the invention, the scene intent description information is a natural language instruction used to describe the goals and core elements of the expected virtual scene (the scene to be generated). This can include entities and objects (vehicles, buildings), spatial relationships (relative positions between entities), dynamic behaviors and interactions (how entities change and interact over time), environmental conditions (rainy weather), and higher-order semantic goals (simulating flooding range). A natural language instruction could be, "Predict the urban area with the highest flood risk in the next three days and simulate the flooding range." This scene intent description information is a natural language instruction input by the user through an interactive terminal; it can be text input or voice input, and this embodiment of the invention does not impose specific limitations. Multimodal geographic data can include optical images, such as two-dimensional RGB images taken by satellite, aerial, or street view cameras; IoT sensor data collected based on IoT sensors; structured geographic element data such as road networks, lane lines, traffic signs, building outlines, and land use types stored in the form of points, lines, and areas; and multi-source inputs such as business reports associated with urban planning and commercial site selection.
[0027] 102. Based on the multimodal geographic data, extract the spatial semantic labels of the scene intent description information using a pre-trained spatial semantic feature extraction model.
[0028] In this embodiment of the invention, to achieve the mapping from abstract intent to specific geographic space, the system needs to deeply understand the spatial structure and semantic elements contained in the description. Specifically, this requires a pre-trained spatial semantic feature extraction model. This model can be based on a natural language processing architecture (such as a multimodal large model based on the LLaMA architecture) and has been fully pre-trained on geographic information text, spatial relationship annotation data, and multimodal geographic datasets, enabling it to map textual vocabulary to geographic spatial concepts. The model's recognition process includes identifying key geographic entities in the text, such as "urban boundaries," "crossroads," and "river networks," and also extracting the spatial relationships and behavioral semantics between these entities. Through this deep spatial semantic understanding and association, the model outputs a set of structured, machine-readable spatial semantic labels. These labels accurately summarize the core spatial and semantic connotations of the scene's intent.
[0029] 103. Generate a spatial operation chain based on the spatial semantic tags.
[0030] In this embodiment of the invention, after obtaining spatial semantic tags, in order to achieve automated scene generation, these high-level semantic concepts are mapped and serialized into a series of low-level, computable, specific spatial operations. For example, for the intent of "simulating the risk of urban flooding due to rainstorms in the city center," the planning module understands that its core requirement starts from loading basic geographic data, through terrain analysis and hydrological modeling, to the final visualization. Therefore, it automatically generates an operation chain such as "loading high-resolution remote sensing images → fusion of elevation model data and extraction of topographic depressions → access to real-time or predicted rainfall data → simulation of urban surface hydrology and flood diffusion based on shallow water equations → three-dimensional dynamic visualization of inundation depth and range." This operation chain not only strictly defines the data and models required for each step, but also ensures the coherence of data flow (such as the output of the digital elevation model as the input of the hydrological model) and logic between steps, thereby systematically decomposing a complex scene intent into a precise automated workflow that can be executed step by step by the geocomputing engine.
[0031] 104. Map the spatial operation chain to the spatial computing engine interface, so that the data registration operation, spatial computing operation and rendering operation in the spatial operation chain are executed sequentially through the spatial computing engine to generate a 3D scene that matches the scene intent description information.
[0032] In this embodiment of the invention, after generating the spatial operation chain, it needs to be converted into specific computational instructions. This means precisely mapping each logical step in the operation chain to a specific application programming interface (API) of the underlying spatial computing engine (such as GeoTrellis, GDAL, PyTorch Geospatial, etc.). The mapping process is not simply instruction passing, but involves in-depth parameter adaptation and resource scheduling. For example, the operation chain "[Remote sensing image loading]" needs to be mapped to a call to the engine's loadImageryProvider interface, and the corresponding tile service URL and coordinate reference system need to be registered. Simultaneously, complex computational operations such as "[Flood diffusion simulation]" need to be mapped to a hydrodynamic model interface called from within or outside the engine, and elevation data, rainfall parameters, etc., need to be input. Subsequently, the spatial computing engine sequentially executes this sequence of instructions comprised of interface calls: First, it performs data registration, unifying multimodal geographic data (imagery, elevation data, vectors) from different sources into the same spatial coordinate system to form a precisely overlaid digital base; then, it executes core spatial computing operations, calling corresponding analysis operators and simulation models (such as terrain analysis and water flow diffusion algorithms) to perform numerical simulations and generate results such as flood inundation range and depth; finally, the engine initiates rendering operations, based on the computational results and original geographic data, calling graphics interfaces (such as WebGL and Three.js) to perform terrain shaping, texture mapping, water effect fusion, dynamic entity implantation, and lighting effects rendering, ultimately generating an immersive and interactive 3D scene on the output interface that highly matches the scene's intended description, thus completing the fully automated construction from abstract intent to concrete scene. This achieves end-to-end automation of "natural language input—AI understanding—spatial computing—neural rendering—Web output".
[0033] In one embodiment of the present invention, for further illustration and limitation, such as Figure 2 As shown, the step of extracting spatial semantic labels for the scene intent description information based on the multimodal geographic data using a pre-trained spatial semantic feature extraction model includes: 201. Extract spatial keywords and intent keywords from the scene intent description information using a large language model.
[0034] 202. Spatial features are extracted from the multimodal geographic data through the spatial semantic embedding layer to obtain spatial entities and spatial relationships between different spatial entities.
[0035] 203. The spatial keywords and geospatial features are spatially semantically aligned through the spatial semantic embedding layer to obtain geographic entities and spatial constraints, and decision targets are generated through intent keywords.
[0036] 204. Generate spatial semantic labels based on the geographic entities, the spatial constraints, and the decision objectives.
[0037] In this embodiment of the invention, the pre-trained spatial semantic feature extraction model includes a large language model, a spatial semantic embedding layer, and a geographic logical constraint mechanism. Spatial keywords include spatial entity keywords and spatial extent keywords. Multimodal geographic data includes remote sensing imagery and sensor data. During spatial feature extraction and spatial semantic alignment, the geographic logical constraint mechanism constrains the spatial feature inference path and the output probability of different spatial semantic labels to ensure that the generated spatial semantic labels satisfy the geographic logical constraints.
[0038] Specifically, firstly, a large language model is used to parse the scene intent description information, extracting spatial keywords (such as "urban area" and "river") and intent keywords (such as "simulation" and "risk"). Next, a spatial semantic embedding layer processes multimodal geographic data (such as remote sensing imagery and sensor data) to identify and extract spatial entities (such as roads and buildings) and their spatial relationships (such as adjacency and containment). Then, again through this spatial semantic embedding layer, the spatial keywords extracted by the language model are semantically aligned and matched with the spatial entity features extracted from the geographic data, thus accurately transforming them into specific geographic entities (such as "intersection of Renmin Road and Jiefang Road"), and parsing spatial constraints (such as "within a radius of 500 meters"). Simultaneously, a decision objective (such as "generate a flooding range map") is generated based on the intent keywords. Throughout the feature extraction and semantic alignment process, a geographic logic constraint mechanism continuously verifies and constrains the inference path and label generation probability.
[0039] It should be noted that by combining a large language model with a spatial semantic embedding layer, the system can deeply understand the complex scene intentions described by users in natural language, and automatically and accurately parse them into machine-understandable structured geographic entities, spatial constraints, and decision-making objectives, significantly reducing the technical threshold and manual costs of spatial scene construction. Simultaneously, by introducing a geographic logical constraint mechanism, the inference path and output probability are constrained during feature extraction and semantic alignment, effectively preventing spatial relationships between geographic entities from violating common sense and ensuring the accuracy of spatial semantic labels.
[0040] In one embodiment of the present invention, for further explanation and limitation, before the step of extracting the spatial semantic labels of the scene intent description information through a pre-trained spatial semantic feature extraction model, the method further includes: Text training samples were constructed based on geographical knowledge corpus, and multimodal geospatial training samples were constructed based on remote sensing images and sensor data. The spatial feature embedding layer is trained based on the multimodal geospatial training samples, and during the training process, the loss function of spatial feature extraction is constrained by the geographic logic constraint mechanism to obtain the trained spatial feature embedding layer. The trained spatial feature embedding layer is introduced into the large language model to obtain the initial spatial semantic feature extraction model; The initial spatial semantic feature extraction model is incrementally fine-tuned based on the text training samples and the multimodal geospatial training samples. During the training process, the loss function of spatial semantic generation is constrained by the geographic logic constraint mechanism to obtain the pre-trained spatial semantic feature extraction model.
[0041] In this embodiment of the invention, before initiating the spatial semantic label extraction process, it is necessary to construct training samples and an initial model, and train the spatial semantic feature extraction model. The training samples include text training samples constructed based on geographical knowledge text corpora and multimodal geospatial training samples containing spatial entities and their relationships, constructed based on unstructured data such as remote sensing images and sensor data. The geographical knowledge text corpora can be constructed based on geographical textbooks, planning reports, emergency manuals, spatial analysis scripts (SQL, Python), etc. Subsequently, based on the multimodal geospatial training samples, the spatial feature embedding layer is specifically trained. During training, a geographical logic constraint mechanism is used to constrain the loss function of spatial feature extraction, ensuring that the spatial features learned by the embedding layer (such as entity shapes and topological relationships) conform to basic geographical principles, thereby obtaining a trained spatial feature embedding layer that accurately represents geospatial information. Furthermore, the trained spatial feature embedding layer is integrated with a large language model to construct the initial spatial semantic feature extraction model. The initial model is then jointly incrementally fine-tuned using the LoRA (Low-Rank Adaptation) strategy, employing both text training samples and multimodal geospatial training samples. In this process, the loss function for spatial semantic generation is constrained again through a geographic logic constraint mechanism, enabling the model to accurately align natural language intent with multimodal geospatial features and output spatial semantic labels that conform to geographic logic. Through this systematic training process, a pre-trained model capable of deeply understanding spatial semantics is obtained.
[0042] In one embodiment of the present invention, for further explanation and limitation, the step of generating a spatial operation chain based on the spatial semantic tag includes: Identify at least one target task template that matches the spatial semantic label from the predefined spatial analysis task templates; The geographic entities, spatial constraints, and decision objectives are filled into the corresponding parameter bits of the target task template to generate at least one executable operation sequence; A spatial operation chain is generated based on all the executable operation sequences described.
[0043] In this embodiment of the invention, a target task template (e.g., a hydrological analysis template) that best matches the core intent described by spatial semantic tags (e.g., "flood simulation" or "site selection analysis") is retrieved and identified from a predefined spatial analysis task template library. Then, the specific geographic entities (e.g., "a certain watershed"), spatial constraints (e.g., "within a 500-meter buffer zone of the river channel") and decision objectives (e.g., "generating an inundation range map") parsed from the spatial semantic tags are used as parameters to fill the corresponding variable positions in the selected target task template, thereby instantiating and generating a specific, executable sequence of operations as a spatial operation chain. For example, remote sensing image loading → digital elevation model analysis → rainfall prediction model → flood diffusion simulation → result visualization.
[0044] It should be noted that the spatial semantic tags are parsed from a predefined spatial analysis task template library to identify multiple sub-tasks necessary to achieve the final decision-making goal. For example, the macro-level intent of "urban flood risk assessment" can be broken down into three core sub-tasks: "hazard analysis of disaster-causing factors," "exposure analysis of disaster-bearing bodies," and "comprehensive risk level classification." The parsed geographic entities, spatial constraints, and decision-making objectives are assigned parameters and filled into the corresponding parameter positions of each sub-task template, thus generating an independent, executable sequence of operations for each sub-task. These sub-task operation sequences are treated as an organic whole, and based on their data dependencies (e.g., the output of sub-task one is the input of sub-task two) and logical order, they are linked and integrated to assemble a complete spatial operation chain that can be sequentially executed by the spatial computing engine. By automatically breaking down a single, complex user intent into multiple logically related yet relatively independent operation tasks, a complete analysis process covering all aspects of the problem can be systematically constructed, avoiding modeling bias or omissions of key links due to the complexity of the intent, and significantly improving the accuracy of complex spatial analysis tasks.
[0045] In one embodiment of the invention, for further explanation and limitation, the step of mapping the spatial operation chain to a spatial computing engine interface, so as to sequentially execute the data registration operation, spatial computing operation, and rendering operation in the spatial operation chain through the spatial computing engine to generate a 3D scene matching the scene intent description information, includes: The data registration operation is called to invoke the data loading interface of the spatial computing engine to load the registration data matching each geographic entity in the spatial semantic tag. Invoke at least one target prediction model in the spatial computing engine that matches the spatial computing operation; The registered data is processed by the target prediction model to obtain the state prediction data of each geographic entity. Based on the geographic entities and the state prediction data, a 3D scene is rendered by a rendering engine that matches the rendering operation.
[0046] In this embodiment of the invention, based on the data registration operation in the operation chain, the data loading interface of the spatial computing engine is invoked to load registration data (such as remote sensing image tiles, building outline vector data, oblique photogrammetry model LiDAR point cloud, etc.) that precisely matches each geographic entity (such as "a certain watershed" or "urban buildings") in the spatial semantic tags and has been aligned with coordinate system 1 and geometry from local or online services. Next, the system invokes one or more target prediction models (such as risk index prediction based on time-series regression, hydrodynamic model, traffic and pedestrian flow prediction model based on graph neural network, land use extraction model based on convolutional neural network based on image segmentation) in the spatial computing engine corresponding to the spatial computing operation in the operation chain, and uses the registration data loaded in the previous step as input to drive these models to perform numerical simulation and prediction calculations, and finally obtain the state prediction data of each geographic entity under specific conditions (such as flood inundation depth and range, traffic density in future periods, etc.). Finally, based on the original geographic entities in the spatial semantic tags, the calculated state prediction data, and other environmental parameters, a rendering engine matching the rendering operation (such as a WebGL-based or Three.js-based graphics engine) is invoked. Through its scene graph interface, material system, and shaders, terrain shaping and entity instantiation are performed. By introducing physical models such as shallow water equations and ray tracing, dynamic phenomena such as flood evolution and solar radiation changes are visualized. The final rendered result is a dynamic and interactive 3D scene that highly matches the intent description information of the scene to be generated. The state prediction data is in GeoJSON or voxel mesh format.
[0047] It should be noted that the above-mentioned method for generating 3D scenes can be used to perform parametric simulations of various emergency response plans (such as flood control and evacuation route planning); the risk value, response time, and resource consumption of different plans can be automatically calculated based on AI prediction models. After generating the 3D scene, users can also interactively adjust parameters on the 3D sandbox and view the simulation effect in real time.
[0048] In one embodiment of the present invention, for further explanation and limitation, the step of performing prediction processing on the registration data through the target prediction model to obtain state prediction data for each geographic entity includes: A graph structure that satisfies spatial constraints is constructed, with each geographic entity as a node, the connection path between geographic entities as an edge, the attribute data and real-time data of geographic entities as node attributes, and the length, capacity and topological direction of the connection path as edge attributes. The target prediction model extracts graph features from the graph structure and predicts the state prediction data of the geographic entities based on the extracted graph features.
[0049] In this embodiment of the invention, when the target prediction model is constructed based on a graph neural network, the registration data includes attribute data and real-time data of geographic entities, as well as the connection paths between geographic entities. State prediction data includes road segment flooding depth and / or intersection traffic risk. Based on the geographic entities and their spatial relationships defined in the spatial semantic tags, a structured spatial graph model is constructed. Each independent geographic entity (such as a road segment, river corridor, or building block) is abstracted as a node, and the spatial connection relationships between them (such as road connectivity and water flow direction) are abstracted as edges of the graph. In this graph structure, the attributes of nodes are composed of static attribute data of geographic entities (such as land cover type and building height) and dynamic real-time data (such as real-time rainfall and traffic flow), while the attributes of edges are defined by the physical length of the connection path, functional capacity (such as the number of lanes and river flood control capacity), and topological direction (such as one-way traffic and water flow direction).
[0050] In the target prediction model constructed using graph neural networks, a multi-layer message passing mechanism is used to extract deep features from the constructed graph structure, effectively aggregating the correlation information between the node's own features and its neighborhood environment. This allows for the accurate capture of the propagation patterns of spatial phenomena such as flood risk and traffic congestion. Finally, the model performs state inference based on the learned graph features, outputting state prediction data that conforms to geographical evolution patterns, such as estimated inundation depth at the road segment level and quantitative indicators like the traffic risk level of intersection units. This provides accurate data support for the subsequent dynamic rendering of 3D scenes.
[0051] In one embodiment of the present invention, for further explanation and limitation, the step of rendering a 3D scene based on the geographic entity and the state prediction data by matching the rendering engine of the rendering operation includes: Retrieve the 3D real-world data of the geographic entity, and invoke the real-time neural graphics primitives that match the rendering operation, the neural radiation field network that incorporates the physical model, and the web-based graphics library. A three-dimensional radiation field is generated based on the three-dimensional real-world data using the instantaneous neural graphics primitives. The state prediction data is used as semantic injection into the neural radiation field network. Based on the three-dimensional radiation field, the neural radiation field network with the physical model is used to generate two-dimensional images and depth maps from any viewpoint. Based on the two-dimensional image and depth map from any given viewpoint, volume rendering is performed using a web-based graphics library to obtain a three-dimensional scene.
[0052] In this embodiment of the invention, 3D real-world data (such as oblique photogrammetry models, point clouds, etc.) corresponding to geographic entities is retrieved, and real-time neural graphics primitives for matching rendering operations, a neural radiation field (NeRF) network incorporating a physical model, and a web-based graphics library (such as WebGL) are invoked. Through real-time neural graphics primitives, a basic 3D radiation field is rapidly generated based on the 3D real-world data, achieving efficient encoding of scene geometry and appearance. Subsequently, state prediction data (such as flooding depth and passage risk) is injected as semantics into the neural radiation field network incorporating a physical model. This network integrates physical laws (such as the absorption and scattering of light in water) into the basic 3D radiation field, dynamically generating high-quality 2D images containing the predicted state and corresponding depth maps from any viewpoint. Finally, based on the web-based graphics library, volume rendering is performed using the generated 2D images and depth maps, fusing semantic information with 3D geometric depth. Ultimately, a high-fidelity, interactive 3D scene incorporating dynamically predicted states is rendered in real-time on the web, supporting real-time rotation, scaling, and viewpoint roaming.
[0053] This invention provides a 3D scene generation method based on semantic analysis and graph neural networks. The method involves acquiring scene intent description information and multimodal geographic data of the scene to be generated; extracting spatial semantic labels from the scene intent description information using a pre-trained spatial semantic feature extraction model based on the multimodal geographic data; generating a spatial operation chain based on the spatial semantic labels; mapping the spatial operation chain to a spatial computing engine interface; and sequentially executing data registration, spatial computing, and rendering operations within the spatial operation chain through the spatial computing engine to generate a 3D scene matching the scene intent description information. By driving the generation of complex 3D scenes through natural language description, this method significantly reduces reliance on professional GIS software operation skills and programming abilities. The automated workflow significantly reduces the time required for manual data processing, model configuration, and parameter adjustment, while ensuring the accuracy of scene generation under complex scene requirements, thereby improving the efficiency of 3D scene generation.
[0054] Furthermore, as a response to the above Figure 1 The implementation of the method shown in this invention provides a 3D scene generation device based on semantic analysis and graph neural networks. This device can be applied to the WeGIS intelligent analysis platform, such as... Figure 3 As shown, the device includes: The acquisition module 31 is used to acquire scene intent description information and multimodal geographic data of the scene to be generated; Extraction module 32 is used to extract spatial semantic labels of the scene intent description information based on the multimodal geographic data and through a pre-trained spatial semantic feature extraction model; Operation chain generation module 33 is used to generate a spatial operation chain based on the spatial semantic tags; The scene generation module 34 is used to map the spatial operation chain to the spatial computing engine interface, so as to sequentially execute the data registration operation, spatial computing operation and rendering operation in the spatial operation chain through the spatial computing engine to generate a 3D scene that matches the scene intent description information.
[0055] Furthermore, the pre-trained spatial semantic feature extraction model includes a large language model, a spatial semantic embedding layer, and a geographic logical constraint mechanism. The extraction module 32 includes: The semantic feature extraction unit is used to extract spatial keywords and intent keywords from the scene intent description information through a large language model, wherein the spatial keywords include spatial entity keywords and spatial range keywords; The spatial feature extraction unit is used to extract spatial features from the multimodal geographic data through the spatial semantic embedding layer to obtain spatial entities and spatial relationships between different spatial entities, wherein the multimodal geographic data includes remote sensing images and sensor data. The spatial semantic alignment unit is used to perform spatial semantic alignment between the spatial keywords and geospatial features through the spatial semantic embedding layer to obtain geographic entities and spatial constraints, and to generate decision targets through intent keywords. The generation unit is used to generate spatial semantic labels based on the geographic entities, the spatial constraints, and the decision objectives; wherein, during the process of spatial feature extraction and spatial semantic alignment, the spatial feature reasoning path and the output probability of different spatial semantic labels are constrained based on the geographic logic constraint mechanism so that the generated spatial semantic labels meet the geographic logic constraints.
[0056] Furthermore, the device also includes: The first construction module is used to construct text training samples based on geographic knowledge corpus and multimodal geospatial training samples based on remote sensing images and sensor data. The first training module is used to train a spatial feature embedding layer based on the multimodal geospatial training samples, and during the training process, constrains the loss function of spatial feature extraction through a geographic logic constraint mechanism to obtain the trained spatial feature embedding layer. The second building module is used to introduce the trained spatial feature embedding layer into the large language model to obtain the initial spatial semantic feature extraction model. The second training module is used to incrementally fine-tune the initial spatial semantic feature extraction model based on the text training samples and the multimodal geospatial training samples. During the training process, the loss function of spatial semantic generation is constrained by the geographic logic constraint mechanism to obtain the pre-trained spatial semantic feature extraction model.
[0057] Furthermore, the operation chain generation module 33 includes: The template recognition unit is used to identify at least one target task template that matches the spatial semantic label from predefined spatial analysis task templates; A fill unit is used to fill the geographic entity, the spatial constraints and the decision objective into the corresponding parameter bits of the target task template to generate at least one executable operation sequence; A generation unit is used to generate a spatial operation chain based on all the executable operation sequences.
[0058] Furthermore, the scene generation module 34 includes: The data loading unit is used to call the data loading interface of the spatial computing engine according to the data registration operation, so as to load the registration data matching each geographic entity in the spatial semantic tag through the data loading interface; The calling unit is used to call at least one target prediction model in the spatial computing engine that matches the spatial computing operation; The prediction unit is used to perform prediction processing on the registration data through the target prediction model to obtain the state prediction data of each geographic entity, wherein the data format of the state prediction data is GeoJSON or voxel grid. The rendering unit is used to render a 3D scene based on the geographic entities and the state prediction data, by using a rendering engine that matches the rendering operation.
[0059] Furthermore, when the target prediction model is constructed based on a graph neural network, the registration data includes the attribute data and real-time data of the geographic entities, as well as the connection paths between the geographic entities. In a specific application scenario, the prediction unit is specifically used to construct a graph structure that satisfies spatial constraints, with each geographic entity as a node, the connection paths between geographic entities as edges, the attribute data and real-time data of the geographic entities as node attributes, and the length, capacity, and topological direction of the connection paths as edge attributes. The target prediction model extracts graph features from the graph structure and predicts the state prediction data of the geographic entities based on the extracted graph features, wherein the state prediction data includes road segment flooding depth and / or intersection traffic risk.
[0060] Furthermore, in specific application scenarios, the rendering unit is specifically used to retrieve the three-dimensional real-scene data of the geographic entity and call the real-time neural graphics primitives that match the rendering operation, the neural radiation field network that introduces the physical model, and the web-based graphics library. A three-dimensional radiation field is generated based on the three-dimensional real-world data using the instantaneous neural graphics primitives. The state prediction data is used as semantic injection into the neural radiation field network. Based on the three-dimensional radiation field, the neural radiation field network with the physical model is used to generate two-dimensional images and depth maps from any viewpoint. Based on the two-dimensional image and depth map from any given viewpoint, volume rendering is performed using a web-based graphics library to obtain a three-dimensional scene.
[0061] This invention provides a 3D scene generation device based on semantic analysis and graph neural networks. In this embodiment, the device acquires scene intent description information and multimodal geographic data of the scene to be generated; based on the multimodal geographic data, it extracts spatial semantic labels from the scene intent description information using a pre-trained spatial semantic feature extraction model; it generates a spatial operation chain based on the spatial semantic labels; and it maps the spatial operation chain to a spatial computing engine interface, so that the spatial computing engine sequentially executes data registration, spatial computing, and rendering operations in the spatial operation chain to generate a 3D scene matching the scene intent description information. By driving the generation of complex 3D scenes through natural language description, the device significantly reduces the reliance on professional GIS software operation skills and programming abilities. The automated workflow significantly reduces the time required for manual data processing, model configuration, and parameter adjustment, while ensuring the accuracy of scene generation under complex scene requirements, thereby improving the efficiency of 3D scene generation.
[0062] According to one embodiment of the present invention, a storage medium is provided, the storage medium storing at least one executable instruction, the computer-executable instruction being able to execute the three-dimensional scene generation method based on semantic analysis and graph neural networks in any of the above method embodiments.
[0063] Figure 4 The diagram shows a structural schematic of a terminal according to an embodiment of the present invention. The specific implementation of the present invention is not limited to the specific implementation of the terminal.
[0064] like Figure 4 As shown, the terminal may include: a processor 402, a communication interface 404, a memory 406, and a communication bus 408.
[0065] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408.
[0066] Communication interface 404 is used for network communication with other devices such as clients or other servers.
[0067] The processor 402 is used to execute program 410, specifically to perform the relevant steps in the above-described embodiment of the 3D scene generation method based on semantic analysis and graph neural networks.
[0068] Specifically, program 410 may include program code that includes computer operation instructions.
[0069] Processor 402 may be a central processing unit (CPU), a specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The terminal may include one or more processors of the same type, such as one or more CPUs; or it may include processors of different types, such as one or more CPUs and one or more ASICs.
[0070] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.
[0071] Specifically, program 410 can be used to cause processor 402 to perform the following operations: Obtain the scene intent description information and multimodal geographic data of the scene to be generated; Based on the multimodal geographic data, spatial semantic labels for the scene intent description information are extracted using a pre-trained spatial semantic feature extraction model; A spatial operation chain is generated based on the spatial semantic tags; The spatial operation chain is mapped to the spatial computing engine interface, so that the data registration operation, spatial computing operation and rendering operation in the spatial operation chain are executed sequentially by the spatial computing engine to generate a 3D scene that matches the scene intent description information.
[0072] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.
[0073] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for generating 3D scenes based on semantic analysis and graph neural networks, characterized in that, include: Obtain the scene intent description information and multimodal geographic data of the scene to be generated; Based on the multimodal geographic data, spatial semantic labels for the scene intent description information are extracted using a pre-trained spatial semantic feature extraction model; A spatial operation chain is generated based on the spatial semantic tags; The spatial operation chain is mapped to the spatial computing engine interface, so that the data registration operation, spatial computing operation and rendering operation in the spatial operation chain are executed sequentially by the spatial computing engine to generate a 3D scene that matches the scene intent description information.
2. The method according to claim 1, characterized in that, The pre-trained spatial semantic feature extraction model includes a large language model, a spatial semantic embedding layer, and a geographic logical constraint mechanism. The step of extracting spatial semantic labels for the scene intent description information based on the multimodal geographic data using a pre-trained spatial semantic feature extraction model includes: Spatial keywords and intent keywords are extracted from the scene intent description information using a large language model, wherein the spatial keywords include spatial entity keywords and spatial range keywords; Spatial features are extracted from the multimodal geographic data through the spatial semantic embedding layer to obtain spatial entities and spatial relationships between different spatial entities. The multimodal geographic data includes remote sensing images and sensor data. The spatial semantic embedding layer aligns the spatial keywords with the geospatial features to obtain geographic entities and spatial constraints, and generates decision targets through intent keywords. Spatial semantic labels are generated based on the geographic entities, the spatial constraints, and the decision objectives; In the process of spatial feature extraction and spatial semantic alignment, a geographic logic constraint mechanism is used to constrain the spatial feature reasoning path and the output probability of different spatial semantic labels so that the generated spatial semantic labels meet the geographic logic constraints.
3. The method according to claim 2, characterized in that, Before extracting the spatial semantic labels of the scene intent description information through the pre-trained spatial semantic feature extraction model, the method further includes: Text training samples were constructed based on geographical knowledge corpus, and multimodal geospatial training samples were constructed based on remote sensing images and sensor data. The spatial feature embedding layer is trained based on the multimodal geospatial training samples, and during the training process, the loss function of spatial feature extraction is constrained by the geographic logic constraint mechanism to obtain the trained spatial feature embedding layer. The trained spatial feature embedding layer is introduced into the large language model to obtain the initial spatial semantic feature extraction model; The initial spatial semantic feature extraction model is incrementally fine-tuned based on the text training samples and the multimodal geospatial training samples. During the training process, the loss function of spatial semantic generation is constrained by the geographic logic constraint mechanism to obtain the pre-trained spatial semantic feature extraction model.
4. The method according to claim 2, characterized in that, The step of generating a spatial operation chain based on the spatial semantic tags includes: Identify at least one target task template that matches the spatial semantic label from the predefined spatial analysis task templates; The geographic entities, spatial constraints, and decision objectives are filled into the corresponding parameter bits of the target task template to generate at least one executable operation sequence; A spatial operation chain is generated based on all the executable operation sequences described.
5. The method according to claim 1, characterized in that, The step of mapping the spatial operation chain to the spatial computing engine interface, so as to sequentially execute the data registration operation, spatial computing operation, and rendering operation in the spatial operation chain through the spatial computing engine, and generate a 3D scene that matches the scene intent description information, includes: The data registration operation is called to invoke the data loading interface of the spatial computing engine to load the registration data matching each geographic entity in the spatial semantic tag. Invoke at least one target prediction model in the spatial computing engine that matches the spatial computing operation; The registered data is processed by the target prediction model to obtain the state prediction data of each geographic entity, wherein the data format of the state prediction data is GeoJSON or voxel grid. Based on the geographic entities and the state prediction data, a 3D scene is rendered by a rendering engine that matches the rendering operation.
6. The method according to claim 5, characterized in that, When the target prediction model is constructed based on a graph neural network, the registration data includes the attribute data and real-time data of the geographic entities, as well as the connection paths between the geographic entities. The step of performing prediction processing on the registered data using the target prediction model to obtain state prediction data for each geographic entity includes: A graph structure that satisfies spatial constraints is constructed, with each geographic entity as a node, the connection path between geographic entities as an edge, the attribute data and real-time data of geographic entities as node attributes, and the length, capacity and topological direction of the connection path as edge attributes. The target prediction model extracts graph features from the graph structure and predicts the state prediction data of the geographic entities based on the extracted graph features. The state prediction data includes road flooding depth and / or intersection traffic risk.
7. The method according to claim 6, characterized in that, The step of rendering a 3D scene based on the geographic entities and the state prediction data by matching the rendering engine of the rendering operation includes: Retrieve the 3D real-world data of the geographic entity, and invoke the real-time neural graphics primitives that match the rendering operation, the neural radiation field network that incorporates the physical model, and the web-based graphics library. A three-dimensional radiation field is generated based on the three-dimensional real-world data using the instantaneous neural graphics primitives. The state prediction data is used as semantic injection into the neural radiation field network. Based on the three-dimensional radiation field, the neural radiation field network with the physical model is used to generate two-dimensional images and depth maps from any viewpoint. Based on the two-dimensional image and depth map from any given viewpoint, volume rendering is performed using a web-based graphics library to obtain a three-dimensional scene.
8. A 3D scene generation device based on semantic analysis and graph neural networks, characterized in that, include: The acquisition module is used to acquire scene intent description information and multimodal geographic data of the scene to be generated; The extraction module is used to extract spatial semantic labels of the scene intent description information based on the multimodal geographic data using a pre-trained spatial semantic feature extraction model. An operation chain generation module is used to generate a spatial operation chain based on the spatial semantic tags; The scene generation module is used to map the spatial operation chain to the spatial computing engine interface, so that the spatial computing engine sequentially executes the data registration operation, spatial computing operation and rendering operation in the spatial operation chain to generate a 3D scene that matches the scene intent description information.
9. A storage medium, characterized in that, The storage medium stores at least one executable instruction that causes the processor to perform the operation corresponding to the three-dimensional scene generation method based on semantic analysis and graph neural networks as described in any one of claims 1-7.
10. A terminal, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the three-dimensional scene generation method based on semantic analysis and graph neural networks as described in any one of claims 1-7.