Semantic map and world model construction method based on 3DGS map and related device

Abstract

The application discloses a semantic map and world model construction method based on a 3DGS map and related equipment, 3DGS map data and asset data of a target scene are acquired, semantic objects containing category information and spatial position information are generated based on the map data, candidate binding relationships are generated based on the category information, position information and asset data to calculate binding confidence, candidate spatial relationships are generated based on the position information, and candidate constraint information is generated in combination with preset constraint data and the category information, and the three constitute candidate semantic association data, the candidate semantic association data is updated to obtain semantic association data according to a confirmation operation result, and map version information is marked to the semantic association data to generate semantic world model data. The automatic association generation of 3DGS map semantic objects, assets and spatial constraint information is realized, and the semantic world model supporting task execution of robots, unmanned aerial vehicles and the like is obtained.

Description

Technical Field

[0001] This application relates to the field of three-dimensional spatial data processing technology, and in particular to a method and related equipment for constructing semantic maps and world models based on 3DGS maps. Background Technology

[0002] With the development of 3D reconstruction and autonomous robot operation technologies, high-fidelity spatial reconstruction methods based on 3D Gaussian Splatting (3DGS) have been gradually applied to 3D modeling of industrial plants, power facilities, warehousing and logistics scenarios due to their advantages such as fast rendering speed and high restoration accuracy. Existing technologies typically generate 3DGS maps of the scene through multi-view image training, and then combine this with image recognition and other methods to label objects such as equipment and areas in the scene. This aims to integrate 3D space with systems such as enterprise asset management, inspection operations, and digital twin displays to support applications such as robot inspection, remote maintenance, and navigation.

[0003] However, existing 3DGS maps and related processing methods primarily focus on the geometric reconstruction of scene appearance. The generated 3D models themselves do not contain information such as equipment identity, asset ownership, spatial relationships, and operational constraints. To use them for robot task execution, digital twin management, or on-site personnel operations, extensive manual or additional system semantic annotation and data completion are still required, a cumbersome process with uncertain accuracy. When different inspection platforms, robot systems, and operational assistance terminals are applied based on the same 3DGS map, they often need to repeatedly perform semantic recognition and verification, resulting in resource waste, integration difficulties, and hindering the effective accumulation and continuation of on-site operational experience. Therefore, how to enable maps to possess semantic expression capabilities suitable for practical business scenarios, based on high-fidelity 3DGS spatial reconstruction, has become a pressing technical problem to be solved. Summary of the Invention

[0004] The main purpose of this application is to solve the technical problem that existing 3DGS map construction methods do not combine semantic objects, asset binding relationships and spatial business constraints for unified modeling, which makes it difficult for maps to directly support robot task execution and personnel on-site operation assistance.

[0005] This application provides a method for constructing a semantic map and world model based on a 3DGS map. The method includes: acquiring 3DGS map data and asset data corresponding to a target scene, wherein the 3DGS map data includes map version information; generating multiple semantic objects based on the 3DGS map data, wherein the semantic objects include category information and spatial location information; calculating the binding confidence between the semantic objects and the asset data based on the category information and spatial location information of the semantic objects and the asset data, and generating candidate binding relationships based on the binding confidence; generating candidate spatial relationships between the semantic objects based on the spatial location information of each semantic object, and generating candidate constraint information corresponding to the semantic objects based on preset constraint relationship data and the category information of the semantic objects, wherein the candidate binding relationships, the candidate spatial relationships, and the candidate constraint information constitute candidate semantic association data; acquiring the confirmation operation result corresponding to the candidate semantic association data, and updating the candidate semantic association data according to the confirmation operation result to obtain semantic association data; and marking the map version information to the semantic association data to generate semantic world model data.

[0006] A second aspect of this application provides a semantic map and world model construction device based on 3DGS maps. The device includes: a data acquisition module for acquiring 3DGS map data and asset data corresponding to a target scene, wherein the 3DGS map data includes map version information; a semantic generation module for generating multiple semantic objects based on the 3DGS map data, wherein the semantic objects include category information and spatial location information; and a calculation module for calculating the binding confidence between the semantic objects and the asset data based on the category information and spatial location information of the semantic objects and the asset data, and generating candidate semantic objects based on the binding confidence. The system includes: a binding relationship module, a relationship generation module, and a data update module. The data update module is used to obtain the confirmation operation result corresponding to the candidate semantic association data and update the candidate semantic association data according to the confirmation operation result to obtain semantic association data. A tagging module is used to tag the map version information to the semantic association data to generate semantic world model data.

[0007] A third aspect of this application provides a semantic map and world model construction device based on 3DGS maps, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the semantic map and world model construction device based on 3DGS maps to perform the various steps of the semantic map and world model construction method based on 3DGS maps described above.

[0008] A fourth aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the steps of the above-described method for constructing a semantic map and world model based on a 3DGS map.

[0009] Compared with the prior art, the advantages of this application are as follows: This application acquires 3DGS map data and asset data corresponding to the target scene, wherein the map data includes map version information; generates semantic objects containing category information and spatial location information based on the 3DGS map data; calculates binding confidence by combining the category information, spatial location information and asset data of the semantic objects, and generates candidate binding relationships; generates candidate spatial relationships based on the spatial location information of the semantic objects, and generates candidate constraint information by combining preset constraint relationship data and category information, the three of which together constitute candidate semantic association data; obtains the confirmation operation results for the candidate semantic association data, and updates the candidate data accordingly to obtain semantic association data; marks the map version information to the semantic association data to generate semantic world model data. This application achieves the transformation from a pure geometric 3DGS map to a deliverable and traceable semantic world model by sequentially overlaying semantic object generation, asset binding assessment, spatial relationship modeling, and constraint information generation on a 3DGS high-fidelity spatial map, and introducing a manual confirmation closed-loop and map version association mechanism. It solves the technical problem that existing 3DGS map construction methods do not combine semantic objects, asset binding relationships, and spatial business constraints for unified modeling, making it difficult for maps to directly support robot task execution and personnel on-site operation assistance. It effectively improves the accuracy and reliability of semantic annotation and asset binding results, and ensures the consistency and integrability of data when different downstream systems share the same semantic world model base. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of the first embodiment of the semantic map and world model construction method based on 3DGS map in this application; Figure 2 This is a schematic diagram of an embodiment of the semantic map and world model construction device based on 3DGS maps in this application. Figure 3 This is a schematic diagram of an embodiment of a semantic map and world model construction device based on 3DGS maps in this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0013] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of each component in a specific posture. If the specific posture changes, the directional indicators will also change accordingly.

[0014] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one feature. Additionally, the use of "and / or" or "and / or" throughout the text includes three parallel solutions. For example, "A and / or B" includes solution A, solution B, or a solution that simultaneously satisfies A and B. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0015] This application discloses a method and related equipment for constructing semantic maps and world models based on 3DGS maps. To make the technical solution, purpose, and effects of this application clearer, the specific implementation methods of this application are described in detail below in conjunction with specific implementation scenarios. These implementation methods are intended to further explain the technical concept of this application, rather than to limit the scope of protection of this application. All equivalent modifications made based on the technical solution of this application should be included within the scope of protection of this application.

[0016] refer to Figure 1 This application provides a method for constructing a semantic map and world model based on a 3DGS map, including the following steps: 101. Obtain the 3DGS map data and asset data corresponding to the target scene. The 3DGS map data includes map version information. In this embodiment, a handheld or mobile industrial camera is used to take panoramic photos along the passageways and work areas within the target scene, capturing images of equipment, cabinets, areas, etc., from multiple angles. This yields multiple observational view images covering different aspects of the target scene (observational view images refer to two-dimensional images of the same area in the target scene acquired from different shooting positions and angles, used to record the appearance information of the area under different viewing directions). The acquired multiple observational view images are input into a 3D reconstruction program. First, image feature matching is used to determine the shooting position and shooting angle corresponding to each observational view image, obtaining the 3D coordinate system of the target scene. Then, based on this 3D coordinate system and the observational view images, 3DGS map data composed of multiple 3D Gaussian primitives is trained and generated (3D Gaussian primitives are...). In the 3D Gaussian sputtering method, the basic unit used to represent the appearance of three-dimensional space is the 3D Gaussian primitive. Each 3D Gaussian primitive carries position coordinates, shape range, color, and transparency information. Multiple 3D Gaussian primitives are superimposed and rendered to present the 3D appearance of the target scene. After the 3DGS map data is generated, the reconstruction program automatically assigns corresponding map version information to the reconstruction result according to the generation time (map version information is used to identify the generation batch of the 3DGS map data; for example, two batches of 3DGS map data reconstructed at different times for the same power distribution room will have different assigned map version information). This ensures that the semantic objects and constraint information generated based on the map data can clearly correspond to a specific reconstruction result, avoiding the confusion and use of multiple batches of 3DGS map data reconstructed at different times. Simultaneously, corresponding asset data is exported from the asset management system to which the target scene belongs. The asset data includes asset number, asset name, asset type, and region fields, providing a comparison basis for subsequent binding between semantic objects and asset data.

[0017] 102. Generate multiple semantic objects based on 3DGS map data. The semantic objects include category information and spatial location information. In this embodiment, generating multiple semantic objects based on 3DGS map data includes: extracting image features from multiple observation view images corresponding to the 3DGS map data to obtain candidate semantic evidence, which includes category information, position information in the corresponding observation view image, and reliability weight of the corresponding observation view; dividing multiple candidate semantic evidence in each observation view image, where the position information satisfies a preset position similarity condition, into a candidate semantic instance; determining the category information of the candidate semantic instance based on the category information and corresponding reliability weight of each candidate semantic evidence assigned to the candidate semantic instance; and determining the spatial position information of the candidate semantic instance in the 3DGS map data based on the position information of each candidate semantic evidence assigned to the candidate semantic instance and the coordinate system corresponding to the 3DGS map data, thereby obtaining a semantic object including category information and spatial position information.

[0018] Specifically, for multiple observation view images corresponding to the 3DGS map data, each image is input into a pre-trained image target detection program to identify targets such as switch cabinets, meters, and access points in the images, obtaining the category label of the target in that image, and simultaneously outputting the detection box range corresponding to the target in that image (the detection box range refers to the rectangular area, expressed in image pixel coordinates, that defines the position of the target in the two-dimensional image); then, the camera shooting position and shooting angle calculated for that image during the 3DGS map data training process are obtained (this camera shooting position and shooting angle are calculated by the 3D Gaussian sputtering method during the training to generate the 3DGS map data for each observation view image). The data obtained after calculating the spatial position and orientation of the corresponding camera (not acquired separately through ranging or positioning devices) is used to correct the category confidence of the target detection program based on the distance between the shooting position and the detection box range of the target, as well as the angle between the camera orientation and the center direction of the detection box range: the farther the shooting distance, or the larger the angle between the camera orientation and the center direction of the detection box range, the lower the corrected reliability weight; the closer the shooting distance, and the closer the camera orientation is to being directly opposite the center direction of the detection box range, the higher the corrected reliability weight. This yields the reliability weight of the target at that observation viewpoint (the reliability weight represents the credibility of the image in classifying the target). The above category label, detection box range, and reliability weight are collectively used as a candidate semantic evidence (candidate semantic evidence refers to target clues identified from a single observation viewpoint image that have not yet been merged and confirmed with information from other viewpoints). For example, for the same switch cabinet in a power distribution room, one candidate semantic evidence is obtained from each of three images taken from different shooting angles. The image taken at close range directly facing the switch cabinet has a higher reliability weight than the image taken at a distance from the side. This process ensures that each image independently provides target clues and includes a credibility rating, avoiding direct determination of the target category due to occlusion, reflection, or skewed shooting angle in a single image. This provides comparable basic data for subsequent cross-view merging and verification.

[0019] For candidate semantic evidence from images from different observation perspectives, based on the camera shooting position, shooting angle, and the three-dimensional coordinate system corresponding to the 3DGS map data, the directional range starting from the camera shooting position and passing through the detection box is determined. The three-dimensional Gaussian elements (three-dimensional Gaussian elements are the basic units used to express the appearance of three-dimensional space in the 3D Gaussian sputtering method, and each three-dimensional Gaussian element carries position coordinates, shape range, color, and transparency information) within this directional range and belonging to the 3DGS map data are determined as the spatial position range of the candidate semantic evidence in the three-dimensional coordinate system. Then, the overlap ratio of the three-dimensional Gaussian elements contained in the spatial position range of candidate semantic evidence from different image sources is compared one by one. If the overlap ratio of the three-dimensional Gaussian elements in the spatial position range of two candidate semantic evidences exceeds a preset ratio threshold (this ratio threshold is the preset position similarity condition, used to determine whether two candidate semantic evidences from different images point to the same actual target in three-dimensional space), then these two candidate semantic evidences are classified into the same candidate semantic instance (a candidate semantic instance refers to a set of candidate semantic evidence that, after preliminary merging, is confirmed to correspond to the same actual target object in the target scene). For example, in the three images mentioned above, the three candidate semantic evidences for the same switch cabinet have a higher overlap ratio of 3D Gaussian primitives within their respective directional ranges than a threshold, and are therefore merged into a single candidate semantic instance. This step utilizes the 3D Gaussian primitive position information already present in the 3DGS map data to determine whether the detection results from different perspectives point to the same actual target. This avoids the same actual target being repeatedly treated as multiple semantic objects because it was captured from multiple observation perspectives, and also avoids the erroneous merging of candidate semantic evidence with significantly different shooting angles that actually point to different targets.

[0020] For each candidate semantic instance, the category labels are weighted according to their corresponding reliability weights. Specifically, for each category label within the instance, the reliability weights of the candidate semantic evidence holding that label are accumulated, and the category label with the highest accumulated weight is determined as the category information of that candidate semantic instance. For example, if a candidate semantic instance has three candidate semantic evidences, two of which are frontal images classifying it as "switch cabinet" with high reliability weights, and one is classified as "distribution box" due to a side-angled shot with a lower reliability weight, the weighted accumulation for "switch cabinet" is larger, thus determining the category information of the candidate semantic instance as "switch cabinet." This step avoids misclassification caused by biased shooting angles or partial obstruction from individual observation perspectives directly determining the category of the semantic object. By determining the category through a weighted voting method based on the reliability of multiple candidate semantic evidences, the probability of category judgment errors is reduced when on-site equipment has similar appearances or partial obstruction exists.

[0021] For each candidate semantic evidence merged within the same candidate semantic instance, the union of the 3D Gaussian primitives corresponding to each candidate semantic evidence merged into that instance is taken. Based on the position coordinates of each 3D Gaussian primitive in the 3D coordinate system corresponding to the 3D map data in the union, the spatial location information (spatial location information refers to the 3D coordinate position and spatial range occupied by the semantic object in the 3D coordinate system) of the candidate semantic instance is determined. The category information determined in the previous step and the spatial location information determined in this step are used together as the attributes of the corresponding semantic object, resulting in a semantic object that includes both category information and spatial location information. This step determines the position by merging the 3D Gaussian primitives hit by multiple candidate semantic evidences corresponding to the same semantic instance, rather than using only the directional range corresponding to the detection box of a single image. This makes the spatial location information of the obtained semantic object in the 3D coordinate system more complete and closer to the actual spatial range it occupies in the target scene.

[0022] 103. Based on the category information, spatial location information and asset data of semantic objects, calculate the binding confidence between semantic objects and asset data, and generate candidate binding relationships based on the binding confidence. In this embodiment, the binding confidence between the semantic object and the asset data is calculated based on the category information, spatial location information, and asset data of the semantic object. This includes: comparing the category information of the semantic object with the asset type field in the asset data to obtain a category matching component; comparing the spatial location information of the semantic object with the region field in the asset data to obtain a location matching component; determining the image region corresponding to the semantic object from the 3DGS map data based on the spatial location information of the semantic object; performing text recognition on the image region to obtain a text recognition result; comparing the text recognition result with the name field or number field in the asset data to obtain a text matching component; and performing a weighted summation of the category matching component, location matching component, and text matching component to obtain the binding confidence. The process of comparing the category information of a semantic object with the asset type field in the asset data to obtain a category matching component, and comparing the spatial location information of a semantic object with the region field in the asset data to obtain a location matching component, includes: calculating the semantic similarity between the category information of the semantic object and the asset type field, and determining the semantic similarity as the category matching component; calculating the location distance between the spatial location information and the spatial region corresponding to the region field in the asset data based on the spatial location information of the semantic object; comparing the location distance with a preset distance threshold, and generating a location matching component based on the comparison result.

[0023] Specifically, for each semantic object, its category information is compared one by one with the asset type field corresponding to each asset record in the asset data. The comparison is based on a category lookup table pre-organized in the asset management system to which the asset data belongs (the category lookup table refers to the pre-established correspondence between the semantic object's category information and the name used in the asset type field, for example, the semantic object category information "switch cabinet" corresponds to the name used in the asset type field "low-voltage distribution cabinet"). If the semantic object's category information and the asset type field of a certain asset record have a corresponding relationship in the category lookup table, or if the two names are the same, then the category matching component corresponding to that record is determined to be a higher value; if the category lookup table does not contain a corresponding relationship and the names are also inconsistent, then the category matching component is determined to be a lower value. This step uses a pre-organized category lookup table for comparison, rather than directly calculating the similarity of the textual meaning. The reason is that the number of equipment types and asset type names involved in the target scenario is limited, and the correspondence between them can be clarified in advance. Using a lookup table for comparison can avoid the distortion of comparison results due to inconsistencies between the semantic object category naming and the asset ledger naming conventions.

[0024] For each semantic object, based on its spatial location information and the spatial region boundary defined by the corresponding region field of each asset record in the asset data (this boundary range is exported from the asset management system along with the asset data, representing the range occupied by the region to which the asset record belongs in the corresponding three-dimensional coordinate system of the 3DGS map data), the shortest distance from the spatial location information of the semantic object to the boundary of the spatial region is calculated to obtain the location distance. If the spatial location information of the semantic object falls within the boundary range of the region, the location distance is determined to be zero. Then, the location distance is compared with a preset distance threshold. The preset distance threshold is pre-set according to the actual interval range between adjacent regions in the target scene to allow for possible positioning deviations during the reconstruction of 3DGS map data: the closer the location distance is to zero, the higher the determination of the location matching component; the closer the location distance is to or exceeds the preset distance threshold, the lower the determination of the location matching component, until the location distance exceeds the preset distance threshold and the location matching component is determined to be zero. This step uses the spatial boundary defined by the region field as the basis for comparison and retains a certain distance tolerance range. This ensures that when a semantic object deviates slightly from the boundary of its region due to 3DGS map data reconstruction errors, it can still obtain a high positional matching component with the corresponding asset record in that region. This avoids the direct exclusion of asset records that should be bound due to reconstruction errors.

[0025] For each semantic object, based on its spatial location information, an observational view image that can fully display the location of the semantic object is selected from multiple observational view images corresponding to the 3DGS map data. The image region corresponding to the spatial location information of the semantic object is then extracted from this observational view image (the image region refers to a local image range in the observational view image corresponding to the spatial location of the semantic object, usually including the nameplate or label area affixed to the surface of the equipment). This image region is then subjected to optical character recognition (OCR, commonly known as OCR, used to identify and extract text content printed or affixed to the image surface) to perform text recognition, obtaining the text recognition result (the text recognition result refers to the text characters identified from the image region, such as the name or number printed on the equipment nameplate). The text recognition result is compared with the name or number field of each asset record in the asset data. If the text recognition result matches or contains the content of the name or number field of a record, the text matching component corresponding to that record is determined to have a higher value; otherwise, if they do not match or no valid text is identified from the image region, a lower value is determined. This step introduces nameplate text as a source of evidence independent of category and spatial location information. When multiple devices of the same type are placed in the same area, and the category matching component and the location matching component are difficult to distinguish, the name or number on the nameplate can be used to differentiate them.

[0026] After obtaining the category matching component, location matching component, and text matching component, a weighted sum is performed by multiplying each component by its corresponding weight coefficient and then adding them together to obtain the binding confidence score. The weight coefficients corresponding to each component are pre-set based on the completeness of the device nameplates in the target scene: if not all devices in the target scene have clearly identifiable nameplates, the weight coefficient corresponding to the text matching component is set relatively low. The category matching component and location matching component are used as the main basis for calculating the binding confidence score to avoid the overall binding confidence score being too low due to individual devices having missing or unidentifiable nameplates. The binding confidence scores of a semantic object are compared with those of each asset record in the asset data. The asset record with the highest binding confidence score, which is not lower than a preset confidence threshold, is selected as the candidate binding relationship (a candidate binding relationship refers to the association between a semantic object and an asset record that has not yet been manually confirmed and is initially generated by the system based on the binding confidence score calculation results). If the binding confidence scores of each asset record corresponding to a semantic object are all lower than the preset confidence threshold, no candidate binding relationship is generated for that semantic object, and the process proceeds to the manual binding supplementation stage. This step integrates the comparison results of the three components into a single binding confidence score value through weighted aggregation. This allows subsequent ranking and judgment of candidate binding relationships based on a unified standard, and filters out semantic objects with insufficient binding evidence by using a preset confidence threshold for manual processing. This avoids forcibly binding asset records with insufficient binding evidence, which could lead to discrepancies between the asset data and the actual situation of the target scenario.

[0027] 104. Based on the spatial location information of each semantic object, generate candidate spatial relationships between semantic objects, and based on the preset constraint relationship data and the category information of semantic objects, generate candidate constraint information corresponding to the semantic objects. Candidate binding relationships, candidate spatial relationships and candidate constraint information constitute candidate semantic association data. In this embodiment, generating candidate spatial relationships between semantic objects based on their spatial location information includes: acquiring geometric range information corresponding to each semantic object; determining spatial location relationships between semantic objects based on their spatial location information and geometric range information, wherein the spatial location relationships include at least one of inclusion relationship, adjacency relationship, top-bottom positional relationship, and region affiliation relationship; constructing a spatial relationship graph between semantic objects based on their spatial location relationships to obtain candidate spatial relationships. In an optional embodiment, determining spatial location relationships between semantic objects based on their spatial location information and geometric range information includes: acquiring preset reference plane data and preset region boundary data; determining inclusion relationships between semantic objects based on the overlap between their spatial location information and geometric range information, and determining adjacency relationships between semantic objects based on the interval distance between their spatial location information; determining top-bottom positional relationships between semantic objects based on their spatial location information and the preset reference plane data; determining region affiliation relationships corresponding to each semantic object based on their spatial location information and the preset region boundary data; and determining inclusion relationship, adjacency relationship, top-bottom positional relationship, and region affiliation relationship as spatial location relationships. Based on the preset constraint relationship data and the category information of semantic objects, the generation of candidate constraint information corresponding to semantic objects includes: searching for physical attribute entries and business attribute entries corresponding to the category information in the preset constraint relationship data according to the category information of semantic objects; determining the accessible attributes corresponding to semantic objects according to the spatial location information of semantic objects and physical attribute entries, and generating physical attribute information; determining the operational constraint attributes corresponding to semantic objects according to business attribute entries and candidate spatial relationships, and generating business attribute information, wherein at least one of physical attribute information and business attribute information constitutes candidate constraint information.

[0028] Specifically, for each semantic object, all 3D Gaussian primitives belonging to that semantic object are taken (the spatial location information of the semantic object is determined by the aggregation of these 3D Gaussian primitives). Based on the position coordinates of these 3D Gaussian primitives in the corresponding 3D coordinate system of the 3DGS map data, the bounding area that can encompass these 3D Gaussian primitives is calculated. This bounding area is determined as the geometric range information corresponding to the semantic object (geometric range information refers to the actual three-dimensional space occupied by the semantic object in the 3D coordinate system, used to distinguish it from spatial location information that only represents a single location point). For example, if the category information of a semantic object is "switch cabinet", the 3D Gaussian primitives belonging to it will present a cuboid shape in the 3D coordinate system. The bounding area determined accordingly is the geometric range information corresponding to the switch cabinet. This step supplements the volume range occupied by the semantic object in 3D space, providing necessary comparison basis for subsequent judgment of whether there are spatial positional relationships such as inclusion or adjacency between semantic objects, avoiding the difficulty of judging the actual spatial occupancy relationship between two semantic objects based on only a single location point.

[0029] A horizontal plane is fitted to the position coordinates of all 3D Gaussian primitives in the 3D coordinate system corresponding to the 3D map data. That is, the horizontal region with the lowest height and the largest continuous distribution area is found in the 3D coordinate system, and the horizontal plane containing this region is determined as the preset reference plane data (the preset reference plane data is used to represent the horizontal reference plane where the ground in the target scene is located. The reason why the horizontal region with the lowest height and the largest continuous distribution area can be used as the ground is that the ground in the target scene usually presents the widest coverage and the lowest height continuous plane distribution relative to the suspended or vertical equipment such as cabinets and pipes. This horizontal plane fitting method is a common practice for extracting the ground plane in 3D reconstruction data processing and does not require additional reliance on the functions of the acquisition device itself); then the preset area boundary data is obtained (the preset area boundary data refers to the boundary range of each functional area in the target scene in the 3D coordinate system, which is the same data as the spatial area boundary defined by the area field in the aforementioned asset data). Based on this, the overlap ratio between the geometric range information of each semantic object is used to determine whether there is an inclusion relationship between two semantic objects: if the geometric range of one semantic object is mostly within the geometric range of another semantic object, then it is determined that there is an inclusion relationship between the two. For example, if the geometric range of "switch" is within the geometric range of "switch cabinet", it is determined that the switch cabinet includes the switch. Then, based on the interval distance between the spatial location information of each semantic object, it is determined whether there is an adjacency relationship between two semantic objects. The smaller the interval distance, the more likely it is to determine that the two are adjacent. The basis for determining the interval distance is the actual layout interval between devices in the same area of ​​the target scene. For example, the interval between two adjacent cabinets in the same power distribution room is generally smaller than the interval between the cabinet and the device in another area in the same power distribution room. Based on this, the adjacency relationship within the same area and the non-adjacency situation across areas can be distinguished.

[0030] Based on the relative height relationship between the spatial location information of each semantic object and the preset reference plane data, the vertical orientation relationship between semantic objects is determined: The height of the spatial location information of two semantic objects relative to the preset reference plane data is calculated respectively, and the one with the higher height is determined to be above the other. For example, if a "meter" is installed at a high position on the surface of a "switch cabinet," by comparing its relative height with the preset reference plane data, the meter is determined to be above the switch cabinet. Next, based on the positional relationship between the spatial location information of each semantic object and the preset area boundary data, it is determined whether the spatial location information of the semantic object falls within a certain preset area boundary range. If it does, the area affiliation of the semantic object is determined to be that area. For example, if the spatial location information of a semantic object falls within the range of a certain power distribution room defined by the preset area boundary data, its area affiliation is determined to be that power distribution room. The inclusion relationship, adjacency relationship, vertical orientation relationship, and area affiliation relationship determined above are collectively used as the spatial location relationship between the semantic objects. This section describes the relationships between semantic objects from four different perspectives: occupancy, spacing, height, and affiliation. This allows the same pair of semantic objects to simultaneously possess multiple types of spatial positional relationships, providing a more complete representation of the actual placement and layout of devices within the target scene compared to a single-dimensional positional comparison.

[0031] Each semantic object is treated as a node, and the spatial relationships between the previously determined semantic objects are used as connecting edges between nodes. The edges are labeled with the corresponding spatial relationship type (inclusion, adjacency, top / bottom, or regional affiliation). This constructs a spatial relationship graph between semantic objects (a spatial relationship graph is a graph structure composed of semantic objects as nodes and spatial relationship types as edges, used to uniformly express the mutual positional relationships between semantic objects within the target scene). This spatial relationship graph is then identified as a candidate spatial relationship. This step integrates the various spatial relationships determined for each pair of semantic objects into a single graph structure. This allows downstream queries to directly retrieve the relationship between a semantic object and other objects within the target scene from this graph structure, eliminating the need to recalculate the positional relationships for each pair of semantic objects. It also facilitates a visual representation of the mutual positional layout of devices within a given area during subsequent manual review.

[0032] Based on the category information of the semantic object, the system searches for the corresponding physical and business attribute entries in the pre-defined constraint relationship data (which is data pre-organized according to the semantic object category information, recording the physical and business attribute entries that the object of that category typically possesses; its content comes from the on-site operation procedures and equipment specifications of the target scenario; for example, the physical attribute entry for the "switch cabinet" category is "the outer shell surface is accessible, but internal live parts are not accessible," and the business attribute entry is "power verification is required before operation, and personnel are prohibited from staying in the passageway during operation when adjacent to it"). For example, if the semantic object category information is "switch cabinet," the corresponding physical and business attribute entries are found in the pre-defined constraint relationship data. This step retrieves the pre-organized entries according to the category information, rather than re-determining the physical attributes and business rules for each semantic object individually. This allows equipment of the same category to share a set of accumulated on-site experience entries, reducing the workload of repetitive data entry.

[0033] Based on the spatial location information of the semantic object and the found physical attribute entries, the accessibility attributes corresponding to the semantic object are further determined (accessibility attributes refer to the accessibility or inaccessibility attribute labels corresponding to different parts within the spatial range occupied by the semantic object). Specifically, by combining the geometric range information of the semantic object, descriptions such as "accessible outer shell surface, inaccessible internal live parts" recorded in the physical attribute entries are mapped to the outer and inner parts within the geometric range of the semantic object, respectively, and labeled as accessible, thereby generating the physical attribute information corresponding to the semantic object. For example, the outer part of the geometric range of a "switch cabinet" semantic object corresponds to the cabinet shell and is labeled as accessible, while the inner part corresponds to the location of the live parts inside the cabinet and is labeled as inaccessible. This step refines the accessibility rules originally described uniformly by category in the physical attribute entries to different parts within the object, making the labeling of accessibility attributes more consistent with the actual spatial form occupied by the semantic object in the target scene.

[0034] Based on the found business attribute entries and the candidate spatial relationships of the semantic object determined in the preceding steps, the operational constraint attributes corresponding to the semantic object are determined (operational constraint attributes refer to constraints such as the operational sequence, safety distance, or personnel stay restrictions that the semantic object needs to follow in actual operations). Specifically, it is determined whether the semantic object has a triggering situation described by the business attribute entry in the candidate spatial relationships. If it does, the constraint content corresponding to the business attribute entry is determined as the operational constraint attribute of the semantic object, thereby generating the business attribute information corresponding to the semantic object. For example, if the business attribute entry records "When the switch cabinet is adjacent to the passage, personnel are prohibited from staying in the passage during operation," and if the "switch cabinet" semantic object is determined to be adjacent to a certain "passage" semantic object in the candidate spatial relationships, then the business attribute entry is triggered, and "personnel are prohibited from staying in the passage during operation" is determined as the operational constraint attribute corresponding to the switch cabinet semantic object. At least one of the aforementioned physical attribute information and the business attribute information generated in this step is determined as the candidate constraint information corresponding to the semantic object. This step does not apply business rules in isolation based solely on category information. Instead, it combines the actual spatial relationship between the semantic object and its surrounding semantic objects to determine whether the rule is triggered. This ensures that the generated operational constraint attributes correspond to the specific surrounding environment of the semantic object in the target scenario. For example, if there are no adjacent objects such as passageways around a switch cabinet of the same category, it will not be assigned operational constraints related to passageways.

[0035] 105. Obtain the confirmation operation result corresponding to the candidate semantic association data, and update the candidate semantic association data according to the confirmation operation result to obtain the semantic association data; In this embodiment, obtaining the confirmation operation result corresponding to the candidate semantic association data and updating the candidate semantic association data according to the confirmation operation result to obtain semantic association data includes: determining whether there is a conflict between the candidate spatial relationship and the candidate constraint information, and obtaining a conflict judgment result; determining the confirmation priority corresponding to each data in the candidate semantic association data according to the binding confidence and the conflict judgment result; obtaining the confirmation operation result corresponding to the candidate semantic association data according to the confirmation priority, and updating the candidate semantic association data according to the confirmation operation result to obtain semantic association data. In an optional embodiment, after updating the candidate semantic association data according to the confirmation operation result to obtain semantic association data, the method further includes: determining at least one semantic object associated with the data that has been corrected or deleted in the candidate semantic association data, and obtaining the category information corresponding to at least one semantic object; calculating the correction frequency or deletion frequency corresponding to the category information; generating a priority adjustment coefficient corresponding to the category information according to the correction frequency or deletion frequency, the priority adjustment coefficient being used to adjust the confirmation priority when determining the confirmation priority again for the semantic object with the same category information.

[0036] Specifically, for candidate spatial relationships and candidate constraint information in the candidate semantic association data, each one is compared with the default requirements corresponding to adjacent objects and regional affiliations determined by the candidate spatial relationships of a semantic object. This comparison checks whether the default requirements of adjacent objects and regional affiliations are mutually exclusive with the operational constraint attributes recorded in the semantic object's own candidate constraint information (the default requirements of adjacent objects and regional affiliations also originate from the physical attribute entries and business attribute entries recorded under the corresponding category information in the preset constraint relationship data). For example, a "switch cabinet" semantic object, being adjacent to a "passage" semantic object, triggers the generation of the operational constraint attribute "personnel are prohibited from staying in the passage during operation." However, the preset constraint relationship data records that the "passage" category itself belongs to an emergency evacuation passage and cannot be restricted at any time. Since these two are mutually exclusive, the conflict judgment result is determined to be a conflict. If no such mutually exclusive situation is found, the conflict judgment result is determined to be a non-conflict. This step cross-checks the constraint attributes of a semantic object with the default requirements of its adjacent objects and the regions it belongs to. This allows for the discovery of contradictory content in the automatically generated stage before manual confirmation, preventing contradictory spatial relationships and constraint information from being simultaneously solidified into the semantic world model data.

[0037] For candidate binding relationships, candidate spatial relationships, and candidate constraint information in the candidate semantic association data, a confirmation priority is determined for each item: the lower the binding confidence of a candidate binding relationship, the higher the confirmation priority; if the conflict judgment result of candidate spatial relationships and candidate constraint information is that a conflict exists, the confirmation priority is higher than the case where there is no conflict. For example, the aforementioned switch cabinet semantic object is given a higher priority because it is determined to have a conflict, while a semantic object with a high binding confidence and no conflict was found is given a lower priority. This step uses binding confidence and conflict judgment results together for unified sorting, so that content with insufficient basis or contradictions in the automatic generation stage is given priority to manual verification, and the higher-risk parts are processed first when the audit workload is limited.

[0038] Candidate semantically related data are presented to reviewers one by one in descending order of confirmation priority on the human-computer interaction interface (i.e., the aforementioned 3D Viewer). The interface provides three operations for each candidate data item: confirmation (retaining the original content), correction (entering or selecting replacement content), and deletion (removing the candidate data item). The processing result corresponding to any of these operations is determined as the confirmation operation result. Based on the confirmation operation result, the corresponding entries in the candidate semantically related data are retained, replaced, or removed respectively. After processing all entries, the semantically related data is obtained. This step pushes candidate data in priority order, ensuring that higher-risk content is verified first, and the manual verification results are uniformly written back through three clear operation methods, forming semantically related data that can be solidified.

[0039] After obtaining the semantic association data, from the candidate semantic association data entries that have undergone correction or deletion operations, at least one semantic object associated with each entry is identified, and the category information corresponding to that semantic object is obtained. For this category information, the number of times the candidate semantic association data entries associated with the semantic object belonging to this category information have been corrected or deleted during the current and previous review processes is counted, obtaining the correction frequency or deletion frequency corresponding to this category information. Based on the correction frequency or deletion frequency, a priority adjustment coefficient corresponding to this category information is generated: the higher the correction frequency or deletion frequency, the larger the generated priority adjustment coefficient. For example, in a certain batch of reviews, the candidate binding relationships associated with the semantic object with the category information "switch cabinet" were corrected multiple times, indicating that the accuracy of the candidate binding relationships corresponding to this category object was relatively insufficient in the automatic generation stage. Therefore, a relatively large priority adjustment coefficient is generated for the "switch cabinet" category information. On the other hand, the candidate data associated with the semantic object with the category information "channel" were rarely corrected or deleted in previous reviews. Therefore, a relatively small priority adjustment coefficient is generated for the "channel" category information. This priority adjustment coefficient is used to further increase or maintain the confirmation priority based on the original confirmation priority obtained based on the binding confidence and conflict judgment results when determining the confirmation priority of semantic objects with the same category information in the future. This allows candidate data of subsequent semantic objects of the same category that have frequently required manual correction or deletion in the past to be manually verified earlier, reducing the situation where the automatic generation error of the same category repeatedly occurs in multiple batches but is not detected in time.

[0040] 106. Mark the map version information onto the semantic association data to generate semantic world model data.

[0041] In one optional implementation of this embodiment, the semantic association data includes binding relationships corresponding to each semantic object. After marking map version information to the semantic association data and generating semantic world model data, the method further includes: obtaining asset version information corresponding to asset data; obtaining updated asset data in response to an update of asset version information; recalculating the binding confidence based on the category information and spatial location information of the semantic object associated with the binding relationship and the updated asset data; if the recalculated binding confidence is lower than a preset threshold, obtaining the confirmation operation result for the binding relationship, and updating the binding relationship based on the confirmation operation result. In another optional implementation, the semantic association data includes binding relationships corresponding to each semantic object. After marking map version information to the semantic association data and generating semantic world model data, the method further includes: obtaining updated 3DGS map data in response to an update of map version information; determining the updated spatial location information corresponding to the semantic object based on the updated 3DGS map data; determining whether the updated spatial location information is consistent with the spatial location information before the update; if inconsistent, updating the semantic object based on the updated spatial location information, and updating the spatial relationships in the semantic association data based on the updated semantic object.

[0042] Specifically, each piece of semantic association data (which includes the binding relationships, spatial relationships, and constraint information corresponding to each semantic object) is appended with a corresponding map version information identifier. This identifier is the same as the map version information assigned when generating the 3DGS map data. This ensures a clear version correspondence between the semantic association data and the batch of 3DGS map data upon which it is based, collectively defining it as semantic world model data (semantic world model data refers to a complete data set containing semantic objects and their binding relationships, spatial relationships, and constraint information after being associated with map version information, for use by downstream systems). For example, semantic association data confirmed through manual review is associated with map version information "V20260701," forming semantic world model data corresponding to that reconstruction result. This step clearly maps the semantic association data to a specific 3DGS map data reconstruction result, making it easier to determine which semantic association data was generated from which version when subsequent map or asset data is updated, preventing the mixing of data generated in different batches.

[0043] After marking map version information to semantic association data and generating semantic world model data, the process also includes: first, obtaining the asset version information corresponding to the asset data (asset version information is used to identify the time batch of the asset data exported by the asset management system, and is updated as the content of the enterprise's asset ledger changes); comparing the asset version information with the asset version information currently recorded in the semantic world model data; if the two are inconsistent, it is determined that the asset version information has been updated, and the updated asset data is re-exported from the asset management system. For semantic objects that are already associated with binding relationships in the semantic world model data, based on their category information and spatial location information, the binding confidence is recalculated with the updated asset data using the weighted summation method of previous category matching, location matching, and text matching; if the recalculated binding confidence is lower than a preset threshold, it indicates that there is a discrepancy between the asset record originally bound to the semantic object and the actual content after the ledger update. At this time, the confirmation operation result of the reviewer for the binding relationship is obtained through the aforementioned human-computer interaction interface, and the binding relationship is retained, replaced, or removed accordingly. For example, if the name field of an asset record previously bound to a switchgear is modified after an asset ledger update, the recalculated binding confidence score falls below a preset threshold, triggering manual verification and updating the binding relationship accordingly. This step ensures that binding relationships are verified synchronously with actual changes in the asset ledger, and through preset threshold filtering, only those binding confidence scores that show a significant decrease trigger manual verification, avoiding the need for repeated manual confirmation of all binding relationships with each asset ledger update.

[0044] Furthermore, in response to map version information updates—that is, when the target scene undergoes re-collection and training to obtain new 3DGS map data and is assigned new map version information—the updated 3DGS map data is acquired. For each semantic object already existing in the semantic world model data, its spatial location information is re-determined in the coordinate system corresponding to the updated 3DGS map data, following the previous method of determining spatial location information based on its associated 3D Gaussian primitives. This yields updated spatial location information. The updated spatial location information is compared with the original spatial location information to determine whether the degree of offset between the two exceeds a pre-set offset range (this offset range is used to allow for errors inherent in the two reconstruction processes). If it exceeds this range, it is judged as inconsistent, indicating that the actual position of the semantic object in the target scene has changed; if it does not exceed this range, it is judged as consistent, and the spatial location information of the semantic object remains unchanged. For example, if a device is moved to another area near its original location after scene modification, and the spatial location information determined after the new 3DGS map data reconstruction has an offset degree exceeding the pre-update range, it is judged as inconsistent. This step enables the spatial location information in the semantic world model data to be verified synchronously with the reconstruction results of the actual target scene, rather than always using the location determined at the time of initial generation.

[0045] If an inconsistency is determined, the recorded content of the semantic object in the semantic world model data is updated based on the updated spatial location information of the semantic object. Then, based on the updated semantic object, the spatial relationship between the semantic object and other semantic objects is re-determined using the previously established method of determining spatial relationships based on geometric overlap ratio, spacing distance, relative height, and region boundaries. Consequently, the spatial relationship portion of the semantic association data involving the semantic object is updated accordingly. For example, the aforementioned moved device's adjacency and region affiliation relationships with its original adjacent semantic objects are re-determined after its spatial location information is updated, and the original adjacency or region affiliation relationships are adjusted to match its current actual location. This step ensures that the spatial relationships in the semantic world model data are adjusted synchronously with changes in the actual location of the semantic objects, preventing the semantic world model data from retaining the pre-adjustment spatial relationships after device migration or layout adjustments in the target scene, thus avoiding discrepancies between the semantic world model data and the current actual layout of the target scene.

[0046] In this embodiment, by sequentially overlaying semantic object generation, asset binding assessment, spatial relationship modeling, and constraint information generation onto a 3DGS high-fidelity spatial map, and introducing a manual confirmation closed-loop and map version association mechanism, the transformation from a pure geometric 3DGS map to a deliverable and traceable semantic world model is achieved. This solves the technical problem that existing 3DGS map construction methods do not combine semantic objects, asset binding relationships, and spatial business constraints for unified modeling, making it difficult for maps to directly support robot task execution and personnel on-site operation assistance. It effectively improves the accuracy and reliability of semantic annotation and asset binding results, and ensures the consistency and integrability of data when different downstream systems share the same semantic world model base.

[0047] The foregoing has described the semantic map and world model construction method based on 3DGS maps in the embodiments of this application. The following describes the semantic map and world model construction apparatus based on 3DGS maps in the embodiments of this application. Please refer to [link to relevant documentation]. Figure 2 One embodiment of the semantic map and world model construction device based on 3DGS maps in this application includes: Data acquisition module 201 is used to acquire 3DGS map data and asset data corresponding to the target scene. The 3DGS map data includes map version information. The semantic generation module 202 is used to generate multiple semantic objects based on 3DGS map data. The semantic objects include category information and spatial location information. The calculation module 203 is used to calculate the binding confidence between the semantic object and the asset data based on the category information, spatial location information and asset data of the semantic object, and generate candidate binding relationships based on the binding confidence. The relation generation module 204 is used to generate candidate spatial relationships between semantic objects based on the spatial location information of each semantic object, and to generate candidate constraint information corresponding to the semantic object based on the preset constraint relationship data and the category information of the semantic object. The candidate binding relationship, candidate spatial relationship and candidate constraint information constitute candidate semantic association data. The data update module 205 is used to obtain the confirmation operation result corresponding to the candidate semantic association data, and update the candidate semantic association data according to the confirmation operation result to obtain the semantic association data. The tagging module 206 is used to tag map version information to semantic association data to generate semantic world model data.

[0048] above Figure 2 The semantic map and world model building device based on 3DGS map in this application embodiment is described in detail from the perspective of modular functional entities. The semantic map and world model building device based on 3DGS map in this application embodiment is described in detail from the perspective of hardware processing.

[0049] Figure 3 This is a schematic diagram of a semantic map and world model building device based on 3DGS maps, provided in an embodiment of this application. The semantic map and world model building device 300 based on 3DGS maps can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 310 (e.g., one or more processors) and a memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 333 or data 332. The memory 320 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the semantic map and world model building device 300 based on 3DGS maps. Furthermore, the processor 310 may be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media 330 on the semantic map and world model building device 300 based on 3DGS maps.

[0050] The semantic map and world model building device 300 based on 3DGS maps may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 3 The illustrated structure of the semantic map and world model building device based on 3DGS maps does not constitute a limitation on the semantic map and world model building device based on 3DGS maps. It may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0051] This application also provides a semantic map and world model construction device based on 3DGS map. The computer device includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor performs each step of the semantic map and world model construction method based on 3DGS map in the above embodiments.

[0052] This application also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform various steps of the semantic map and world model construction method based on 3DGS maps.

[0053] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0054] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0055] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0056] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for constructing semantic maps and world models based on 3DGS maps, characterized in that, The method for constructing semantic maps and world models based on 3DGS maps includes: Acquire 3DGS map data and asset data corresponding to the target scene, wherein the 3DGS map data includes map version information; Multiple semantic objects are generated based on the 3DGS map data, and the semantic objects include category information and spatial location information. Based on the category information and spatial location information of the semantic object and the asset data, the binding confidence between the semantic object and the asset data is calculated, including: comparing the category information of the semantic object with the asset type field in the asset data to obtain a category matching component, and comparing the spatial location information of the semantic object with the region field in the asset data to obtain a location matching component; determining the image region corresponding to the semantic object from the 3DGS map data according to the spatial location information of the semantic object, performing text recognition on the image region to obtain a text recognition result; comparing the text recognition result with the name field or number field in the asset data to obtain a text matching component, and performing a weighted sum of the category matching component, the location matching component, and the text matching component to obtain a binding confidence, and generating candidate binding relationships based on the binding confidence. Based on the spatial location information of each semantic object, candidate spatial relationships between the semantic objects are generated, and based on the preset constraint relationship data and the category information of the semantic objects, candidate constraint information corresponding to the semantic objects is generated. The candidate binding relationship, the candidate spatial relationship and the candidate constraint information constitute candidate semantic association data. Obtain the confirmation operation result corresponding to the candidate semantic association data, and update the candidate semantic association data according to the confirmation operation result to obtain the semantic association data; The map version information is labeled to the semantic association data to generate semantic world model data.

2. The method for constructing semantic maps and world models based on 3DGS maps according to claim 1, characterized in that, The generation of multiple semantic objects based on the 3DGS map data includes: Image features are extracted from multiple observation view images corresponding to the 3DGS map data to obtain candidate semantic evidence. The candidate semantic evidence includes category information, location information in the corresponding observation view image, and reliability weight of the corresponding observation view. Multiple candidate semantic evidences from different observation perspectives, whose location information meets the preset location similarity condition, are divided into a single candidate semantic instance. The category information of the candidate semantic instance is determined based on the category information of each candidate semantic evidence assigned to the candidate semantic instance and the corresponding reliability weight. Based on the location information of each candidate semantic evidence assigned to the candidate semantic instance and the coordinate system corresponding to the 3DGS map data, the spatial location information of the candidate semantic instance in the 3DGS map data is determined, resulting in a semantic object including category information and spatial location information.

3. The method according to claim 1, characterized in that, The steps of comparing the category information of the semantic object with the asset type field in the asset data to obtain a category matching component, and comparing the spatial location information of the semantic object with the region field in the asset data to obtain a location matching component, include: Calculate the semantic similarity between the category information of the semantic object and the asset type field, and determine the semantic similarity as a category matching component; Based on the spatial location information of the semantic object and the boundary of the spatial region corresponding to the region field in the asset data, calculate the positional distance between the spatial location information and the spatial region; The location distance is compared with a preset distance threshold, and the location matching component is generated based on the comparison result.

4. The method according to claim 1, characterized in that, The step of generating candidate spatial relationships between the semantic objects based on their spatial location information includes: Obtain the geometric range information corresponding to each of the semantic objects; Based on the spatial location information and geometric range information of each semantic object, the spatial location relationship between the semantic objects is determined, and the spatial location relationship includes at least one of the following: inclusion relationship, adjacency relationship, top-bottom position relationship, and region affiliation relationship; Based on the spatial positional relationships between the semantic objects, a spatial relationship graph is constructed to obtain candidate spatial relationships.

5. The method according to claim 1, characterized in that, The step of generating candidate constraint information corresponding to the semantic object based on preset constraint relationship data and the category information of the semantic object includes: Based on the category information of the semantic object, search for the physical attribute entries and business attribute entries corresponding to the category information in the preset constraint relationship data; Based on the spatial location information of the semantic object and the physical attribute entries, the accessibility attribute corresponding to the semantic object is determined, and physical attribute information is generated; Based on the relationship between the business attribute entries and the candidate space, the operation constraint attributes corresponding to the semantic object are determined, and business attribute information is generated. The physical attribute information and at least one of the business attribute information constitute candidate constraint information.

6. The method according to claim 1, characterized in that, The step of obtaining the confirmation operation result corresponding to the candidate semantic association data, and updating the candidate semantic association data according to the confirmation operation result to obtain the semantic association data includes: Determine whether there is a conflict between the candidate spatial relationship and the candidate constraint information, and obtain the conflict determination result; Based on the binding confidence and the conflict judgment result, the confirmation priority of each data in the candidate semantic association data is determined; According to the confirmation priority, obtain the confirmation operation result corresponding to the candidate semantic association data, and update the candidate semantic association data according to the confirmation operation result to obtain the semantic association data.

7. A semantic map and world model construction device based on 3DGS maps, characterized in that, The semantic map and world model construction device based on 3DGS maps includes: The data acquisition module is used to acquire 3DGS map data and asset data corresponding to the target scene, wherein the 3DGS map data includes map version information; The semantic generation module is used to generate multiple semantic objects based on the 3DGS map data, wherein the semantic objects include category information and spatial location information; The calculation module is used to calculate the binding confidence between the semantic object and the asset data based on the category information and spatial location information of the semantic object and the asset data. This includes: comparing the category information of the semantic object with the asset type field in the asset data to obtain a category matching component; comparing the spatial location information of the semantic object with the region field in the asset data to obtain a location matching component; determining the image region corresponding to the semantic object from the 3DGS map data based on the spatial location information of the semantic object; performing text recognition on the image region to obtain a text recognition result; comparing the text recognition result with the name field or number field in the asset data to obtain a text matching component; and performing a weighted summation of the category matching component, the location matching component, and the text matching component to obtain a binding confidence; and generating candidate binding relationships based on the binding confidence. The relationship generation module is used to generate candidate spatial relationships between the semantic objects based on the spatial location information of each semantic object, and to generate candidate constraint information corresponding to the semantic objects based on preset constraint relationship data and the category information of the semantic objects. The candidate binding relationship, the candidate spatial relationship and the candidate constraint information constitute candidate semantic association data. The data update module is used to obtain the confirmation operation result corresponding to the candidate semantic association data, and update the candidate semantic association data according to the confirmation operation result to obtain semantic association data; The tagging module is used to tag the map version information to the semantic association data to generate semantic world model data.

8. A semantic map and world model construction device based on 3DGS maps, characterized in that, The semantic map and world model building device based on 3DGS map includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the 3DGS map-based semantic map and world model building device to perform the steps of the 3DGS map-based semantic map and world model building method as described in any one of claims 1-6.

9. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the various steps of the semantic map and world model construction method based on 3DGS maps as described in any one of claims 1-6.

Images