A spatial information management method, device and medium

Abstract

This application discloses a spatial information management method, device, and medium, which has the advantage of facilitating the storage and retrieval of spatial information. Targeting large-scale dynamic scenarios such as office buildings and commercial complexes, it solves the problems of high storage and maintenance costs, inability to link asset database updates, and limited knowledge extraction methods in traditional mapping solutions. It constructs an intelligent knowledge base system based on image array spatial memory and object-level language encoding, achieving a closed loop of accurate perception, traceability, and automatic updating of building asset status. The spatial information management method includes the following steps: acquiring images of the target area; constructing or updating a global base for image array spatial memory, building a basic spatial framework through topological relationships and spatial indexes, and storing images of the target area; and recognizing the images within the base to obtain static and dynamic semantic objects.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a spatial information management method. Background Technology

[0002] In the fields of embodied intelligence, robotic inspection, and 3D mapping, common solutions typically use LiDAR, RGB-D cameras, or multi-view reconstruction algorithms to obtain point clouds, mesh models, or dense 3D models, and then use these models for localization, navigation, and scene understanding. For small-scale or fixed scenarios, the above methods can provide a relatively intuitive spatial geometric representation; however, in large-scale, long-term operational scenarios such as office buildings, commercial complexes, and parking lots, high-density point clouds and dense 3D models often lead to higher storage, update, transmission, and maintenance costs.

[0003] Existing SLAM, RGB-D mapping, and semantic SLAM technologies can combine geometric information with semantic targets to some extent, but their core is still largely based on geometric maps or point cloud maps. As the scale of the scene increases and changes occur frequently in shop layout, decoration, door signs, shelves, or shop signs, the cost of continuously maintaining a high-precision, dense model is high, and pure geometric maps cannot directly express asset operation semantics such as "who the tenant is, what industry they are in, whether they are actually operating, whether the door signs have changed, whether parking spaces are occupied, and the status of customer or vehicle traffic."

[0004] Visual location recognition, image retrieval, and six-degree-of-freedom pose estimation techniques can infer the spatial location and orientation of observed images based on image matching or neural networks. Recent advancements in visual geometry models can infer camera parameters, depth, point maps, or trajectories from one or more images, offering new possibilities for rapid pose inference. However, if these models are only used as single-shot localization tools without forming a closed loop with long-term accumulated image arrays, object semantics, asset ledgers, and change detection results, they remain difficult to directly serve the operation of building and commercial property assets.

[0005] On the other hand, building and retail asset management typically relies on manual inspections, manual data entry, leasing system records, and fragmented monitoring. The lack of a unified spatial semantic index between surveillance cameras, inspection robots, and asset databases makes it difficult to update information such as store rental status, tenant company, industry type, renovation changes, address changes, hard furnishing alterations, area verification, customer flow, actual operating status, parking space occupancy, and traffic flow in a timely, automatic, and low-cost manner.

[0006] In terms of long-term memory systems, existing solutions often suffer from three main shortcomings: First, video language models typically retrieve information based on fragment-level features or fragment descriptions, which can answer what happened in a particular fragment, but lack indexes that are directly linked to spatial units such as 3D location, floor, shop, and parking space; Second, traditional geometric maps or SLAM trajectories can preserve location, but struggle to express natural language state transitions such as "the door number has been changed," "the shop is under renovation," or "a roadblock has been moved from the fire lane to the freight elevator entrance"; Third, event-based memories are usually organized around task or activity fragments, easily folding the long-term state changes of individual objects into larger events, making it impossible to form an independent timeline that can be traced for each object.

[0007] Therefore, a new spatial information organization method and business process is needed: instead of relying solely on long-term storage of high-density point clouds, an image array with pose, depth, object, semantics, time, and evidence images should be used as the main body of spatial memory; at the same time, object-level language trajectory encoding should be constructed for key objects, so that the spatial location, visual evidence, semantic state changes, and asset field updates of objects can be uniformly queried, incrementally updated, and traced. Summary of the Invention

[0008] This application provides a spatial information management method, which has the advantage of facilitating the storage and retrieval of spatial information. Targeting large-scale dynamic scenarios such as office buildings and commercial complexes, it solves the problems of high storage and maintenance costs, inability to link asset database updates, and limited knowledge extraction methods in traditional mapping solutions. It constructs an intelligent knowledge base system based on image array spatial memory and object-level language encoding, achieving a closed loop of accurate perception, traceability, and automatic updating of building asset status.

[0009] The above-mentioned objective of this application is achieved through the following technical solution: On the one hand, this application provides a spatial information management method, including the following steps: S1: Acquire an image of the target region; S2: Construct or update the global base of spatial memory for image arrays, build a basic spatial framework through topological relationships and spatial indexes, and store images of the target region; S3: Recognize the image inside the base to obtain static semantic objects and dynamic semantic objects.

[0010] Furthermore, in the global base of spatial memory for the image array, the images include image content, acquisition time, camera number, floor, region, shop or parking space association, and estimate or calibrate their three-dimensional coordinates and orientation; it also records the adjacency relationship, passability relationship, field of view overlap relationship and topological relationship between images, forming an image array index for spatial query.

[0011] Furthermore, its three-dimensional coordinates and orientation are estimated or calibrated through the following steps: The image capture location is obtained through GPS to obtain a rough position. Then, by using the overlapping relationship of multiple source images, the precise relative pose is calculated through a forward collapsing SLAM model. By combining the GPS position of the multiple source images, the marking information of the public map, and any one of the manual correction methods, the precise relative pose is mapped to the absolute pose.

[0012] Furthermore, the topological relationships between images are constructed by combining absolute spatial coordinates, the content overlap between images, and the marker information in the images.

[0013] Furthermore, in step S3, the image is subjected to object detection, instance segmentation, character recognition, semantic segmentation, depth estimation, and scene understanding. One or more objects are extracted from the following: door signs, shop signs, company names, industry logos, walls, partitions, shelves, counters, decoration materials, fire-fighting facilities, parking space lines, vehicles, pedestrian flow, and business equipment. The object category, object location, object confidence, correspondence between the object and the image, correspondence between the object and spatial units, and semantic attributes of the object are written into the object field and the semantic field.

[0014] Furthermore, after obtaining the object detection, tracking, and location results, an O-LTE record is created or updated for each object; if the current object and the historical object meet the matching conditions in terms of category, appearance features, OCR text, spatial location, and temporal continuity, then the current object is merged into the historical object; if not, a new object record is created.

[0015] Furthermore, for each object range, the system generates a language phase description based on keyframes, target boxes, OCR text, object categories, spatial units, and historical states. The language phase description is simultaneously written into the object record, text view, and event view, and linked with spatial anchors, visual anchors, and asset state anchors.

[0016] Furthermore, a 2D semantic mask to 3D point cloud surface mapping technique is adopted, relying on the pixel-level index correspondence between the 2D semantic map and the 3D reconstructed point map to achieve direct reverse projection of 2D semantic labels to 3D space. First, open vocabulary 2D semantic segmentation is performed on the input image, and a pixel-by-pixel instance mask is generated based on the text prompts. After filtering by a confidence threshold, the union is taken to output a binary semantic label map. At the same time, the 3D reconstruction backbone network predicts the 3D spatial coordinates of the same image pixel by pixel, outputting a 3D point map with a resolution proportional to the image, along with a point-by-point reconstruction confidence score. The two naturally share the index relationship of the pixel grid. The core of the mapping is the functional composition of the pixel dimension: the 3D coordinates of each pixel position directly inherit the semantic label of the corresponding pixel. In engineering implementation, the semantic mask needs to be resampled to the point map resolution using nearest neighbor interpolation, strictly preserving discrete label attributes and avoiding the blurring of category boundaries caused by bilinear interpolation, which would destroy the discreteness assumption of semantic labels. Then, the aligned mask and the point map are flattened synchronously, and the spatial coordinates and semantic labels are matched pixel by pixel to complete the reverse mapping and output the initial 3D semantic point cloud.

[0017] The initial point cloud is located in the camera coordinate system and needs to be adapted to multi-view fusion by coordinate system normalization: first, the basis vectors from the camera to the body are rearranged, and then the attitude is corrected by pitch, yaw and rotation matrices; a three-level backoff strategy is adopted to restore the true scale, prioritizing the anchor scale with the motion baseline, secondly selecting the ground installation height prior, and taking the default value of the unit scale when there is no effective prior; finally, low-quality points are filtered according to the confidence percentile, and the corresponding semantic labels are removed simultaneously to eliminate mapping misalignment noise.

[0018] In another aspect, this application provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and when the computer program is invoked and executed by the processor, it implements the steps in the method described above.

[0019] In another aspect, this application provides a computer-readable medium storing a computer program that, when executed by a computer, implements the steps in the method described above.

[0020] In summary, the beneficial effects of this application are as follows: For large-scale dynamic scenarios such as office buildings and commercial complexes, this solution addresses the problems of high storage and maintenance costs, inability to link asset database updates, and limited knowledge extraction methods in traditional mapping solutions. It constructs an intelligent knowledge base system based on image array spatial memory and object-level language encoding, enabling precise perception, traceability, and automatic updating of building asset status in a closed loop. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall process of an embodiment of this application; Figure 2 This is a schematic diagram of the closed-loop architecture for multi-source image acquisition, image array spatial memory, and asset operation in an embodiment of this application; Figure 3 This is a schematic diagram of spatial large model positioning and image alignment in an embodiment of this application; Figure 4 This is a structural diagram of the global memory system and object-level language trajectory coding (O-LTE) according to an embodiment of this application; Figure 5 This is a flowchart illustrating the construction and updating of object-level language trajectory encoding in this application embodiment. Detailed Implementation

[0022] The specific embodiments of this application are described in detail below with reference to the accompanying drawings.

[0023] Example 1: Reference Figure 1-5 A spatial information management method includes the following steps: S1: Acquire images of the target area. This is achieved through a multi-source image acquisition module, which includes a matrix of surveillance cameras located in office buildings, public areas of shops, corridors, entrances / exits, parking lots, etc., as well as cameras on mobile inspection robots, handheld devices, or other mobile acquisition devices. Time information uses either the network time or system time of the original acquisition device, with network time being preferred.

[0024] S2: Construct or update the global spatial memory base for the image array. This involves building a basic spatial framework using topological relationships and spatial indexes to store images of the target area. The global spatial memory base for the image array adopts an incremental, differentiated construction mode. First, it relies on monitoring and inspection robots to collect multi-view images, labeling image time, location, regional association information, and six-dimensional pose. This allows for the construction of the global spatial memory base for the image array. The basic spatial framework is then built using topological relationships and various types of spatial indexes, compressing and storing static spatial structures such as floors, shops, and parking spaces.

[0025] In the global base of spatial memory for the image array, for each frame or each group of key images, the image includes the image content, acquisition time, camera number, floor, area, shop or parking space association, and estimates or marks its three-dimensional coordinates and orientation; it also records the adjacency relationship, passability relationship, field of view overlap relationship and topological relationship between images, forming an image array index for spatial query.

[0026] Estimate or calibrate its three-dimensional coordinates and orientation using the following steps: The image capture location is roughly determined using GPS. Then, by analyzing the overlap between multiple image sources, a precise relative pose (including x, y, z viewpoint coordinates, and rou, pitch, yaw directions) is calculated using a forward-collision SLAM model (such as VGGT). Combining the GPS location from the multi-source images, the marker information from public maps, and manual correction methods, the precise relative pose is mapped to the absolute pose (latitude and longitude coordinates).

[0027] VGGT feedforward visual localization technology, based on deep neural networks, takes multiple images from different perspectives as input (which can be photos taken by different cameras at different times, with different resolutions, without any calibration), and directly outputs the relative 6DOF pose (including 3 positional degrees of freedom and 3 directional degrees of freedom) of each photo relative to the first photo through a single inference. Feedforward visual localization networks are generally pre-trained with a large amount of data, thus having strong robustness (resistant to noise, unconstrained lighting, partial occlusion, dynamic interference, etc.) and versatility.

[0028] The topological relationships between images are constructed by combining absolute spatial coordinates, the content overlap between images, and the landmark information in the images (such as floor information, text information, doorways, stairwells, etc.).

[0029] S3: Recognize the image inside the base to obtain static semantic objects and dynamic semantic objects.

[0030] The system performs object detection, instance segmentation, character recognition, semantic segmentation, depth estimation, and scene understanding on images. It extracts one or more objects from the following categories: door signs, shop signs, company names, industry logos, walls, partitions, shelves, counters, decoration materials, fire-fighting facilities, parking space lines, vehicles, pedestrian flow, and business equipment. The system writes the object category, object location, object confidence, correspondence between the object and the image, correspondence between the object and spatial units, and semantic attributes of the object into the object field and the semantic field.

[0031] The target location is obtained as follows: the target detection and segmentation in the image yields a two-dimensional mask. This two-dimensional mask is then combined with the image's depth information (usually obtained synchronously through a feedforward visual localization network) and the camera's intrinsic and extrinsic parameters. By inverse projection, the position and size of the target object in three-dimensional space can be obtained.

[0032] This paper employs a 2D semantic mask to 3D point cloud surface mapping technique, relying on the pixel-level index correspondence between the 2D semantic map and the 3D reconstructed point map to achieve direct reverse projection of 2D semantic labels to 3D space. First, open-vocabulary 2D semantic segmentation is performed on the input image. A pixel-by-pixel instance mask is generated based on text prompts, filtered by a confidence threshold, and then the union is taken to output a binary semantic label map. Simultaneously, a 3D reconstruction backbone network predicts the 3D spatial coordinates of the same image pixel by pixel, outputting a 3D point map with a resolution proportional to the image, along with point-by-point reconstruction confidence. The two naturally share the pixel grid index relationship. The core of the mapping is the functional composition of pixel dimensions: the 3D coordinates of each pixel directly inherit the semantic label of the corresponding pixel. In engineering implementation, the semantic mask needs to be resampled to the point map resolution using nearest-neighbor interpolation, strictly preserving discrete label attributes and avoiding the blurring of category boundaries caused by bilinear interpolation, which would violate the discreteness assumption of semantic labels. Then, the aligned mask and point map are simultaneously flattened, and the spatial coordinates and semantic labels are matched pixel by pixel to complete the reverse mapping and output the initial 3D semantic point cloud.

[0033] The initial point cloud is located in the camera coordinate system and needs to be adapted to multi-view fusion by coordinate system normalization: first, the basis vectors from the camera to the body are rearranged, and then the attitude is corrected by pitch, yaw and rotation matrices; a three-level backoff strategy is adopted to restore the true scale, prioritizing the anchor scale with the motion baseline, secondly selecting the ground installation height prior, and taking the default value of the unit scale when there is no effective prior; finally, low-quality points are filtered according to the confidence percentile, and the corresponding semantic labels are removed simultaneously to eliminate mapping misalignment noise.

[0034] When an inspection robot or other embodied intelligent agent enters the scene, its current camera captures one or more frames of observation images. The system inputs the current observation image, along with the stored image array, the object field and semantic field corresponding to the image, and the six-dimensional pose information corresponding to the image, into the visual geometric pose inference module. Based on the visual geometric relationships, semantic correspondences, and historical pose indexes between images, this module outputs the coordinates and orientation corresponding to the current observation image, thereby obtaining the six-degree-of-freedom pose of the embodied intelligent agent in the building space.

[0035] Object-level language trajectory encoding is constructed. After obtaining the object detection, tracking, and localization results, an O-LTE record is established or updated for each object. If the current object matches a historical object in terms of category, appearance features, OCR text, spatial location, and temporal continuity, it is merged into the historical object; otherwise, a new object record is created. The system divides intervals based on the object's spatial displacement, semantic changes, and visibility state, and extracts sparse anchor points at state change boundaries, spatial inflection points, before and after occlusion, before and after text changes, and before and after changes in business fields.

[0036] Language phase description generation. For each object region, the system generates a language phase description based on keyframes, bounding boxes, OCR text, object category, spatial units, and historical states. This description can be generated from a visual language model, rule templates, business dictionaries, or manual correction results. The language phase description is simultaneously written to the object record, text view, and event view, and linked to spatial anchors, visual anchors, and asset status anchors.

[0037] Asset Status Identification and Update. After obtaining the current image pose, the system aligns and compares the image with historical object-level linguistic trajectory codes of the same shops, corridors, parking spaces, or adjacent spatial units. Through object changes, text changes, structural changes, semantic changes, and event chain updates, the system determines whether a shop has been rented out, whether the tenant's company name has changed, whether the business industry has changed, whether it has been renovated, whether the address has changed, whether the hard furnishings or partitions have been altered, whether the shop area needs to be recalculated, whether there are abnormal customer flows, whether there are signs of actual operation, whether parking spaces are occupied, and whether the traffic flow status has changed.

[0038] The asset database operates in a closed loop. For identified changes, the system generates asset status update events. Each event includes at least the spatial unit number, object ID, event type, evidence image, event time, semantics before and after the change, confidence level, database field mapping, and suggested action. The asset database update module automatically writes the data or submits it for manual review according to preset rules.

[0039] Operational analysis is extended. The image array spatial memory and object-level language trajectory encoding of this invention are not only used for positioning but also serve as a long-term visual semantic foundation for building operations. The system can calculate indicators such as customer flow, vehicle flow, business activity, abnormal occupancy, and risks of unauthorized modifications or asset idleness based on the status of people, vehicles, shop openings / closings, renovations / construction, and operating equipment within a continuous time window.

[0040] The depth information in the image array can come from RGB-D cameras, binocular vision, multi-view reconstruction, or monocular depth estimation; the object field can come from object detection models, visual language models, character recognition models, or manual correction; the spatial index can use octrees, quadtrees, R-trees, raster graphs, multi-branch trees, or relational databases; the language stage description in object-level language trajectory encoding can be generated by large language models, visual language models, rule templates, or combinations thereof. Therefore, the innovation of this invention does not lie in a specific model itself, but in the overall method and system of using image array spatial memory to carry spatial geometric, topological, and semantic information, maintaining long-term state changes of objects with object-level language trajectory encoding, and then using it for embodied intelligent positioning and building asset operation database updates.

[0041] Example 2: An electronic device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the steps of the method described in Example 1.

[0042] Example 3: A computer-readable medium storing a computer program, which, when executed by a computer, implements the steps of the method described in Example 1.

[0043] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the inventive concept of this application, and these all fall within the protection scope of this application.

Claims

1. A spatial information management method, characterized in that, Includes the following steps: S1: Acquire an image of the target region; S2: Construct or update the global base of spatial memory for image arrays, build a basic spatial framework through topological relationships and spatial indexes, and store images of the target region; S3: Recognize the image inside the base to obtain static semantic objects and dynamic semantic objects.

2. The spatial information management method according to claim 1, characterized in that, In the global base of spatial memory for image arrays, images include image content, acquisition time, camera number, floor, region, shop or parking space association, and estimate or calibrate their three-dimensional coordinates and orientation; it also records the adjacency relationship, passability relationship, field of view overlap relationship and topological relationship between images, forming an image array index for spatial query.

3. The spatial information management method according to claim 1, characterized in that, Estimate or calibrate its three-dimensional coordinates and orientation using the following steps: The image capture location is obtained through GPS to obtain a rough position. Then, by using the overlapping relationship of multiple source images, the precise relative pose is calculated through a forward collapsing SLAM model. By combining the GPS position of the multiple source images, the marking information of the public map, and any one of the manual correction methods, the precise relative pose is mapped to the absolute pose.

4. The spatial information management method according to claim 1, characterized in that, The topological relationships between images are constructed by combining absolute spatial coordinates, the content overlap between images, and the marker information in the images.

5. The spatial information management method according to claim 1, characterized in that, In step S3, the image is subjected to object detection, instance segmentation, text recognition, semantic segmentation, depth estimation, and scene understanding. One or more objects are extracted from the following: door signs, shop signs, company names, industry logos, walls, partitions, shelves, counters, decoration materials, fire-fighting facilities, parking space lines, vehicles, pedestrian flow, and business equipment. The object category, object location, object confidence, correspondence between the object and the image, correspondence between the object and spatial units, and semantic attributes of the object are written into the object field and the semantic field.

6. The spatial information management method according to claim 5, characterized in that, After obtaining the object detection, tracking and location results, an O-LTE record is created or updated for each object; if the current object and the historical object meet the matching conditions in terms of category, appearance features, OCR text, spatial location and temporal continuity, the current object is merged into the historical object; if not, a new object record is created.

7. The spatial information management method according to claim 6, characterized in that, For each object range, the system generates a language phase description based on keyframes, target boxes, OCR text, object category, spatial unit, and historical state. The language phase description is simultaneously written to the object record, text view, and event view, and linked with spatial anchors, visual anchors, and asset state anchors.

8. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is invoked and executed by the processor, it implements the steps of the method as described in any one of claims 1-7.

9. A computer-readable medium, characterized in that, The computer-readable medium stores a computer program that, when executed by a computer, implements the steps of the method as described in any one of claims 1-7.

Images