Hierarchical scene matching method and device based on topological constraints, equipment and medium

Abstract

The application relates to the technical field of image processing, and discloses a layered scene matching method and device based on topological constraints, equipment and a medium, which comprises the following steps: performing multi-level visual coding processing on an input image to obtain local visual markers, middle-level visual markers and global visual markers; performing semantic segmentation and topological analysis processing on the input image to obtain a local topological graph, a middle-level topological graph and a global topological graph; generating corresponding adjacency matrices based on the topological graphs, and performing cross-attention processing based on the topological constraints on the visual markers based on the adjacency matrices to obtain feature embeddings; performing hierarchical feature fusion processing on the feature embeddings to obtain a unified representation vector; and performing scene matching processing based on the unified representation vector to obtain a matching result. The application can be applied to business scenes such as financial technology and medical health, and the topological constraints are introduced to participate in visual feature modeling and combined with hierarchical feature fusion, so that the accuracy and stability of matching in a complex scene are improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a hierarchical scene matching method, apparatus, device, and medium based on topological constraints. Background Technology

[0002] The application of visual position recognition in complex indoor environments faces multiple challenges, especially in situations with numerous structurally similar areas, significant changes in viewing angle, and unstable lighting conditions. Existing scene representation and matching methods based on visual features struggle to achieve stable and highly discriminative results. On one hand, existing methods often rely on global features or single-scale features, making it difficult to simultaneously depict the relationship between local details and the overall layout. On the other hand, the visual feature modeling process lacks effective representation of spatial structural relationships, resulting in the underutilization of key connectivity and structural constraints within the scene. Furthermore, existing technologies typically model based on static structures or fixed relationships, lacking the ability to dynamically adjust spatial relationships according to the input image, thus easily leading to inaccurate matching in complex environments.

[0003] In the fintech sector, visual location recognition is widely used in scenarios such as intelligent branch management, risk monitoring, asset verification, and security patrols. However, existing technologies often struggle to distinguish subtle differences when dealing with standardized branch layouts or equipment environments, such as similar counter layouts or repetitive equipment arrangements, easily leading to misjudgments. Furthermore, current methods lack effective modeling of spatial relationships and regional connectivity between devices, preventing the system from utilizing spatial structure information for auxiliary judgments during risk identification or asset location. In addition, single-level feature representations cannot simultaneously reflect local equipment characteristics and the overall branch structure, affecting matching accuracy in complex business environments.

[0004] In the healthcare field, visual location recognition is primarily used in scenarios such as hospital navigation, medical equipment positioning, and ward environment recognition. Due to the large number of structurally similar ward areas and corridor layouts within hospitals, existing recognition methods based on global features struggle to achieve accurate differentiation, easily leading to false matches. Furthermore, current technologies lack the ability to express spatial constraints such as ward connectivity and access control structures during the modeling process, failing to reflect the structural characteristics of the actual environment. Moreover, when faced with dynamic changes in ward conditions or changes in area status, existing methods rely on static structural modeling, unable to dynamically adjust spatial relationships based on current input information, thus limiting their adaptability and reliability in real-world medical environments. Summary of the Invention

[0005] The main objective of this invention is to provide a hierarchical scene matching method, apparatus, device, and storage medium based on topological constraints. This invention aims to solve the technical problem that existing technologies fail to introduce dynamically adjustable spatial topological constraints during multi-level feature representation and matching, resulting in difficulties in achieving accurate and stable scene matching in indoor environments with similar structures and complex spatial relationships.

[0006] To achieve the above objectives, the present invention provides a hierarchical scene matching method based on topological constraints, comprising: Multi-level visual coding processing is performed on the input image to obtain local visual tags, intermediate visual tags, and global visual tags; Semantic segmentation processing is performed on the input image to obtain a semantic segmentation map. Topological parsing processing is then performed on the semantic segmentation map to obtain a local topological map, a mid-level topological map, and a global topological map. Based on the local topology graph, the intermediate topology graph, and the global topology graph, a local adjacency matrix, an intermediate adjacency matrix, and a global adjacency matrix are generated, respectively. Based on the local adjacency matrix, the intermediate adjacency matrix, and the global adjacency matrix, topological constraint cross-attention processing is performed on the local visual labels, intermediate visual labels, and global visual labels respectively to obtain local feature embedding, intermediate feature embedding, and global feature embedding; Hierarchical feature fusion processing is performed on the local feature embedding, the mid-level feature embedding, and the global feature embedding to obtain a unified representation vector; Scene matching processing is performed based on the unified representation vector to obtain the matching result.

[0007] Furthermore, to achieve the above objectives, the present invention provides a hierarchical scene matching device based on topological constraints, comprising: The visual encoding module is used to perform multi-level visual encoding processing on the input image to obtain local visual tags, intermediate visual tags, and global visual tags. The topology parsing module is used to perform semantic segmentation processing on the input image to obtain a semantic segmentation map, and to perform topology parsing processing on the semantic segmentation map to obtain a local topology map, a mid-level topology map, and a global topology map. An adjacency matrix generation module is used to generate a local adjacency matrix, a mid-level adjacency matrix, and a global adjacency matrix based on the local topology graph, the mid-level topology graph, and the global topology graph, respectively. The topological constraint cross-attention module is used to perform topological constraint cross-attention processing on the local visual labels, mid-level visual labels, and global visual labels based on the local adjacency matrix, mid-level adjacency matrix, and global adjacency matrix, respectively, to obtain local feature embedding, mid-level feature embedding, and global feature embedding. The hierarchical feature fusion module is used to perform hierarchical feature fusion processing on the local feature embedding, the intermediate feature embedding, and the global feature embedding to obtain a unified representation vector; The scene matching module is used to perform scene matching processing based on the unified representation vector to obtain the matching result.

[0008] Furthermore, to achieve the above objectives, the present invention also provides a computer device, the computer device including a memory, a processor, and a topology-constrained hierarchical scene matching program stored in the memory and executable on the processor, wherein when the topology-constrained hierarchical scene matching program is executed by the processor, it implements the steps of the topology-constrained hierarchical scene matching method as described above.

[0009] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a topology-constrained hierarchical scene matching program, wherein the topology-constrained hierarchical scene matching program, when executed by a processor, implements the steps of the topology-constrained hierarchical scene matching method as described above.

[0010] Beneficial Effects: This invention relates to the field of image processing technology and discloses a hierarchical scene matching method, apparatus, device, and medium based on topological constraints. The method includes: performing multi-level visual encoding processing on an input image to obtain local visual markers, mid-level visual markers, and global visual markers; performing semantic segmentation processing and topological parsing processing on the input image to obtain a local topological graph, a mid-level topological graph, and a global topological graph; generating local adjacency matrices, mid-level adjacency matrices, and global adjacency matrices based on the local, mid-level, and global topological graphs, respectively; performing cross-attention processing based on topological constraints on the local, mid-level, and global visual markers based on the local, mid-level, and global adjacency matrices to obtain local feature embeddings, mid-level feature embeddings, and global feature embeddings; performing hierarchical feature fusion processing on the local, mid-level, and global feature embeddings to obtain a unified representation vector; and performing scene matching processing based on the unified representation vector to obtain a matching result. This invention can be applied to business scenarios such as fintech and healthcare. By converting the multi-level topological graph obtained from topological analysis into an adjacency matrix and introducing a cross-attention process, the visual feature modeling is constrained by the spatial structure. At the same time, it combines hierarchical feature fusion to form a unified representation vector, thereby improving the accuracy and stability of scene matching in complex indoor environments. Attached Figure Description

[0011] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a schematic diagram of an application environment for a hierarchical scene matching method based on topological constraints according to an embodiment of the present invention. Figure 2 This is a flowchart illustrating an embodiment of the hierarchical scene matching method based on topological constraints according to the present invention. Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the hierarchical scene matching device based on topological constraints of the present invention. Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0012] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0013] The hierarchical scene matching method based on topological constraints provided in this invention can be applied to, for example... Figure 1 In this application environment, the client communicates with the server via a network. The server can perform multi-level visual encoding processing on the input image through the client, obtaining local visual tags, mid-level visual tags, and global visual tags; perform semantic segmentation and topological parsing processing on the input image, obtaining local, mid-level, and global topological graphs; generate local, mid-level, and global adjacency matrices based on the local, mid-level, and global topological graphs, respectively; perform topological constraint-based cross-attention processing on the local, mid-level, and global visual tags based on the local, mid-level, and global adjacency matrices, obtaining local, mid-level, and global feature embeddings; perform hierarchical feature fusion processing on the local, mid-level, and global feature embeddings to obtain a unified representation vector; and perform scene matching processing based on the unified representation vector to obtain the matching result. This invention can be applied to business scenarios such as fintech and healthcare. By transforming the multi-level topological graph obtained from topological analysis into an adjacency matrix and introducing a cross-attention process, visual feature modeling is constrained by spatial structure. Simultaneously, it combines hierarchical feature fusion to form a unified representation vector, thereby improving the accuracy and stability of scene matching in complex indoor environments. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0014] Please see Figure 2 , Figure 2This is a flowchart illustrating an embodiment of the hierarchical scene matching method based on topological constraints provided by the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0015] like Figure 2 As shown, the hierarchical scene matching method based on topological constraints proposed in this invention includes the following steps: S10, Perform multi-level visual coding processing on the input image to obtain local visual tags, intermediate visual tags and global visual tags; In this embodiment, the input image represents a set of two-dimensional pixel data acquired by an imaging device. Pixel data consists of spatial coordinates and pixel values. Each pixel value can originate from the visible light channel, infrared channel, or grayscale channel. In the fintech business field, this data can correspond to branch monitoring screens, equipment inspection images, or asset identification images; in the healthcare business field, it can correspond to behavioral acquisition images or health status record images. Multi-level visual coding processing involves constructing feature representation structures at multiple scales on the same input image. Image content is encoded through different spatial resolutions and different receptive regions. This processing stems from the combination of hierarchical visual representation structures and attention mechanisms, enabling the same image to form differentiated expressions in different spatial ranges.

[0016] The implementation of multi-level visual coding processing relies on a visual coding network structure, which consists of multiple sequentially connected coding layers. Each coding layer receives the feature representation output from the previous layer and updates it. In the specific implementation, the input image is divided into multiple image patches. Each image patch is converted into a feature vector through a linear embedding map. The linear embedding map transforms the pixel values ​​through a weight matrix and a bias term, transforming the image patch from pixel space to feature space. Within each coding layer, attention units and feature transformation units process the feature vectors. The attention unit establishes the relationships between image patches, and the feature transformation unit performs a non-linear mapping on the association results to obtain the updated feature representation.

[0017] Local visual markers represent feature representations extracted based on a small spatial range. They originate from high-resolution feature maps or fine-grained image patch sets. Local visual markers retain more spatial detail information, such as texture structure, boundary shape, or local object appearance. In implementation, by limiting the attention computation range, feature interactions are concentrated between adjacent image patches, thus avoiding interference from distant information on local details. Mid-level visual markers represent feature representations extracted based on a medium spatial range. They originate from the feature results of aggregating multiple local regions. In implementation, downsampling or window merging operations are performed on the feature map to fuse multiple adjacent image patches into a single region feature, thus forming a mid-level visual marker. This marker reflects region-level structural information, such as functional area distribution or local layout relationships. Global visual markers represent feature representations extracted based on the overall spatial range. They originate from global feature interaction processing of all image patches. By expanding the attention computation range in the encoding layer, each feature vector can establish associations with all other feature vectors, thus forming an overall structural representation. This marker reflects global layout information, such as the overall environmental structure or spatial organization relationships.

[0018] The local, mid-level, and global visual markers maintain a consistent feature space representation through hierarchical relationships. During implementation, a unified feature dimension mapping method ensures that feature vectors at different levels have the same representation dimension, facilitating fusion or comparison in subsequent processing stages. This hierarchical structure relies on resolution adjustment and feature aggregation operations. Resolution adjustment achieves feature map size changes through downsampling or stride variations, while feature aggregation achieves region-level representation by combining features from multiple image patches, thus forming a fine-to-coarse hierarchical structure.

[0019] In fintech, multi-level visual coding can be used to represent branch environments in a layered manner. Local visual markers are used to identify equipment status or risk details, mid-level visual markers are used to identify regional distribution structures, and global visual markers are used to identify overall layout consistency. In healthcare, multi-level visual coding can be used to represent behavioral images in a layered manner. Local visual markers reflect subtle movement features, mid-level visual markers reflect the relationship between movement combinations, and global visual markers reflect the overall behavioral pattern, thus providing a layered visual representation basis for subsequent processing.

[0020] In one implementation, the input image is divided into multiple image blocks of a fixed size. Each image block is converted into a feature vector through linear embedding mapping and then input into a multi-layer coding structure for processing. The first few layers maintain a high resolution to generate local visual labels. In the middle layers, a downsampling operation is used to generate intermediate visual labels. In the subsequent layers, a global visual label is generated through global feature interaction.

[0021] In another implementation, the image patch partitioning adopts an adaptive strategy, dynamically adjusting the partitioning granularity according to the complexity of the image content. Smaller image patches are used in areas with complex textures, while larger image patches are used in areas with simple structures. This improves the ability to express details when generating local visual labels, while reducing the computational complexity when generating global visual labels.

[0022] A multi-branch structure can also be used to process features at different scales separately. Each branch outputs local visual labels, mid-level visual labels, and global visual labels respectively, and the consistency of representation between different levels is maintained through feature alignment.

[0023] This embodiment uses multi-level visual coding to form local visual markers, mid-level visual markers, and global visual markers, so that the same input image can obtain unified and complementary feature representations at different spatial scales, thereby improving the integrity of the representation of detailed information and overall structural information.

[0024] S20, perform semantic segmentation processing on the input image to obtain a semantic segmentation map, and perform topological parsing processing on the semantic segmentation map to obtain a local topological map, a mid-level topological map, and a global topological map; In this embodiment, the input image serves as the basic data carrier for semantic parsing. It consists of a two-dimensional pixel matrix, where each pixel contains spatial location information and pixel value information. Performing semantic segmentation on the input image means mapping pixel-level data to a semantic category distribution. This process is implemented through a semantic segmentation network, which includes an encoding structure and a decoding structure. The encoding structure is used to extract multi-scale semantic features, and the decoding structure is used to restore spatial resolution and generate pixel-level category prediction results. In the implementation process, the input image is fed into the encoding structure, where semantic features are extracted through multiple convolutional units or attention units. Subsequently, in the decoding structure, upsampling and feature fusion operations are used to restore the feature map size, and a semantic segmentation map with the same size as the input image is output. Each pixel position in the semantic segmentation map corresponds to a semantic category identifier, which comes from a predefined set of categories. For example, in fintech, this could include categories such as equipment, channels, counters, and risk areas; in healthcare, it could include categories such as body parts, action areas, and environmental areas.

[0025] Semantic segmentation maps are used to represent the semantic distribution relationships of different regions in an input image. Performing topological parsing on the semantic segmentation map represents the construction of structured connectivity information based on semantic categories and spatial relationships. Topological parsing takes the semantic segmentation map as input and extracts multiple semantic regions by performing connectivity analysis on continuous regions of the same semantic category. Each semantic region is then mapped to a topological node. A topological node contains region boundary information, region center location, and region category identifier. Region boundary information is obtained by extracting the boundaries of pixel-connected regions, region center location is obtained by statistical calculation of pixel coordinates within the region, and region category identifier is derived from the semantic segmentation results. Topological connections are constructed based on the spatial relationships between topological nodes. These spatial relationships are determined by the relative position, distance, or contact relationship between regions. For example, if two regions have a boundary contact relationship or are within a preset distance range, a connection is established between the corresponding topological nodes.

[0026] Local topology graphs represent structural graphs built upon fine-grained semantic regions. They originate from the extraction of nodes and the construction of relationships within small-scale semantic regions. Nodes in a local topology graph correspond to relatively small semantic regions, such as a single device component or a local area of ​​the human body. The connections between nodes reflect local spatial adjacency relationships. In implementation, by setting a small spatial neighborhood, node connections are established only within this neighborhood, thus forming a local topology. Mid-level topology graphs represent structural graphs built upon medium-scale semantic regions. They originate from merging or aggregating multiple adjacent semantic regions, combining multiple local regions into mid-level regions, and constructing topological connections based on the connectivity relationships between regions. In implementation, mid-level nodes are formed through region merging rules or region grouping strategies, thus reflecting the region-level spatial structure. Global topology graphs represent structural graphs built upon the overall semantic layout. They originate from a global-scale node relationship analysis of the entire semantic segmentation graph. By extracting key semantic regions or structural anchor points and establishing global connections between these nodes, the overall spatial structure is reflected.

[0027] The construction of connectivity relationships in topology analysis relies on spatial relationship metrics, which can be determined based on the distance between node centers, the contact state between region boundaries, or the relative directional relationships between regions. In fintech, topology analysis can be used to identify spatial relationships between devices and channels in a branch environment, such as the adjacency between devices or the connectivity between channels, thereby providing structured information for risk identification. In healthcare, topology analysis can be used to identify spatial relationships between body parts or connectivity between action areas, thereby providing structured representations for behavioral analysis.

[0028] In one implementation, semantic segmentation processing employs an encoding-decoding structure. Semantic features are extracted through multiple convolutional units, and spatial resolution is restored through progressive upsampling during the decoding stage. Simultaneously, cross-layer feature fusion is introduced during the encoding stage, enabling high-level semantic information and low-level spatial information to jointly participate in the generation of the segmentation result, thereby obtaining a semantic segmentation map.

[0029] In another implementation, semantic segmentation processing uses an attention-based segmentation network to improve semantic consistency through global feature interaction. After generating the semantic segmentation graph, a set of semantic regions is generated by connecting region extraction, and each semantic region is mapped to a topological node.

[0030] It is also possible to construct topological graphs of different levels through multi-scale region parsing. At the fine-grained level, local topological graphs can be generated directly based on semantic regions. At the intermediate level, mid-level topological graphs can be generated through region aggregation. At the global level, global topological graphs can be generated through key region filtering, thus forming a hierarchical topological structure.

[0031] This embodiment obtains pixel-level semantic distribution through semantic segmentation and constructs multi-level structural relationships through topological parsing, transforming image content from pixel representation to spatial structural representation, thereby enhancing the ability to express regional relationships and overall layout in complex scenes.

[0032] S30, based on the local topology graph, the intermediate topology graph, and the global topology graph, generate the local adjacency matrix, the intermediate adjacency matrix, and the global adjacency matrix, respectively; In this embodiment, the local topology graph, the mid-level topology graph, and the global topology graph represent sets of structural relationships at different spatial granularities. The nodes and connections in the local topology graph reflect the spatial relationships between fine-grained regions, the nodes and connections in the mid-level topology graph reflect the organizational relationships between region-level structures, and the nodes and connections in the global topology graph reflect the key structural relationships in the overall layout. Based on the local, mid-level, and global topology graphs, local adjacency matrices, mid-level adjacency matrices, and global adjacency matrices are generated, respectively, representing the conversion of graph-structured relationship descriptions into matrix-structured relationship descriptions, enabling subsequent calculations to directly access structural relationship information. The rows and columns in the adjacency matrix correspond to the node sequences in the topology graph, and the matrix elements represent whether relationships exist between nodes and the strength of those relationships. Therefore, the adjacency matrix is ​​both a numerical representation of the topology graph and a fundamental representation for spatial structural information to enter the subsequent feature calculation process.

[0033] The generation of the local adjacency matrix depends on the set of local nodes and local connectivity relationships in the local topology graph. The set of local nodes is derived from the extraction results of each node in the local topology graph, with each local node corresponding to a local region unit. Local connectivity relationships are derived from adjacency relationships, contact relationships, or distance relationships within a local area in the local topology graph. In the implementation process, the nodes in the local topology graph are first numbered to form a local node index sequence, and then the corresponding elements in the local adjacency matrix are written according to the relationship status between each pair of local nodes. When there is a direct spatial relationship between two local nodes, the corresponding matrix element is assigned a non-zero value; when there is no spatial relationship between two local nodes, the corresponding matrix element retains a zero value or a preset suppression value. If it is necessary to distinguish the strength of the relationship, the value of the matrix element is determined based on the distance between the center positions of the local nodes, the degree of boundary contact, or the relative directional relationship, so that the local adjacency matrix reflects not only the presence or absence of connections but also the degree of local spatial association.

[0034] The generation of the mid-level adjacency matrix depends on the mid-level node set and mid-level connectivity relationships in the mid-level topology graph. The mid-level node set typically corresponds to region-level structural units, and the mid-level connectivity relationships reflect the connectivity, adjacency, or reachability states between regions. In implementation, the nodes in the mid-level topology graph are sorted and numbered, and a mid-level node index sequence is established. Based on the region connectivity states between mid-level nodes, the connectivity results are mapped to the matrix elements of the mid-level adjacency matrix. If there is a region connectivity relationship or path reachability relationship between two mid-level nodes, the corresponding matrix element is assigned a higher correlation value; if there is a separation relationship or inaccessibility relationship between two mid-level nodes, the corresponding matrix element is assigned a lower correlation value or zero. This process enables the mid-level adjacency matrix to reflect the organizational relationships between mid-level structures and express the density of region-level spatial relationships.

[0035] The generation of the global adjacency matrix depends on the global node set and global connectivity relationships in the global topology graph. The global node set typically corresponds to key global structural units or structural anchors in the overall layout, while global connectivity relationships describe the connection methods and structural distribution relationships in the overall space. In implementation, global nodes in the global topology graph are uniformly indexed, and the corresponding matrix elements are determined based on the topological connectivity status between global nodes. Global connectivity relationships can be measured by the relative positions of global nodes, overall reachability, path organization relationships, or layout distribution relationships. When there is a clear topological association between two global nodes, the matrix element is assigned a positive association value; when there is no association between two global nodes, the matrix element is assigned a zero value or a weak association value. This process enables the global adjacency matrix to reflect the strength of structural associations in the overall layout.

[0036] The determination of matrix elements in the adjacency matrix generation process can employ either a binary representation or a continuous weight representation. The binary representation distinguishes the existence of node associations, while the continuous weight representation indicates the strength of these associations. Continuous weights can be derived from distance metrics between node centers, boundary contact metrics, region connectivity metrics, or topological connectivity metrics. Distance metrics reflect the spatial proximity between nodes, boundary contact metrics reflect the contact range between region edges, region connectivity metrics reflect the degree of interconnectivity between regions, and topological connectivity metrics reflect the tightness of structural associations in the global layout. By transforming the discrete relationships in the graph structure into numerical relationships in a matrix using these metrics, the local adjacency matrix, the intermediate-level adjacency matrix, and the global adjacency matrix can each carry structural information at different spatial scales.

[0037] The dimensions of the local, intermediate, and global adjacency matrices are determined by the number of nodes in the corresponding topological graph. Local topological graphs have a larger number of nodes, so local adjacency matrices typically offer finer-grained representations of relationships. Intermediate topological graphs have a relatively smaller number of nodes, so intermediate adjacency matrices focus more on regional organizational relationships. Global topological graphs have an even smaller number of nodes, so global adjacency matrices focus more on overall layout relationships. By constructing adjacency matrices at three different levels, structural constraint information at different spatial scales is preserved, avoiding the loss of detailed information or ambiguity of global relationships caused by compressing all structural relationships into a single matrix.

[0038] In fintech, local adjacency matrices can express the local spatial relationships between devices and between devices and terminal components; mid-level adjacency matrices can express the organizational relationships between business areas, service areas, and security areas; and global adjacency matrices can express the key structural distribution relationships in the entire branch network layout. In healthcare, local adjacency matrices can express fine-grained spatial relationships between local behavioral areas; mid-level adjacency matrices can express the combination relationships between functional areas; and global adjacency matrices can express the structural distribution relationships in the overall environment. By generating local, mid-level, and global adjacency matrices respectively, spatial relationships in different business scenarios can be transformed into a unified numerical representation.

[0039] In one implementation, the nodes in the local, intermediate, and global topology graphs all use their center positions as the basis for relation calculation. Corresponding weights are generated based on the distances between node center positions, and these weights are then constrained according to the connectivity relationships within their respective topology graphs, thus forming the local, intermediate, and global adjacency matrices, respectively. This implementation is suitable for image scenarios where node boundaries are clear and region centers are easily determined.

[0040] In another implementation, the local adjacency matrix is ​​generated based on the degree of boundary contact between local nodes, the intermediate-level adjacency matrix is ​​generated based on the connectivity between regions, and the global adjacency matrix is ​​generated based on the topological connectivity between global nodes. This implementation emphasizes using different relational measurement methods at different levels, making the generation of local, intermediate, and global matrices more closely match the structural characteristics of the corresponding spatial scale.

[0041] This embodiment transforms the local topology graph, the intermediate topology graph, and the global topology graph into local adjacency matrices, intermediate adjacency matrices, and global adjacency matrices, respectively. The structural relationships at different spatial scales are uniformly represented in a computable matrix form, thereby allowing local spatial associations, regional organization relationships, and overall layout relationships to be preserved and expressed in numerical form simultaneously.

[0042] S40, based on the local adjacency matrix, the intermediate adjacency matrix, and the global adjacency matrix respectively, perform topological constraint cross-attention processing on the local visual markers, the intermediate visual markers, and the global visual markers to obtain local feature embeddings, intermediate feature embeddings, and global feature embeddings; In this embodiment, local visual labels are subjected to cross-attention processing based on topological constraints based on the local adjacency matrix to obtain local feature embeddings; mid-level visual labels are subjected to cross-attention processing based on topological constraints based on the mid-level adjacency matrix to obtain mid-level feature embeddings; and global visual labels are subjected to cross-attention processing based on topological constraints based on the global adjacency matrix to obtain global feature embeddings.

[0043] The local adjacency matrix, mid-level adjacency matrix, and global adjacency matrix are derived from the structural relationship representations of different levels of the topological graph. The elements in the matrices are used to describe the spatial association strength and connection state between nodes. Local visual labels, mid-level visual labels, and global visual labels are derived from feature representations at different scales after multi-level visual encoding processing. Local visual labels reflect fine-grained visual information, mid-level visual labels reflect region-level structural information, and global visual labels reflect overall layout information. Performing topological constraint-based cross-attention processing on the local visual labels based on the local adjacency matrix introduces spatial structural constraints into the feature interaction process at the local scale, making feature fusion restricted by adjacency relationships.

[0044] Cross-attention processing relies on feature mapping and attention computation structures. Local visual labels are input into the feature mapping unit and mapped into local query vectors, local key vectors, and local value vectors. The mapping process performs a linear transformation on the feature vectors through a parameter matrix, giving vectors with different functions discriminative representation capabilities. Correlation calculation is performed between the local query vectors and local key vectors to obtain a local initial attention distribution, which characterizes the degree of information association between different local visual labels. The local adjacency matrix participates in the modulation process. By weighting the local initial attention distribution, higher weights are retained only for features with connections in the local adjacency matrix, while interactions between features without connections are suppressed, thus forming local topological modulation weights. The local topological modulation weights and local value vectors undergo weighted fusion operations to output local feature embeddings. These local feature embeddings introduce local spatial structural relationships while preserving fine-grained visual information.

[0045] Performing topologically constrained cross-attention processing on mid-level visual tags based on the mid-level adjacency matrix is ​​structurally consistent with local processing, but its scope is extended from fine-grained to the region level. After mapping, mid-level visual tags generate mid-level query vectors, mid-level key vectors, and mid-level value vectors. The correlation between the mid-level query vectors and mid-level key vectors yields the initial mid-level attention distribution. The mid-level adjacency matrix constrains the initial mid-level attention distribution based on region connectivity or region organization, ensuring effective interaction between features that are only related in terms of region structure, thus obtaining mid-level topological modulation weights. Weighted fusion of the mid-level topological modulation weights and mid-level value vectors outputs a mid-level feature embedding, which expresses the joint result of region-level spatial structure and visual information.

[0046] Global visual markers are subjected to topologically constrained cross-attention processing based on the global adjacency matrix to introduce structural constraints within the overall space. Global visual markers are mapped to generate a global query vector, a global key vector, and a global value vector. The correlation between the global query vector and the global key vector yields the initial global attention distribution. The global adjacency matrix modulates the attention distribution based on overall layout or global connectivity relationships, ensuring feature interactions conform to the overall spatial organization, resulting in global topological modulation weights. These global topological modulation weights and the global value vectors undergo a weighted fusion operation to output a global feature embedding, which includes both overall structural and semantic information.

[0047] The above processing relies on a hierarchical cross-attention model structure. The model consists of multiple stacked attention computation layers, each containing a feature mapping unit, an attention computation unit, and a feature fusion unit. The model input includes local visual labels, mid-level visual labels, global visual labels, and the adjacency matrix of the corresponding layer. The model output includes local feature embeddings, mid-level feature embeddings, and global feature embeddings. During training, paired images or image sequences are used as input data. Parameters are updated by applying matching and structural consistency constraints to the embedding results. Training parameters include feature dimension, number of attention heads, number of layers, and weight update coefficients. The weights are optimized through a gradient propagation mechanism, ensuring that feature embeddings at different layers simultaneously reflect visual information and spatial structural relationships.

[0048] In fintech business scenarios, mobile robots can be deployed in branches, self-service areas, or asset management areas. They acquire environmental images through visual acquisition and form local, mid-level, and global visual tags. Simultaneously, semantic parsing is used to obtain local, mid-level, and global adjacency matrices. In topological constraint-based cross-attention processing, the local adjacency matrix focuses the feature interactions between device components on actually adjacent areas, thus distinguishing subtle differences between similar devices in local feature embedding. The mid-level adjacency matrix accurately expresses the connectivity between counter areas and aisle areas, thus distinguishing the functional area organization in mid-level feature embedding. The global adjacency matrix incorporates the overall layout of the branch into feature interactions, thus distinguishing branch environments with similar structures but different layouts in global feature embedding, enabling robots to avoid mismatches when performing inspections or asset location.

[0049] In healthcare scenarios, mobile robots can be deployed in rehabilitation training environments or health behavior monitoring environments. They acquire information about human activity areas and environmental structure through visual acquisition, forming multi-level visual labels. Simultaneously, adjacency matrices are obtained based on semantic parsing at the corresponding levels. In cross-attention processing, the local adjacency matrix allows the spatial relationships between local human regions to participate in feature fusion, thereby distinguishing subtle differences in movements in local feature embedding. The mid-level adjacency matrix expresses the combination relationships between action regions, thereby distinguishing different action patterns in mid-level feature embedding. The global adjacency matrix allows the relationship between the overall environmental structure and human behavior to participate in feature interaction, thereby distinguishing scenarios with similar structures but different behavioral paths in global feature embedding, enabling the robot to maintain stable recognition capabilities during behavior analysis and position determination.

[0050] This embodiment introduces adjacency matrix constraints in cross-attention processing at different levels, so that the visual feature interaction process is simultaneously modulated by spatial structural relationships. This allows spatial organization information to be reflected at fine-grained, regional, and overall scales, thereby improving the ability of feature representation to depict the differences in complex environmental structures.

[0051] S50, perform hierarchical feature fusion processing on the local feature embedding, the mid-level feature embedding and the global feature embedding to obtain a unified representation vector; In this embodiment, local feature embedding, mid-level feature embedding, and global feature embedding are derived from visual representation results at different spatial scales. Local feature embedding carries fine-grained regional information, mid-level feature embedding carries regional structural information, and global feature embedding carries overall layout information. Performing hierarchical feature fusion processing on local feature embedding, mid-level feature embedding, and global feature embedding means integrating information at different scales in a unified feature space, so that the information at each level forms a consistent representation while maintaining its differences.

[0052] Hierarchical feature fusion processing relies on feature alignment and weight allocation mechanisms. Before entering the fusion stage, local, intermediate, and global feature embeddings undergo dimension mapping processing through a feature mapping unit to ensure consistency in feature dimensions, thus meeting the input requirements for subsequent fusion computation. Dimension mapping performs a linear transformation on the feature embeddings through a parameter matrix, unifying features from different sources into the same representation space. This process makes the local, intermediate, and global feature embeddings numerically comparable and composable.

[0053] After dimensional alignment, hierarchical weight modeling is performed. Hierarchical weights describe the relative importance of local, mid-level, and global feature embeddings in the current input image; these weights are derived from the response strength or structural saliency of each level of feature. In implementation, statistical mapping or feature compression is performed on the local, mid-level, and global feature embeddings to generate local, mid-level, and global hierarchical fusion coefficients, ensuring that different levels of features contribute differently during the fusion process.

[0054] In the fusion computation stage, weighted combination processing is performed on the three types of feature embeddings based on the aforementioned hierarchical weights. Local feature embeddings are weighted with local hierarchical fusion coefficients, mid-level feature embeddings are weighted with mid-level hierarchical fusion coefficients, and global feature embeddings are weighted with global hierarchical fusion coefficients. The weighted results are then combined to obtain the fused feature. The fused feature simultaneously contains fine-grained information, regional structure information, and overall layout information, achieving a collaborative expression of multi-scale information.

[0055] After the fusion result is generated, the fused features are subjected to linear projection processing by the feature mapping unit, transforming the fused features into a unified representation vector. The unified representation vector is structurally a single feature vector or feature set, which can simultaneously reflect information at different scales in the numerical space, thus providing a stable input for subsequent matching or recognition processing.

[0056] In fintech business scenarios, mobile robots collect images in branch environments and generate multi-level feature embeddings. Local feature embeddings reflect detailed equipment information, mid-level feature embeddings reflect the relationship between counters and aisle areas, and global feature embeddings reflect the overall branch layout. By fusing these hierarchical features to form a unified representation vector, the robot can distinguish different layout structures in structurally similar branch environments, thereby improving inspection and positioning accuracy.

[0057] In healthcare scenarios, mobile robots collect images in health behavior monitoring environments and generate multi-level feature embeddings. Local feature embeddings reflect local human movements, mid-level feature embeddings reflect the relationships between action combinations, and global feature embeddings reflect the overall behavioral path structure. By fusing these hierarchical features to form a unified representation vector, the robot can distinguish between scenarios with similar actions but different structural relationships during behavior recognition and location determination, thereby improving recognition stability.

[0058] This implementation performs hierarchical feature fusion processing on local feature embedding, mid-level feature embedding, and global feature embedding, so that information at different spatial scales is integrated in a unified representation, thereby simultaneously preserving detailed features, regional structural features, and overall layout features, and improving the integrity and distinguishability of feature representation in complex environments.

[0059] S60, perform scene matching processing based on the unified representation vector to obtain the matching result.

[0060] In this embodiment, the unified representation vector represents a single feature representation result formed after multi-level visual representation, spatial structure constraint processing, and hierarchical feature fusion processing. This representation result simultaneously includes local detail information, regional organization information, and overall layout information. Scene matching processing based on the unified representation vector means using the unified representation vector as query input and comparing it with a pre-established reference feature set to determine the environment category, environment location, or target scene identity corresponding to the current input image. The matching result represents the final determination information output by the scene matching processing. The matching result can be represented as a target scene label, scene number, location identifier, or corresponding environment index, used to characterize the optimal corresponding object of the unified representation vector in the reference feature set.

[0061] Scene matching processing relies on the construction of a reference feature database. This database contains multiple reference representation vectors and corresponding scene labels. The reference representation vectors are derived from feature extraction results of known environment images, while the scene labels are derived from environment annotation information. The establishment of the reference feature database includes environment image acquisition, environment image encoding, feature storage, and label binding. Environment image acquisition is used to obtain multi-view image data in a known space; environment image encoding is used to convert images into comparable vector representations; feature storage is used to establish a vector index structure; and label binding is used to associate each reference representation vector with an actual scene identity. After entering the scene matching processing stage, the unified representation vectors are used as the vectors to be matched in the similarity measurement calculation with multiple reference representation vectors.

[0062] Similarity measurement is used to assess the closeness between a unified representation vector and each reference representation vector. Similarity measurement can be implemented through vector angle relationships, vector distance relationships, or normalized correlation relationships. To ensure the comparability of comparison results between the unified representation vector and the reference representation vectors, numerical normalization can be performed on both before performing the similarity measurement, ensuring that different feature vectors are within a consistent numerical range. The similarity measurement process outputs multiple similarity scores, each corresponding to a reference representation vector, reflecting the degree of matching between the unified representation vector and its corresponding reference representation vector.

[0063] After obtaining multiple similarity scores, a ranking and filtering process is performed. This process arranges the multiple reference representation vectors in order of matching degree and determines the target reference representation vector that is closest to the unified representation vector. Ranking and filtering can be performed using a single highest score or by retaining the top few high-scoring objects, with final filtering based on additional constraints. Once the target reference representation vector is determined, target reference scene labels bound to it are extracted from the reference feature database, and these labels are used as the matching results. This process transforms the unified representation vector from an abstract feature representation into a directly usable environment recognition result.

[0064] In robot visual position recognition applications, the unified representation vector corresponds to the environmental feature representation from the robot's current perspective, while the reference representation vector corresponds to the visual feature representation of each location or environmental unit in the known map. Scene matching processing determines which scene on the map has the highest correspondence between the robot's current environment and the known environment by comparing the feature consistency between the current perspective and the known environment. In fintech business scenarios, robots can move within branch offices, self-service areas, or equipment management areas. The unified representation vector is used to represent the current inspection view, and the matching result is used to determine the current business area or target equipment environment. In healthcare business scenarios, robots can move within rehabilitation training environments or health behavior monitoring environments. The unified representation vector is used to represent the current activity area view, and the matching result is used to determine the current activity area or target behavior scene. Through this processing method, the unified representation vector is transformed into a matching result with clear business semantics.

[0065] If a learning-based matching model is used for scene matching, a matching network structure needs to be constructed. This matching network structure can include a vector input unit, a similarity calculation unit, a ranking unit, and a result output unit. The vector input unit receives a unified representation vector and a reference representation vector; the similarity calculation unit outputs a similarity score; the ranking unit selects the target reference representation vector; and the result output unit outputs the matching result. The training data consists of unified representation vectors generated from labeled scene images and their corresponding reference scene labels. During training, similar scene vectors and dissimilar scene vectors are compared and constrained. Parameter updates reduce the vector distance within the same scene and increase the vector distance between different scenes. Training parameters include batch size, feature dimension, learning rate, training epochs, and a similarity threshold. Parameter updates are achieved through gradient propagation. In fintech businesses, input data can be set as image features of branch outlets, equipment areas, and passageways, and output data can be set as labels for the corresponding business areas. In healthcare businesses, input data can be set as image features of activity areas, rehabilitation areas, and behavioral areas, and output data can be set as labels for the corresponding scene or area.

[0066] This embodiment compares the uniform representation vector with multiple reference representation vectors in the reference feature database based on similarity, and determines the target reference representation vector and the corresponding target reference scene label based on the comparison results. This allows the uniform representation vector formed in the previous step to be directly transformed into a matching result with a clear environmental meaning, thereby enhancing the determinability and applicability of the scene recognition results.

[0067] In one embodiment, step S10 above includes: S101, Input the input image into the visual coding network to perform a staged feature extraction operation to obtain a multi-scale pyramid feature map; S102, perform multi-scale feature extraction operation on the multi-scale pyramid feature map according to different preset receptive field scales to generate high-resolution local feature map, mid-level feature map and low-resolution global feature map respectively. S103, Perform spatial dimension mapping operation on the high-resolution local feature map, the mid-level feature map and the low-resolution global feature map respectively to obtain the local one-dimensional feature sequence, the mid-level one-dimensional feature sequence and the global one-dimensional feature sequence. S104, Visual tag generation processing is performed on the local one-dimensional feature sequence, the middle-level one-dimensional feature sequence and the global one-dimensional feature sequence respectively to obtain local visual tags, middle-level visual tags and global visual tags.

[0068] In this embodiment, the input image represents the raw visual data entering the visual coding network. The data format is a two-dimensional pixel array. The spatial position of the pixel array determines the geometric distribution of the image content, and the pixel value determines brightness, texture, color, or grayscale information. Before entering the visual coding network, the input image can maintain its original resolution or be uniformly sized before entering the network for computation. However, this processing does not change the functional positioning of the current stage, which revolves around multi-level visual coding. The visual coding network is used to convert pixel domain information into feature domain information. It consists of multiple sequentially connected feature extraction layers, and the layers form a hierarchical abstract structure through feature transfer relationships. This network can use convolutional coding structures, attention coding structures, or hybrid convolutional and attention connection structures. Convolutional structures are suitable for maintaining local spatial continuity, attention structures are suitable for establishing response relationships between distant regions, and hybrid connection structures are suitable for simultaneously maintaining local texture expression and cross-regional correlation expression. After entering the visual coding network, the input image no longer directly participates in the generation of subsequent local visual labels, mid-level visual labels, and global visual labels. Subsequent calculations are all based on the feature results output by the visual coding network.

[0069] Staged feature extraction refers to the sequential execution of feature extraction, feature transformation, and feature compression across different computational layers of a visual encoding network. The term "staged" is not limited to a fixed number of layers but signifies the existence of at least multiple interconnected feature processing layers, with each layer's output contributing to the input of the next. Early layers more closely approximate pixel distribution, exhibiting high responsiveness to edges, corners, texture undulations, and local brightness variations; intermediate layers more closely approximate region combinations, exhibiting high responsiveness to local components, region contours, and local layouts; later layers more closely approximate the overall organization, exhibiting high responsiveness to large-scale structures, long-distance relationships, and global distributions within the scene. The results of staged feature extraction are expressed as multi-scale pyramid feature maps. "Multi-scale" in multi-scale pyramid feature maps indicates that feature maps of different resolutions, receptive ranges, and semantic abstraction levels collectively constitute a hierarchical feature set; "pyramid" indicates that the feature maps decrease in spatial size from large to small and increase in semantic abstraction from low to high. This pyramid structure allows fine-grained textures, region-level structures, and overall layouts in the input image to be preserved in parallel within the same visual encoding network, rather than being compressed into a uniform representation at a single scale.

[0070] Multi-scale feature extraction is performed on multi-scale pyramid feature maps based on different preset receptive field scales, representing the feature representation of local, intermediate, and global layers distinguished by spatial coverage from the multi-scale pyramid feature maps. The receptive field scale represents the size of the coverage area corresponding to a feature unit in the input image; a smaller coverage area is more focused on detail resolution, while a larger coverage area is more focused on overall relationship expression. Different preset receptive field scales can be determined by the convolution kernel size, stride, window division range, attention computation range, or layer downsampling ratio. High-resolution local feature maps originate from the shallow layers of the pyramid or feature layers maintaining a high spatial size. The input area corresponding to a single feature position in this feature map is relatively small, thus accurately reflecting texture boundaries, local component outlines, and fine-grained structural differences. Intermediate feature maps originate from the intermediate layers of the pyramid; their spatial size is smaller than that of high-resolution local feature maps, but they have stronger semantic aggregation capabilities, expressing the internal organization of a region and the relationship between adjacent regions. Low-resolution global feature maps originate from the upper layers of the pyramid; their spatial size is further compressed, and the input area corresponding to a single feature position is larger, making them more suitable for expressing global layout, scene geometric organization, and long-distance semantic relationships. This multi-scale feature extraction operation does not simply replicate features from three scales. Instead, it uses different receptive fields to correspond to different spatial granularities, enabling the subsequently obtained local visual labels, mid-level visual labels, and global visual labels to undertake information expression tasks at different levels.

[0071] After high-resolution local feature maps, mid-level feature maps, and low-resolution global feature maps are formed, a spatial dimension mapping operation is performed. Spatial dimension mapping transforms the organization of two-dimensional or three-dimensional feature maps into a structural representation suitable for sequence computation. Each spatial location in the high-resolution local feature map corresponds to a local feature unit. The spatial dimension mapping operation expands all local feature units in a preset order while maintaining the correspondence between units, thus forming a local one-dimensional feature sequence. Mid-level feature maps and low-resolution global feature maps are processed in the same way, forming mid-level one-dimensional feature sequences and global one-dimensional feature sequences, respectively. Each sequence element in the one-dimensional feature sequence contains semantic and spatial location information of the corresponding position in the original feature map. Spatial location information can be preserved through absolute position encoding, relative position encoding, or coordinate mapping, thereby avoiding the loss of spatial order after sequence expansion. Local one-dimensional feature sequences emphasize fine-grained spatial order, mid-level one-dimensional feature sequences emphasize region-level organizational order, and global one-dimensional feature sequences emphasize overall structural order. Spatial dimension mapping does not change the source of feature content, but only changes the organizational form of features, enabling subsequent visual label generation processing to be completed on a unified sequence representation.

[0072] Visual tag generation processing transforms local, mid-level, and global one-dimensional feature sequences into tag representations that can directly participate in subsequent feature interactions. Local visual tags originate from fine-grained sequence elements within the local one-dimensional feature sequence; each local visual tag corresponds to a local spatial unit, and its feature representation focuses more on local texture, boundary differences, local components, and near-field structures. Mid-level visual tags originate from region-level sequence elements within the mid-level one-dimensional feature sequence; each mid-level visual tag corresponds to a medium-scale region, and its feature representation focuses more on the compositional relationships within the region, the combination of regions, and the local spatial layout. Global visual tags originate from global-level sequence elements within the global one-dimensional feature sequence; each global visual tag corresponds to a large-scale spatial unit, and its feature representation focuses more on the overall layout, global structural organization, and long-range dependencies. Visual tag generation processing can be accomplished through linear mapping, normalization computation, sequence transformation, or embedding projection, with the aim of bringing one-dimensional feature sequences of different levels into a unified tag representation space while preserving the differences in their respective scale features. Local visual markers, mid-level visual markers, and global visual markers thus form three sets of visual expression results with distinct hierarchical levels, different semantic functions, and different spatial granularities.

[0073] The visual encoding network comprises an input embedding layer, multiple staged feature extraction layers, a scale transformation layer, and a label generation layer. The input embedding layer converts image patches or local regions in the input image into initial feature representations; the staged feature extraction layers generate multi-scale pyramid feature maps level by level; the scale transformation layer organizes features at different levels into high-resolution local feature maps, mid-level feature maps, and low-resolution global feature maps; and the label generation layer outputs local visual labels, mid-level visual labels, and global visual labels. During training, the input data consists of scene image samples, and the output data consists of multi-level visual encoding results. If the training objective is robot visual position recognition, the sample organization can employ a pairing format of images from different perspectives of the same scene with images from different scenes, enabling the visual encoding network to learn to distinguish detailed differences at the local scale, express region organization at the intermediate scale, and express layout relationships at the global scale.

[0074] In a fintech business environment, input images can come from indoor visual data acquired by branch inspection robots, self-service terminal inspection robots, or asset inventory robots. High-resolution local feature maps can be used to express details such as device panels, interface structures, marking areas, and local risk indicators; mid-level feature maps can be used to express the organizational information of counter units, self-service terminal areas, aisle areas, and device group areas; and low-resolution global feature maps can be used to express the overall business layout, area distribution, and spatial orientation. After spatial dimension mapping and visual tagging, local visual tags, mid-level visual tags, and global visual tags undertake device-level, area-level, and environment-level expression tasks, respectively. In a healthcare business environment, input images can come from environmental visual data acquired by rehabilitation companion robots, health monitoring robots, or behavior acquisition robots. High-resolution local feature maps can be used to express details such as local human movements, instrument edges, and local interaction points; mid-level feature maps can be used to express the relationships between action areas, activity areas, and equipment usage areas; and low-resolution global feature maps can be used to express the overall activity environment layout, traffic direction, and scene organization. Through visual tag generation processing, the robot obtains multi-level visual representation results closely related to the spatial structure of the scene, preserving the fine-grained, medium-grained, and global-grained visual foundation for subsequent processing stages outside the current step.

[0075] This embodiment performs phased feature extraction, multi-scale feature extraction, spatial dimension mapping, and visual label generation on the input image through a visual coding network. It can generate local visual labels, mid-level visual labels, and global visual labels respectively, so that fine-grained differences, regional organizational relationships, and overall layout information are expressed separately while maintaining a unified feature space, thereby improving the hierarchical expression and differentiation capabilities of visual information in complex indoor environments.

[0076] In one embodiment, step S20 above includes: S201, Perform pixel-level semantic category segmentation processing on the input image to obtain a semantic segmentation map; S202, Parse object-level entity nodes, room-level partition nodes, and scene-level structural anchor points from the semantic segmentation graph; S203, obtain the relative spatial arrangement of the object-level entity nodes in the two-dimensional plane, and establish the object spatial relationship between the object-level entity nodes based on the relative spatial arrangement to obtain a local topology map; S204, extract the regional connectivity status between the room-level partition nodes, and establish regional spatial connectivity relationships between the room-level partition nodes based on the regional connectivity status to obtain the mid-level topology graph. S205, extract the macroscopic topological connection state between the scene-level structural anchor points, and establish scene spatial connectivity between the scene-level structural anchor points based on the macroscopic topological connection state to obtain a global topology graph.

[0077] In this embodiment, the input image serves as the semantic parsing object in the current processing stage. Each pixel in the input image contains both spatial coordinate information and observational information reflecting changes in material, boundaries, texture, brightness, and color. Performing pixel-level semantic category classification processing on the input image means that instead of treating the entire image as a single whole for coarse-grained judgment, the semantic attribution of each pixel is determined separately, clearly dividing different regions in the image into category regions with stable semantic meanings. Pixel-level semantic category classification processing corresponds to the reasoning process of the semantic segmentation model, which includes at least a feature extraction layer, a semantic aggregation layer, and a pixel classification layer. The feature extraction layer extracts multi-scale visual features from the input image, the semantic aggregation layer fuses shallow spatial boundary information with deep semantic information, and the pixel classification layer outputs the category distribution results for each pixel location. The input data is scene images collected by the robot in an indoor environment, and the output data is a semantic category distribution map corresponding to the spatial size of the input image, forming a semantic segmentation map after category determination. Each pixel location in the semantic segmentation map corresponds to a clear semantic category identifier, and different category identifiers together constitute the region boundary basis and node generation basis for subsequent topology parsing processing.

[0078] Semantic segmentation graphs are not simply sets of pixel classification results, but rather structured semantic representations constrained by spatial continuity, region integrity, and category consistency. Adjacent pixels in a semantic segmentation graph that share the same category and satisfy connectivity conditions are considered the same semantic region; those that share the same category but are spatially separated are considered different semantic regions. This approach allows the semantic segmentation graph to simultaneously preserve both category and region information. In this current stage, the semantic segmentation graph serves as both the output of pixel-level semantic category partitioning and the direct input for topology analysis. The role of the semantic segmentation graph is not to provide an abstract set of semantic labels, but rather to provide region boundaries, region categories, region adjacency relationships, and spatial distribution relationships, giving subsequent node analysis and relationship modeling a clear computational object.

[0079] Parsing object-level entity nodes, room-level partition nodes, and scene-level structural anchors from the semantic segmentation graph represents a shift from simple region representation to hierarchical structural representation. Object-level entity nodes correspond to fine-grained semantic regions, typically derived from object regions with independent boundaries and categories in the semantic segmentation graph. The parsing process for object-level entity nodes includes region connectivity extraction, region boundary localization, region center coordinate calculation, and node numbering. Region connectivity extraction aggregates similar pixels into independent regions; region boundary localization determines the spatial outline of the region; region center coordinate calculation forms the positional description required for subsequent relative position analysis; and node numbering establishes the index order for subsequent relational matrices or graph structures. Object-level entity nodes emphasize the representation of fine-grained objects. In robot vision position recognition scenarios, door frames, obstacles, equipment cabinets, service terminals, and nursing instruments can all be organized as object-level entity nodes.

[0080] Room-level partition nodes correspond to a set of semantic regions with medium spatial granularity, derived from a higher-level spatial merging of multiple adjacent regions in the semantic segmentation map. Room-level partition nodes are not simply enlarged object-level entity nodes, but rather functional area representations formed based on region usage, boundary closure relationships, enclosed relationships, and access relationships. When parsing room-level partition nodes, multiple object-level regions and scene boundary regions need to be included in the analysis. Based on openings, passageways, enclosures, partitions, and boundary extension directions, it is determined which regions belong to the same partition and which belong to different partitions. The parsing results allow the medium-level structure to no longer remain at the level of a single object, but to represent medium-scale spatial units such as passageways, operation areas, dwelling areas, observation areas, and service areas.

[0081] Scene-level structural anchors correspond to key regions or structural units that play an organizational role at the global scale. While a large number of scene-level structural anchors are not required, they must represent the main supporting positions or key turning points in the overall layout. These anchors originate from large-scale structural regions, spatial skeleton intersections, major connection points, entrances, pathway changes, or enclosing structure changes in the semantic segmentation map. The parsing of scene-level structural anchors needs to be performed across the entire map, considering not only region categories but also the structural significance of each region within the overall layout. This results in scene-level structural anchors that provide the node foundation for global topology representation, enabling the global topology map to reflect the overall organization of the robot's environment, rather than a simple accumulation of several local regions.

[0082] Obtaining the relative spatial arrangement of object-level entity nodes in a 2D plane represents an explicit modeling of the geometric relationships between nodes at a local level. The relative spatial arrangement in the 2D plane includes the relative distance between node centers, relative direction, top-bottom and left-right distribution, adjacency order, and occlusion relationships. This process does not rely on absolute world coordinates but rather on the pixel coordinates of the input image plane or the plane coordinates after projection transformation. Each object-level entity node has a corresponding positional description in the 2D plane; the positional difference, directional difference, and adjacency relationship between two nodes collectively form the relative spatial arrangement. Based on the relative spatial arrangement, object spatial relationships are established between object-level entity nodes, representing the transformation of discrete positional differences into structural relationship expressions. Object spatial relationships can represent categories such as proximity, spacing, enclosure, juxtaposition, opposition, attachment, or occlusion. After establishment, a local topology graph is obtained. In the local topology graph, nodes are object-level entity nodes, edges represent object spatial relationships, nodes express object-level semantic units, and edges express local structural relationships between objects. The computational significance of local topology graphs lies in enabling fine-grained visual regions to no longer be treated in isolation, but rather organized into local structural networks with clear spatial relationships.

[0083] Extracting the connectivity status between room-level partition nodes represents analyzing whether there are reachable, traversable, switchable, or continuously extending spatial relationships between functional partitions in the mid-level structure. Regional connectivity status is not simply determining whether two regions are in contact, but rather whether there is a connection sufficient to constitute a continuous activity space. This process is typically completed by combining partition boundary openings, connectivity bandwidth, path orientation, and transition conditions between boundaries. In robot vision position recognition scenarios, connectivity status determines whether a robot can enter from one region to another, and also determines whether a strong connection should be established between the two regions in their structural representation. Based on regional connectivity status, establishing regional spatial connectivity relationships between room-level partition nodes represents formalizing the reachability and organizational relationships between mid-level spatial units as graph structure edges. Regional spatial connectivity relationships can describe categories such as direct connectivity, indirect connectivity, restricted connectivity, or non-connectivity. After establishment, a mid-level topology graph is obtained. Nodes in the mid-level topology graph are room-level partition nodes, and edges represent regional spatial connectivity relationships. The mid-level topology diagram represents the organization of regional spatial units such as functional areas, activity areas, service areas, monitoring areas, and training areas, and can reflect the mid-scale connectivity structure of the environment.

[0084] Extracting the macro-topological connection state between scene-level structural anchor points represents determining the organizational relationships between key anchor points at the overall layout level. The macro-topological connection state is not limited to physical contact but emphasizes the existence of path continuity, layout orientation, spatial axis association, regional skeleton association, and core node support relationships within the overall environmental organization. Extracting this state requires analyzing the layout order, path dependence, main path direction, and skeleton branching relationships between scene-level structural anchor points across the entire graph. Based on the macro-topological connection state, establishing scene spatial connectivity relationships between scene-level structural anchor points represents transforming the connection organization method in the overall layout into a global-scale graph structure. Scene spatial connectivity relationships can reflect main path connections, branch layouts, organization methods from entrances to core areas, and connection methods between key turning points. After establishment, a global topology graph is obtained. Nodes in the global topology graph are scene-level structural anchor points, and edges represent scene spatial connectivity relationships. The purpose of the global topology graph is to express the global structure of the environment and distinguish indoor scenes that appear visually similar but have different overall organizational methods, such as environments with different corridor branches, different room connection paths, and different regional skeleton structures.

[0085] Pixel-level semantic category classification relies on semantic segmentation models, while topological parsing relies on rule parsing and graph generation structures. Semantic segmentation models can employ an encoder-decoder connection structure or a multi-scale feature aggregation structure. The encoder extracts multi-layer visual features from the input image, the decoder restores spatial resolution and outputs a category distribution map, and the multi-scale feature aggregation unit transmits semantic and boundary information between different layers. Training data consists of scene images and pixel-level labeled maps. In fintech, input data can be set as robot inspection images of branch offices, self-service areas, and equipment management areas, while output data can be set as semantic category maps of equipment areas, passageways, service areas, and security isolation areas. In healthcare, input data can be set as robot-collected images of rehabilitation training areas, health monitoring areas, and activity areas, while output data can be set as semantic category maps of human body areas, equipment areas, activity areas, and passageways.

[0086] In fintech environments, when robots move through branches, self-service areas, cash service areas, and equipment maintenance areas, the visual images containing equipment objects, passageway objects, service area objects, and enclosing boundary objects need to be converted into multi-level spatial nodes and relationships to distinguish indoor scenes with repetitive structures but different organizational methods. In healthcare environments, when robots move through rehabilitation activity areas, behavior observation areas, training equipment areas, and passageways, human activity objects, equipment objects, region objects, and structural boundary objects need to be converted into multi-level spatial nodes and relationships to distinguish environments with similar behavioral scenarios but different connectivity methods. Through the continuous generation of semantic segmentation graphs, local topology graphs, mid-level topology graphs, and global topology graphs, the input image is elevated from a pixel-based representation to a hierarchical structural representation, enabling subsequent processing to directly utilize object relationships, region relationships, and global organizational relationships.

[0087] This embodiment performs pixel-level semantic category classification processing on the input image and generates a semantic segmentation map. Then, it parses object-level entity nodes, room-level partition nodes, and scene-level structural anchor points from the semantic segmentation map and establishes local topology maps, mid-level topology maps, and global topology maps respectively. This can transform pixel-level visual information into a hierarchical spatial structure expression, so that object relationships, regional connectivity relationships, and overall layout relationships are preserved at the same time, thereby improving the ability to distinguish structurally similar scenes in complex indoor environments.

[0088] In one embodiment, step S30 above includes: S301, extract the local topology nodes contained in the local topology graph, the middle-layer topology nodes contained in the middle-layer topology graph, and the global topology nodes contained in the global topology graph, and obtain the node center coordinates of the local topology nodes, the middle-layer topology nodes, and the global topology nodes in the two-dimensional plane respectively; S302, based on the node center coordinates, determine the local relationship status between the local topology nodes, the intermediate relationship status between the intermediate topology nodes, and the global relationship status between the global topology nodes respectively; S303, based on the local relation state, the intermediate relation state, the global relation state, and the preset Gaussian kernel decay distribution, determine the local spatial association weight between the local topology nodes, the intermediate spatial association weight between the intermediate topology nodes, and the global spatial association weight between the global topology nodes, respectively. S304, the local topology graph is transformed into a local adjacency matrix based on the local spatial association weight, the mid-level topology graph is transformed into a mid-level adjacency matrix based on the mid-level spatial association weight, and the global topology graph is transformed into a global adjacency matrix based on the global spatial association weight.

[0089] In this embodiment, local, mid-level, and global topology graphs correspond to structural relationship expressions at different spatial granularities. Nodes in the local topology graph reflect the spatial distribution between fine-grained objects, nodes in the mid-level topology graph reflect the organizational relationships between regional units, and nodes in the global topology graph reflect the key structural locations in the overall environment. Further transforming these three types of topology graphs into local, mid-level, and global adjacency matrices aims to convert the node relationship expressions in the graph structure into matrix expressions that can directly participate in numerical calculations, enabling subsequent calculations to read spatial constraint information at different levels within a unified data format. Row and column indices in the matrix correspond to node numbers, and matrix elements correspond to the degree of association between nodes. Therefore, the adjacency matrix preserves the hierarchical structure of the topology graph while allowing for continuous quantification of spatial relationships.

[0090] Extracting local topological nodes from the local topological graph, mid-level topological nodes from the mid-level topological graph, and global topological nodes from the global topological graph represents reading node sets from the graph structures at three different levels. Node extraction is not simply reading node identifiers; it involves forming structured node data required for subsequent matrix calculations. Local topological nodes typically correspond to local object regions or local structural units, mid-level topological nodes typically correspond to room-level partitions or region-level units, and global topological nodes typically correspond to scene-level structural anchor points. Each node, in addition to retaining its category, also needs to retain its boundary, coverage area, and index position in the graph. After the node sets are formed, the center coordinates of the local, mid-level, and global topological nodes in the two-dimensional plane are further obtained. Node center coordinates are used to establish the spatial relationships between nodes and are the direct basis for subsequently determining the local, mid-level, and global relationship states. Node center coordinates can be calculated from the geometric center, bounding box center, or region centroid of the region corresponding to the node. The geometric center is more suitable for regions with regular boundaries, the bounding box center is more suitable for fast calculations, and the region centroid is more suitable for irregularly shaped regions. Different calculation methods do not change the role of the node center coordinates; their role is to provide a unified reference for comparing the positions of nodes.

[0091] Based on the node center coordinates, the local relationship states between local topological nodes, the intermediate relationship states between mid-level topological nodes, and the global relationship states between global topological nodes are determined, representing independent modeling of node relationships at three levels. Local relationship states focus on the relative positions, adjacency states, contact states, occlusion order, and local reachability of fine-grained objects. The determination of local relationship states can be based on the distance between node center coordinates, direction difference, overlap ratio, and boundary proximity. Local topological nodes that are close together and directionally continuous are usually assigned stronger local associations, while those with obvious occlusion or separation are usually assigned weaker local associations. Intermediate relationship states focus on the organizational relationships, connectivity relationships, penetrating relationships, and transitional relationships between functional zones between regions. In addition to relying on the relative positions between node center coordinates, the determination of intermediate relationship states also needs to consider the opening status of region boundaries, travel directions, and region connectivity conditions to distinguish between different intermediate structural states such as adjacent but not connected, adjacent and connected, and far-distance but penetrable. The global relational state focuses on the main path relationships, main region relationships, long-distance spatial organization relationships, and overall environmental skeleton relationships within the overall layout. It doesn't solely rely on proximity but pays more attention to the macroscopic organization between scene-level structural anchor points, such as the direction of main paths, the order of key area distribution, and the consistency of overall spatial axial relationships with the global layout. Determining local, mid-level, and global relational states hierarchically avoids mixing relationships of different spatial granularities under the same judgment criterion, ensuring that each of the three levels maintains an expression scale consistent with its corresponding topology.

[0092] Based on local relation states, mid-level relation states, global relation states, and a pre-defined Gaussian kernel decay distribution, the local spatial association weights between local topological nodes, the mid-level spatial association weights between mid-level topological nodes, and the global spatial association weights between global topological nodes are determined, respectively, representing a further transformation of discrete relation states into continuous weight expressions. The local, mid-level, and global relation states indicate whether nodes are associated and the association category, while the Gaussian kernel decay distribution shows the weight change law of nodes decreasing with increasing spatial distance. After the introduction of the Gaussian kernel decay distribution, even nodes in the same relation category can still reflect different association strengths based on actual distance differences. Closer nodes receive higher weights under the Gaussian kernel decay effect, while farther nodes receive lower weights. The resulting local, mid-level, and global spatial association weights are no longer simple binary relationships, but continuous quantities generated by combining relation states and spatial distance. Local spatial association weights emphasize fine-grained differences between local objects, mid-level spatial association weights emphasize differences in regional organizational strength, and global spatial association weights emphasize the degree of coupling of key structures in the overall layout. The preset Gaussian kernel decay distribution can use a uniform bandwidth, or different decay widths can be set for the local, intermediate, and global levels. A narrower decay width is suitable for the local level to enhance the ability to distinguish close-range details; a medium decay width is suitable for the intermediate level to balance local proximity and regional connectivity; and a wider decay width is suitable for the global level to preserve long-distance structural relationships. Even when expressed in a uniform form, the role of the Gaussian kernel decay distribution in all three levels remains the same: to stably incorporate spatial distance factors into the calculation of node relationship weights.

[0093] The local topology graph is transformed into a local adjacency matrix based on local spatial association weights. This means that the local spatial association weights between any two local topology nodes are written into the corresponding elements of the matrix according to the index order of the local topology nodes. Each row and column of the matrix corresponds to a local topology node, and the matrix elements correspond to the local spatial association weights between two nodes. After the local adjacency matrix is ​​formed, the node relationships in the local topology graph, expressed based on the graph structure, are transformed into weight relationships expressed based on the matrix. The mid-level topology graph is transformed into a mid-level adjacency matrix based on mid-level spatial association weights. This means that the mid-level spatial association weights between any two mid-level topology nodes are written into the corresponding elements of the matrix according to the index order of the mid-level topology nodes. The mid-level adjacency matrix retains the regional organizational relationships, expressing the degree of connectivity, connection, and organizational tightness between regions. The global topology graph is transformed into a global adjacency matrix based on global spatial association weights. This means that the global spatial association weights between any two global topology nodes are written into the corresponding elements of the matrix according to the index order of the global topology nodes. The global adjacency matrix retains the main structural relationships and global organizational strength in the overall layout. Although all three adjacency matrices originate from topology graph transformations, they have different functional focuses. Local adjacency matrices emphasize detailed structure, mid-level adjacency matrices emphasize regional organization, and global adjacency matrices emphasize overall layout. Constructing these three separately ensures that spatial relationships at different levels are fully preserved, without losing information at specific scales due to uniform compression into a single matrix.

[0094] In fintech scenarios, when robots move through branches, self-service areas, and equipment inspection areas, local topology nodes can correspond to fine-grained objects such as terminal devices, access control components, and counter components; mid-level topology nodes can correspond to medium-scale units such as business areas, passageways, and service areas; and global topology nodes can correspond to structural anchor points such as entrance locations, main passageway turning points, and area skeleton locations. By calculating local, mid-level, and global spatial association weights, device-level relationships, region-level relationships, and overall layout relationships can be written into the local adjacency matrix, mid-level adjacency matrix, and global adjacency matrix, respectively. In healthcare scenarios, when robots move through rehabilitation training environments, health monitoring environments, and behavior acquisition environments, local topology nodes can correspond to instrument objects, local human body regions, and local obstacle units; mid-level topology nodes can correspond to training areas, activity areas, and observation areas; and global topology nodes can correspond to key structural turning points, main passageways, and scene skeleton locations within the environment.

[0095] For example, the formula for generating adjacency matrix elements is:

[0096] in, This represents the adjacency matrix element values ​​between the i-th and j-th nodes at the l-th level, used to characterize the spatial association strength between the two nodes. l represents the level index, corresponding to the local, intermediate, or global level. i represents the index of the first node currently participating in the relationship calculation. j represents the index of the second node currently participating in the relationship calculation. This represents the position coordinates of the i-th node in the two-dimensional plane. This represents the position coordinates of the j-th node in the two-dimensional plane. This represents the squared Euclidean distance between the i-th node and the j-th node, used to measure the spatial proximity of the two nodes in the plane. This represents the bandwidth parameter corresponding to the l-th level, used to control the distance decay rate. When the bandwidth parameter is small, the weight difference between neighboring nodes is more obvious; when the bandwidth parameter is large, more distant nodes can still retain a certain weight. exp() represents the exponential mapping, used to generate a Gaussian decay term based on the distance between nodes, so that the larger the distance between nodes, the smaller the corresponding weight. () represents the path connectivity constraint function, which is used to constrain the Gaussian decay term based on the reachability and relationship between nodes. This indicates whether the path between node i and node j is unobstructed, reflecting whether there is a direct reachable condition between the two nodes. This represents the state value of the relationship between node i and node j, used to characterize the spatial relationship category or connectivity between the two nodes.

[0097] It can be further written as:

[0098] in, =1 indicates that node i and node j have the strongest spatial association. =0.5 indicates that there is a restricted connectivity relationship between node i and node j. =0 indicates that there is no valid spatial association between node i and node j.

[0099] This embodiment extracts nodes from the local topology graph, the intermediate topology graph, and the global topology graph, determines the relationship status, introduces a Gaussian kernel decay distribution, and generates local adjacency matrices, intermediate adjacency matrices, and global adjacency matrices respectively. This enables the structural relationships at different spatial scales to be transformed into continuous weighted matrix expressions, allowing fine-grained object relationships, regional organization relationships, and overall layout relationships to be preserved and directly participate in subsequent numerical calculations.

[0100] In one embodiment, step S40 above includes: S401, perform linear mapping processing on the local visual markers to generate local query vectors, local key vectors and local value vectors; perform linear mapping processing on the mid-level visual markers to generate mid-level query vectors, mid-level key vectors and mid-level value vectors; perform linear mapping processing on the global visual markers to generate global query vectors, global key vectors and global value vectors. S402, perform matrix multiplication on the local query vector and the transposed local key vector to obtain the local initial attention distribution; perform matrix multiplication on the mid-level query vector and the transposed mid-level key vector to obtain the mid-level initial attention distribution; perform matrix multiplication on the global query vector and the transposed global key vector to obtain the global initial attention distribution. S403, perform topological constraint modulation processing on the local initial attention distribution based on the local adjacency matrix, and perform normalization mapping processing to obtain local topological modulation weights; perform topological constraint modulation processing on the mid-layer initial attention distribution based on the mid-layer adjacency matrix, and perform normalization mapping processing to obtain mid-layer topological modulation weights; perform topological constraint modulation processing on the global initial attention distribution based on the global adjacency matrix, and perform normalization mapping processing to obtain global topological modulation weights; S404, perform weighted interaction processing on the local topology modulation weights and the local value vector, and perform linear mapping processing to obtain local feature embedding; perform weighted interaction processing on the mid-layer topology modulation weights and the mid-layer value vector, and perform linear mapping processing to obtain mid-layer feature embedding; perform weighted interaction processing on the global topology modulation weights and the global value vector, and perform linear mapping processing to obtain global feature embedding.

[0101] In this embodiment, the local adjacency matrix, the mid-level adjacency matrix, and the global adjacency matrix correspond to the numerical relationship expressions of the local, mid-level, and global spatial structures, respectively. The local visual markers, mid-level visual markers, and global visual markers correspond to the encoding results of fine-grained visual regions, region-level visual units, and overall layout visual units, respectively. Performing topological constraint-based cross-attention processing on local visual markers based on the local adjacency matrix means that spatial relationship information is directly embedded into the attention weight calculation process at the local granularity level. This ensures that information exchange between local visual markers is no longer solely determined by appearance similarity but is also constrained by the connection strength, proximity, and positional relationships of the local spatial structure. Performing topological constraint-based cross-attention processing on mid-level visual markers based on the mid-level adjacency matrix means that region connectivity, region organization, and region reachability are incorporated into the attention modeling process at the region granularity level, enabling region-level visual representations to reflect spatial organization methods. Based on the global adjacency matrix, cross-attention processing based on topological constraints is performed on global visual labels. This means that the main path relationships, key structural anchor point relationships, and overall organizational relationships in the environmental layout are incorporated into the attention modeling process at the overall granularity, enabling the global visual representation to reflect scene-level structural differences. Local feature embedding, mid-level feature embedding, and global feature embedding are the output results of the above three cross-attention processing methods, which differ in spatial granularity but maintain a uniformly processable vector representation in terms of data format.

[0102] Linear mapping is performed on local visual markers to generate local query vectors, local key vectors, and local value vectors. The same type of processing is applied to mid-level and global visual markers, generating mid-level query vectors, mid-level key vectors, mid-level value vectors, as well as global query vectors, global key vectors, and global value vectors, respectively. This represents projecting the same set of visual markers into different functional spaces. The query vector acts as a relation, representing the information demand of the current feature location from other feature locations; the key vector acts as a relation response, representing whether the current feature should be considered by other locations; and the value vector carries the content, storing the semantic information ultimately used in feature fusion. The linear mapping is accomplished through a learnable parameter matrix, enabling local, mid-level, and global visual markers to have differentiated representation capabilities in different functional spaces. This processing is not a mere formal vector copying, but rather applies different parameter transformations to each visual marker, thus allowing the query vector, key vector, and value vector to achieve functional differentiation based on the same input. Local query vectors, local key vectors, and local value vectors focus on expressing the relationships between detailed information; mid-level query vectors, mid-level key vectors, and mid-level value vectors focus on expressing the relationships between regions; and global query vectors, global key vectors, and global value vectors focus on expressing the relationships between the overall layout.

[0103] Matrix multiplication is performed on the local query vector and its transposed local key vector to obtain the initial local attention distribution, representing the original interaction strength between local visual markers established through vector correlation calculation. Each element of the initial local attention distribution corresponds to the matching degree between this local visual marker and other local visual markers; the higher the matching degree, the stronger the correlation between the two in the feature space. Matrix multiplication is performed on the mid-level query vector and its transposed mid-level key vector to obtain the mid-level initial attention distribution, and matrix multiplication is performed on the global query vector and its transposed global key vector to obtain the global initial attention distribution. These three distributions establish initial relationship graphs at different spatial granularities. The local initial attention distribution reflects the original similarity between fine-grained units, the mid-level initial attention distribution reflects the original similarity between region-level units, and the global initial attention distribution reflects the original similarity between overall layout units. At this stage, topological constraints have not yet been introduced; therefore, the high weights in the distribution may be due to visual appearance similarity or interference from structurally similar but differently positioned units. Calculating the initial attention distribution separately provides a clear input object for the subsequent topology modulation process and ensures that visual similarity relationships and structural constraint relationships are separable at the computational level.

[0104] Topological constraint modulation and normalization mapping are performed on the local initial attention distribution based on the local adjacency matrix to obtain local topological modulation weights. These weights represent the direct application of local spatial structure information to the initial attention distribution. The element values ​​in the local adjacency matrix express the spatial association strength between local topological nodes. Modulation can be achieved through addition, multiplication, or weighted combination, enhancing positions in the local initial attention distribution that align with the real spatial relationships and suppressing those that do not. Normalization mapping transforms the modulated distribution into a stable weight form, ensuring all weights maintain a comparable numerical scale and meet the input requirements for subsequent weighted interaction calculations. The mid-level adjacency matrix performs topological constraint modulation and normalization mapping on the mid-level initial attention distribution to obtain mid-level topological modulation weights. These weights express the constraint results of region-level structural relationships on region-level visual interaction. The global adjacency matrix performs topological constraint modulation and normalization mapping on the global initial attention distribution to obtain global topological modulation weights. These weights express the constraint results of the overall layout relationships on global visual interaction. As a result, the original relationships generated by visual similarity are re-selected and redistributed by topological relationships at the local, intermediate, and global scales, forming a weight structure that is more consistent with the spatial organization of the environment.

[0105] The local topology modulation weights and local value vectors undergo weighted interaction processing, followed by linear mapping, to obtain local feature embeddings. This means that the weights, which have been modulated by local topology constraints, are used to reintegrate the semantic content in the local value vectors. The weighted interaction processing ensures that each local visual unit retains not only its own local value vector information in the output but also integrates the content of other local value vectors with which it has local spatial relationships, thus forming a local feature embedding that simultaneously contains local visual appearance information and local structural relationship information. The mid-level topology modulation weights and mid-level value vectors undergo weighted interaction processing, followed by linear mapping, to obtain mid-level feature embeddings. Mid-level feature embeddings preserve region-level visual organization and region structural relationships. The global topology modulation weights and global value vectors undergo weighted interaction processing, followed by linear mapping, to obtain global feature embeddings. Global feature embeddings preserve the overall layout semantics and global structural relationships. Linear mapping is used to integrate dimensions and reorganize features in the interaction results, ensuring that local, mid-level, and global feature embeddings maintain consistency in the output dimension and further enhance expressive power. Thus, local feature embedding, mid-level feature embedding, and global feature embedding have completed the transformation from visual encoding results to structurally constrained expression results.

[0106] The model structure comprises three input branches and three relation constraint branches. The input branches receive local, mid-level, and global visual labels, respectively, while the relation constraint branches receive local, mid-level, and global adjacency matrices, respectively. Each granularity branch contains at least a linear mapping layer, a matrix correlation calculation layer, a topology constraint modulation layer, a normalization layer, a weighted interaction layer, and an output mapping layer. The linear mapping layer generates the query vector, key vector, and value vector; the matrix correlation calculation layer generates the initial attention distribution; the topology constraint modulation layer incorporates the adjacency matrix into the attention calculation; the normalization layer outputs the topology modulation weights; the weighted interaction layer applies the modulation weights to the value vectors; and the output mapping layer generates the feature embedding. The three granularity branches can be configured with independent parameters or, in some mapping layers, with shared parameters. The training data should include image samples collected by the robot in an indoor environment and corresponding environmental structure information. Input data for fintech businesses can be image samples acquired by branch inspection robots, self-service terminal inspection robots, or asset management robots. These images contain areas such as equipment regions, passageways, and service areas. The corresponding structural input can be a local adjacency matrix, a mid-level adjacency matrix, and a global adjacency matrix obtained through semantic parsing. Input data for healthcare businesses can be image samples acquired by rehabilitation companion robots, health monitoring robots, or behavior recognition robots. These images contain human activity areas, equipment areas, training areas, and passageways. The corresponding structural input is also a three-level adjacency matrix. Output data consists of local feature embeddings, mid-level feature embeddings, and global feature embeddings. During training, contrast constraints, matching constraints, or structural consistency constraints can be used to update parameters, ensuring that embedding results for the same scene or adjacent locations remain similar, and that embedding results for locations with inconsistent topological relationships or different scenes remain distinct. Training parameters include at least the feature dimension, number of branch layers, number of attention heads, learning rate, batch size, and training epochs. The feature dimension determines the embedding capacity, the number of branch layers determines the relationship modeling depth, the number of attention heads determines the ability to model multi-subspace relationships, and the learning rate, batch size, and training epochs determine the parameter update process. Through the above model organization and training settings, local feature embedding, mid-level feature embedding, and global feature embedding can stably express the fine-grained differences, region-level connectivity information, and global layout relationships that are most needed to be preserved in robot visual position recognition tasks.

[0107] For example, the formula for topological constraint modulation attention weights is:

[0108] in, This represents the attention weight between the i-th and j-th labels after topological constraint modulation and normalization. This represents the query vector corresponding to the i-th tag, which can be a local query vector, a mid-level query vector, or a global query vector. This represents the key vector corresponding to the j-th tag, which can be a local key vector, a mid-level key vector, or a global key vector. This represents the transpose of the j-th key vector. d represents the feature dimensions of the query vector and the key vector. This represents the adjacency matrix element value between the i-th node and the j-th node in the l-th level, where l corresponds to local, intermediate, or global. This represents a very small positive number added to avoid taking the logarithm of zero. softmax() represents a normalization mapping operation, used to convert the modulated weights into a comparable probability distribution.

[0109] The formula for generating hierarchical feature embedding is:

[0110]

[0111]

[0112] in, This represents the embedding of local features. This indicates the embedding of mid-level features. This indicates global feature embedding. This represents the local topology modulation weight. This represents the modulation weights of the middle layer topology. This represents the global topology modulation weights. Indicates local visual markers. This indicates a mid-level visual marker. Represents global visual markers. This represents the linear mapping weights at the local level. This represents the linear mapping weights of the middle-level hierarchy. This represents the linear mapping weights at the global level.

[0113] This embodiment generates query vectors, key vectors, and value vectors from local visual markers, mid-level visual markers, and global visual markers, respectively, to form an initial attention distribution. Then, the initial attention distribution is subjected to topological constraint modulation using local adjacency matrices, mid-level adjacency matrices, and global adjacency matrices to generate topological modulation weights. Furthermore, the value vectors are combined to form local feature embeddings, mid-level feature embeddings, and global feature embeddings. This enables visual representations at different spatial granularities to simultaneously include appearance information and structural relationship information, thereby improving the feature discrimination ability in structurally similar indoor environments.

[0114] In one embodiment, step S50 above includes: S501, obtain the local fusion weight matrix, the mid-level fusion weight matrix, and the global fusion weight matrix corresponding to the local feature embedding, the mid-level feature embedding, and the global feature embedding, respectively; S502, perform matrix multiplication on the local feature embedding and the local fusion weight matrix to obtain local weighted embedding; perform matrix multiplication on the mid-layer feature embedding and the mid-layer fusion weight matrix to obtain mid-layer weighted embedding; perform matrix multiplication on the global feature embedding and the global fusion weight matrix to obtain global weighted embedding. S503, perform hierarchical fusion coefficient generation processing based on the local weighted embedding, the middle-layer weighted embedding and the global weighted embedding to obtain the local hierarchical fusion coefficient, the middle-layer hierarchical fusion coefficient and the global hierarchical fusion coefficient; S504, perform local-level fusion processing on the local weighted embedding based on the local-level fusion coefficient to obtain a local fusion result; perform mid-level fusion processing on the mid-level weighted embedding based on the mid-level fusion coefficient to obtain a mid-level fusion result; and perform global-level fusion processing on the global weighted embedding based on the global-level fusion coefficient to obtain a global fusion result. S505, perform a weighted summation process on the local fusion result, the mid-layer fusion result, and the global fusion result to obtain the fusion feature; S506, Perform linear projection processing on the fused features to obtain a unified representation vector.

[0115] In this embodiment, local feature embedding, mid-level feature embedding, and global feature embedding correspond to feature representations at three spatial levels: fine-grained region, region-level structure, and overall layout, respectively. Local feature embedding preserves detailed information such as door frame edges, equipment outlines, appliance local shapes, and corner transitions. Mid-level feature embedding reflects the organizational relationships between channels and functional areas, the connectivity relationships between regions, and the combination relationships of local layouts. Global feature embedding reflects the overall association between the main pathways, the order of region distribution, and key structural anchor points in the entire scene. Hierarchical feature fusion processing is performed on the three types of feature embeddings. The goal is not simply to superimpose the three sets of vectors, but rather to use unified weight mapping, hierarchical coefficient generation, and hierarchical weighted combination to form a comparable, fusionable, and scene-matching unified representation vector in the same vector space, based on information at different scales.

[0116] Obtain the local fusion weight matrix, mid-level fusion weight matrix, and global fusion weight matrix corresponding to local feature embeddings, mid-level feature embeddings, and global feature embeddings, respectively, representing independent mapping parameters for different levels. The fusion weight matrix serves two purposes: firstly, it projects feature embeddings from different levels onto a unified representation space; secondly, it preserves hierarchical differences, ensuring that the local, mid-level, and global layers maintain different semantic emphases when entering the fusion stage. The local fusion weight matrix emphasizes the fidelity of local detailed structures, the mid-level fusion weight matrix emphasizes the compressed expression of regional organizational relationships, and the global fusion weight matrix emphasizes the stable expression of the overall layout structure. The three sets of matrices are independent of each other, preventing excessive homogenization of features at different scales under the same parameters. The matrix size is determined by both the feature dimension and the fusion space dimension. The input dimension comes from the vector lengths of the local, mid-level, and global feature embeddings, and the output dimension is determined by the expected information capacity carried by the unified representation vector. If the original dimension of the local layer is higher than that of the global layer, the local fusion weight matrix has a stronger compression effect; if the original dimension of the global layer is lower, the global fusion weight matrix has a stronger structural alignment effect. This setup ensures that the three types of features are in a unified but not overly compressed expression space before entering the subsequent fusion calculation.

[0117] Matrix multiplication is performed on the local feature embeddings and the local fusion weight matrix to obtain the local weighted embeddings. Similarly, matrix multiplication is performed on the mid-level feature embeddings and the mid-level fusion weight matrix to obtain the mid-level weighted embeddings. Finally, matrix multiplication is performed on the global feature embeddings and the global fusion weight matrix to obtain the global weighted embeddings. This process transforms the original hierarchical features into intermediate representations suitable for unified aggregation through linear mapping. Local weighted embeddings are the result of reorganizing local feature embeddings using the local fusion weight matrix. While preserving local detail differences, irrelevant and redundant dimensions are compressed to a lower proportion, making edge details, local components, and local occlusion variations more suitable for cross-level combination. Mid-level weighted embeddings are mapped from mid-level feature embeddings, focusing on preserving region-level connectivity information, region functional relationships, and region combination methods. Global weighted embeddings are mapped from global feature embeddings, focusing on preserving the overall layout skeleton, path structure, and global topological organization. Matrix multiplication allows the three types of features to enter the fusion space from the original encoding space. Although they originate from different sources, they share consistency in output dimensions and composability in representation. Locally weighted embedding, mid-level weighted embedding, and globally weighted embedding are not directly superimposed with equal weights in the subsequent process, but are further involved in the generation of hierarchical fusion coefficients.

[0118] Hierarchical fusion coefficient generation is performed based on local weighted embedding, mid-level weighted embedding, and global weighted embedding to obtain local, mid-level, and global hierarchical fusion coefficients. These coefficients represent the dynamic evaluation of the contribution of each level to the final unified representation vector under the current input scene conditions. The hierarchical fusion coefficient generation process does not use a fixed ratio but automatically determines the relative importance of detailed, regional, and overall information in the current scene based on the distribution of input features. For scenes with significant differences in local structure but similar overall layout, the local hierarchical fusion coefficient will increase; for scenes where regional organization is the main differentiating factor, the mid-level hierarchical fusion coefficient will increase; and for scenes with more significant changes in the overall spatial skeleton, the global hierarchical fusion coefficient will increase. The hierarchical fusion coefficient generation process can be completed by a dedicated coefficient generation unit. This unit receives local, mid-level, and global weighted embeddings and outputs the three hierarchical fusion coefficients through statistical compression, vector mapping, and normalization transformation. Statistical compression can employ global average aggregation, global maximum aggregation, or regional response summarization to transform features at each layer into condensed representations. Vector mapping projects these condensed representations into comparable weighted responses. Normalization transformation ensures that local, mid-level, and global fusion coefficients are on a uniform numerical scale and meets the stability requirements of subsequent weighted combinations. The resulting coefficients are not additional labels but rather moderating variables that directly drive the formation of local, mid-level, and global fusion results.

[0119] Local-level fusion processing is performed on locally weighted embeddings based on local-level fusion coefficients to obtain local fusion results. Mid-level fusion processing is performed on mid-level weighted embeddings based on mid-level fusion coefficients to obtain mid-level fusion results. Global-level fusion processing is performed on globally weighted embeddings based on global-level fusion coefficients to obtain global fusion results. This indicates that adaptive weight adjustment is completed within each of the three levels. Local-level fusion processing enhances fine-grained components in the locally weighted embeddings that are more relevant to the current scene discrimination, while suppressing local components that are less relevant. Mid-level fusion processing gives higher weights to key components of region organization, region connectivity, and local layout structure. Global-level fusion processing enhances global information such as the overall layout skeleton, main channel direction, and key anchor point order. All three types of fusion results are derived from feature reweighting within their respective levels, thus maintaining hierarchical purity in semantic content and avoiding the introduction of cross-level information aliasing at this stage. Local fusion results still represent fine-grained structural expressions, mid-level fusion results still represent region-level expressions, and global fusion results still represent overall layout expressions. The purpose of this is to ensure that the input received by the subsequent weighted summation process is not the unfiltered intermediate results of the three categories, but the fusion result that has already undergone importance adjustment within its respective level.

[0120] A weighted summation process is performed on the local, mid-level, and global fusion results to obtain a fusion feature. This feature represents the integration of effective information from the three levels into a single fusion representation while maintaining sufficient modeling of hierarchical differences. The weighted summation is not a simple vector addition; rather, based on the hierarchical weight adjustment completed in the previous stage, it combines the local, mid-level, and global fusion results into a new feature vector according to their corresponding contributions. This fusion feature simultaneously carries detail-level discriminative information, region-level organizational information, and overall layout information. Compared to direct concatenation, the weighted summation process reduces feature redundancy while maintaining representation compactness, which is beneficial for maintaining numerical stability of the unified representation vector in subsequent scene matching processing. Compared to fixed-coefficient summation, the weighted summation in this process is based on the dynamic generation of local, mid-level, and global hierarchical fusion coefficients, thus the fusion feature formed under different scenarios has stronger adaptability. Scenarios with similar structures but different details rely more on local fusion results, scenarios with significant differences in regional organization rely more on mid-level fusion results, and scenarios with obvious differences in overall layout rely more on global fusion results. The contributions of the three will change dynamically in the fusion features.

[0121] Linear projection is applied to the fused features to obtain a unified representation vector. This vector represents the mapping of the fused features to the final matching space after hierarchical integration, using unified projection parameters. Linear projection serves two purposes. First, it further eliminates redundant dimensions in the fused features, making the unified representation vector suitable for subsequent similarity comparison processing in terms of length and numerical space. Second, it reintegrates the combined information from different levels of the fused features into a target space more suitable for distinguishing scene differences. The vector length, numerical distribution, and directional relationship of the unified representation vector are jointly determined by the linear projection parameters. Therefore, the unified representation vector inherits information from local, mid-level, and global fusion results, and also possesses an expression form suitable for scene indexing and scene matching. The unified representation vector is not a simple average of local, mid-level, and global information, nor is it a substitute for single-level information. Instead, it is the result of adaptively integrating three types of information—local details, region connectivity, and overall layout—according to scene conditions.

[0122] The model internally includes a local fusion branch, a mid-level fusion branch, a global fusion branch, a hierarchical coefficient generation branch, and a unified projection branch. The local fusion branch receives local feature embeddings and outputs a local weighted embedding and a local fusion result. The mid-level fusion branch receives mid-level feature embeddings and outputs a mid-level weighted embedding and a mid-level fusion result. The global fusion branch receives global feature embeddings and outputs a global weighted embedding and a global fusion result. The hierarchical coefficient generation branch simultaneously receives local weighted embeddings, mid-level weighted embeddings, and global weighted embeddings, and outputs local hierarchical fusion coefficients, mid-level hierarchical fusion coefficients, and global hierarchical fusion coefficients. The unified projection branch receives fused features and outputs a unified representation vector. Training input data includes image samples collected by the robot in indoor scenes, processed to form local feature embeddings, mid-level feature embeddings, and global feature embeddings. In fintech business, input data can be set as three-layer embedding representations obtained from preprocessing of images of branch inspections, self-service terminal areas, and equipment maintenance areas. Output data can be set as a unified representation vector supervision target corresponding to scene labels or a similar scene constraint target. In healthcare applications, input data can be configured as a three-layer embedded representation obtained from preprocessing of rehabilitation training environment images, behavior monitoring environment images, and activity area images. Output data can be configured as a unified representation vector supervision target corresponding to activity scene labels, region labels, or location identifiers. The training steps include feature embedding input, fusion weight matrix mapping, hierarchical fusion coefficient generation, hierarchical fusion processing, weighted summation, linear projection, loss calculation, and parameter update. The loss function can employ scene matching constraints, vector distance constraints, or class discrimination constraints to make unified representation vectors for the same scene or adjacent locations closer in the feature space, while maintaining sufficient separation between unified representation vectors for different scenes. Training parameters include fusion space dimension, batch size, learning rate, training epochs, weight initialization method, and normalization coefficient range. Through these structures and training processes, the unified representation vector can stably express the robot's position-related information in an indoor environment.

[0123] For example, the formula for hierarchical feature fusion is:

[0124] in, This represents the fused feature after the fusion of hierarchical features. This represents the fusion weight corresponding to the local level. This indicates the fusion weight corresponding to the middle layer. This represents the fusion weight corresponding to the global level. This represents the embedding of local features. This indicates the embedding of mid-level features. This indicates global feature embedding.

[0125] This embodiment maps local feature embedding, mid-level feature embedding, and global feature embedding using local fusion weight matrix, mid-level fusion weight matrix, and global fusion weight matrix, respectively. Then, it generates local hierarchical fusion coefficients, mid-level hierarchical fusion coefficients, and global hierarchical fusion coefficients, forming local fusion results, mid-level fusion results, and global fusion results, respectively. After weighted summation and linear projection processing, a unified representation vector is obtained. This can adaptively integrate fine-grained differences, regional organization relationships, and overall layout information into the same vector expression according to scene conditions, thereby improving the completeness and distinguishability of scene representation in complex indoor environments.

[0126] In one embodiment, step S60 above includes: S601, Obtain a pre-constructed reference feature database, the reference feature database including multiple reference representation vectors and reference scene labels corresponding to the multiple reference representation vectors respectively; S602, Perform similarity comparison processing on the unified representation vector and the multiple reference representation vectors respectively to obtain multiple similarity scores; S603, perform sorting and filtering processing on the multiple reference representation vectors based on the multiple similarity scores to obtain the target reference representation vector; S604, extract the target reference scene label corresponding to the target reference representation vector from the reference feature database, and use the target reference scene label as the matching result.

[0127] In this embodiment, the unified representation vector represents a single vector expression formed after multi-level visual encoding, topological constraint cross-attention processing, and hierarchical feature fusion processing. In terms of data form, the unified representation vector is a fixed-dimensional feature vector, and semantically it simultaneously carries information about local structural differences, regional organizational relationships, and overall layout. Scene matching processing based on the unified representation vector involves comparing the currently acquired scene representation with pre-stored environment representations to establish a correspondence between the currently observed scene and a specific scene unit in the known environment. The matching result represents the final output of the scene matching process. The matching result can be represented as a scene identity identifier, scene location identifier, region name, or environment index, used to characterize the corresponding position of the robot's current environment within the known reference environment.

[0128] The system acquires a pre-built reference feature database, indicating that a set of reference data for retrieval and comparison already exists in the system before the current scene matching process begins. The reference feature database includes multiple reference representation vectors and corresponding reference scene labels. Reference representation vectors are derived from vector representations formed after preprocessing of known environment images, and reference scene labels are derived from semantic identifiers formed after labeling the known environment. The construction process of the reference feature database includes scene acquisition, scene annotation, feature generation, label binding, and index organization. Scene acquisition is used to obtain multi-view images of the robot in the known environment; scene annotation is used to assign reference scene labels to each reference scene; feature generation is used to perform the same feature extraction process as the currently queried image on the known environment images to obtain multiple reference representation vectors; label binding is used to establish a one-to-one correspondence between reference representation vectors and reference scene labels; and index organization is used to enable fast access and comparison of multiple reference representation vectors in the reference feature database. The reference feature database does not simply store image files, but rather stores a set of vectors and their labels that can be directly used for numerical comparison; therefore, the content of the reference feature database has retrieval computability and result interpretability.

[0129] Similarity comparison is performed on the unified representation vector and multiple reference representation vectors to obtain multiple similarity scores. This represents the correlation calculation between the current scene representation and all reference representation vectors in the reference feature database. The input to the similarity comparison process includes the unified representation vector and multiple reference representation vectors, and the output is multiple similarity scores. Each similarity score uniquely corresponds to a reference representation vector and is used to measure the closeness between the unified representation vector and that reference representation vector. The higher the similarity score, the closer the current scene is to the corresponding reference scene in the feature space or the more consistent the direction; the lower the similarity score, the greater the difference between the two in the feature space. The similarity comparison process can be performed using vector angle correlation, normalized correlation, or inverse distance. To ensure the comparability of the comparison results between the unified representation vector and multiple reference representation vectors, numerical normalization can be performed on the unified representation vector and multiple reference representation vectors before comparison, so that different vectors are within the same numerical range. After normalization, the unified representation vector retains its directional information, and the multiple reference representation vectors form a unified reference set after normalization, and then the similarity scores are calculated for each one. The resulting multiple similarity scores form the basis for the relative matching and ranking between the current scene and all reference scenes.

[0130] Multiple reference representation vectors are sorted and filtered based on multiple similarity scores to obtain the target reference representation vector. This process organizes the comparison results and determines the optimal candidate. The input to the sorting and filtering process is multiple similarity scores and the correspondence between multiple reference representation vectors, and the output is the target reference representation vector. In the sorting and filtering process, multiple similarity scores are arranged according to their numerical value, and multiple reference representation vectors are rearranged synchronously according to their similarity scores. The reference representation vectors in higher priority positions after sorting are closer to the unified representation vector. The purpose of the filtering process is to determine an optimal reference object from multiple reference representation vectors, or to further determine a single optimal object after retaining a small number of candidate objects. The target reference representation vector represents the selected reference representation vector after sorting and filtering. This vector represents the scene unit in the reference feature database that is closest to the current unified representation vector. The sorting and filtering process can directly determine the target reference representation vector based on the highest similarity, or it can retain multiple high-scoring candidates and then perform a secondary filtering based on additional constraints. Additional constraints can include robot continuous displacement constraints, environmental adjacency constraints, time series consistency constraints, and historical matching smoothing constraints. This process can avoid instability in the selection of the target reference representation vector due to noise from a single observation.

[0131] The process involves extracting target reference scene labels from the reference feature database that correspond to the target reference representation vector. These labels are then used as matching results, representing the conversion of the optimal match in the numerical space into a directly usable environment recognition result. Each reference representation vector in the reference feature database is bound to a reference scene label. After the target reference representation vector is determined during the sorting and filtering process, the corresponding target reference scene label can be directly read based on the binding relationship. Target reference scene labels can be location numbers, scene area names, functional area identifiers, map node numbers, or other label formats suitable for robot navigation and environment recognition. Using the target reference scene label as the matching result signifies that the scene matching process has completed the transformation from a unified representation vector to a business-interpretable result. The matching result is no longer an abstract vector or numerical score, but rather judgment information that can be directly used in robot localization, area confirmation, navigation switching, or business status assessment.

[0132] If the scene matching process adopts a retrieval-based comparison structure, the model internally includes a reference feature database management unit, a vector normalization unit, a similarity calculation unit, a ranking and filtering unit, and a label output unit. The reference feature database management unit stores the correspondence between multiple reference representation vectors and reference scene labels; the vector normalization unit performs numerical standardization on the unified representation vector and multiple reference representation vectors; the similarity calculation unit outputs multiple similarity scores; the ranking and filtering unit determines the target reference representation vector; and the label output unit reads the target reference scene label based on the target reference representation vector and forms the matching result. If the scene matching process uses a learning-based matching network, this network can include a query input branch, a reference input branch, a similarity determination branch, and a result output branch. The query input branch receives the unified representation vector, the reference input branch receives multiple reference representation vectors, the similarity determination branch outputs multiple similarity scores, and the result output branch outputs the target reference scene label. The training data consists of unified representation vectors generated from known scene images and their corresponding reference scene labels. During training, positive and negative samples can be paired to make the unified representation vector under the same scene label closer to the reference representation vector in the feature space, while the vectors under different scene labels remain sufficiently separated in the feature space.

[0133] This embodiment acquires a reference feature database containing multiple reference representation vectors and reference scene labels, performs similarity comparisons between the unified representation vector and multiple reference representation vectors one by one to obtain multiple similarity scores, sorts and filters based on multiple similarity scores to determine the target reference representation vector, and finally extracts the corresponding target reference scene label as the matching result. This can directly transform the unified representation vector into a scene determination result that can be used for robot localization and environmental recognition, thereby improving the stability and interpretability of scene matching in complex indoor environments.

[0134] In one embodiment, a hierarchical scene matching device based on topological constraints is provided, which corresponds one-to-one with the hierarchical scene matching method based on topological constraints described in the above embodiments. (Refer to...) Figure 3 , Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the hierarchical scene matching device based on topological constraints of the present invention. The modules include a visual encoding module 10, a topology parsing module 20, an adjacency matrix generation module 30, a topology constraint cross-attention module 40, a hierarchical feature fusion module 50, and a scene matching module 60. Detailed descriptions of each functional module are as follows: The visual encoding module 10 is used to perform multi-level visual encoding processing on the input image to obtain local visual tags, intermediate visual tags and global visual tags; Topology parsing module 20 is used to perform semantic segmentation processing on the input image to obtain a semantic segmentation map, and to perform topology parsing processing on the semantic segmentation map to obtain a local topology map, a mid-level topology map, and a global topology map; The adjacency matrix generation module 30 is used to generate a local adjacency matrix, a mid-level adjacency matrix, and a global adjacency matrix based on the local topology graph, the mid-level topology graph, and the global topology graph, respectively. The topological constraint cross-attention module 40 is used to perform topological constraint cross-attention processing on the local visual labels, mid-level visual labels, and global visual labels based on the local adjacency matrix, mid-level adjacency matrix, and global adjacency matrix, respectively, to obtain local feature embedding, mid-level feature embedding, and global feature embedding. The hierarchical feature fusion module 50 is used to perform hierarchical feature fusion processing on the local feature embedding, the intermediate feature embedding and the global feature embedding to obtain a unified representation vector; The scene matching module 60 is used to perform scene matching processing based on the unified representation vector to obtain the matching result.

[0135] Specific limitations regarding the topology-constraint-based hierarchical scene matching device can be found in the aforementioned limitations of the topology-constraint-based hierarchical scene matching method, and will not be repeated here. Each module in the aforementioned topology-constraint-based hierarchical scene matching device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0136] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When executed by the processor, the computer program implements the functions or steps of a topology-constrained hierarchical scene matching method on the server side.

[0137] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements client-side functions or steps of a topology-constrained hierarchical scene matching method.

[0138] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Multi-level visual coding processing is performed on the input image to obtain local visual tags, intermediate visual tags, and global visual tags; Semantic segmentation processing is performed on the input image to obtain a semantic segmentation map. Topological parsing processing is then performed on the semantic segmentation map to obtain a local topological map, a mid-level topological map, and a global topological map. Based on the local topology graph, the intermediate topology graph, and the global topology graph, a local adjacency matrix, an intermediate adjacency matrix, and a global adjacency matrix are generated, respectively. Based on the local adjacency matrix, the intermediate adjacency matrix, and the global adjacency matrix, topological constraint cross-attention processing is performed on the local visual labels, intermediate visual labels, and global visual labels respectively to obtain local feature embedding, intermediate feature embedding, and global feature embedding; Hierarchical feature fusion processing is performed on the local feature embedding, the mid-level feature embedding, and the global feature embedding to obtain a unified representation vector; Scene matching processing is performed based on the unified representation vector to obtain the matching result.

[0139] In one embodiment, a computer-readable storage medium is provided, which may be non-volatile or volatile, and a computer program is stored thereon, which, when executed by a processor, performs the following steps: Multi-level visual coding processing is performed on the input image to obtain local visual tags, intermediate visual tags, and global visual tags; Semantic segmentation processing is performed on the input image to obtain a semantic segmentation map. Topological parsing processing is then performed on the semantic segmentation map to obtain a local topological map, a mid-level topological map, and a global topological map. Based on the local topology graph, the intermediate topology graph, and the global topology graph, a local adjacency matrix, an intermediate adjacency matrix, and a global adjacency matrix are generated, respectively. Based on the local adjacency matrix, the intermediate adjacency matrix, and the global adjacency matrix, topological constraint cross-attention processing is performed on the local visual labels, intermediate visual labels, and global visual labels respectively to obtain local feature embedding, intermediate feature embedding, and global feature embedding; Hierarchical feature fusion processing is performed on the local feature embedding, the mid-level feature embedding, and the global feature embedding to obtain a unified representation vector; Scene matching processing is performed based on the unified representation vector to obtain the matching result.

[0140] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0141] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0142] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0143] It should be noted that any software tools or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0144] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A hierarchical scene matching method based on topological constraints, characterized in that, Includes the following steps: Multi-level visual coding processing is performed on the input image to obtain local visual tags, intermediate visual tags, and global visual tags; Semantic segmentation processing is performed on the input image to obtain a semantic segmentation map. Topological parsing processing is then performed on the semantic segmentation map to obtain a local topological map, a mid-level topological map, and a global topological map. Based on the local topology graph, the intermediate topology graph, and the global topology graph, a local adjacency matrix, an intermediate adjacency matrix, and a global adjacency matrix are generated, respectively. Based on the local adjacency matrix, the intermediate adjacency matrix, and the global adjacency matrix, topological constraint cross-attention processing is performed on the local visual labels, intermediate visual labels, and global visual labels respectively to obtain local feature embedding, intermediate feature embedding, and global feature embedding; Hierarchical feature fusion processing is performed on the local feature embedding, the mid-level feature embedding, and the global feature embedding to obtain a unified representation vector; Scene matching processing is performed based on the unified representation vector to obtain the matching result.

2. The hierarchical scene matching method based on topological constraints as described in claim 1, characterized in that, Multi-level visual coding is performed on the input image to obtain local visual tags, intermediate visual tags, and global visual tags, including: The input image is fed into a visual coding network to perform a staged feature extraction operation, resulting in a multi-scale pyramid feature map. Multi-scale feature extraction is performed on the multi-scale pyramid feature map according to different preset receptive field scales to generate high-resolution local feature maps, mid-level feature maps and low-resolution global feature maps respectively. Spatial dimension mapping operations are performed on the high-resolution local feature map, the mid-level feature map, and the low-resolution global feature map respectively to obtain local one-dimensional feature sequences, mid-level one-dimensional feature sequences, and global one-dimensional feature sequences. Visual tag generation processing is performed on the local one-dimensional feature sequence, the intermediate one-dimensional feature sequence, and the global one-dimensional feature sequence respectively to obtain local visual tags, intermediate visual tags, and global visual tags.

3. The hierarchical scene matching method based on topological constraints as described in claim 1, characterized in that, Semantic segmentation is performed on the input image to obtain a semantic segmentation map. Topological parsing is then performed on the semantic segmentation map to obtain a local topological map, a mid-level topological map, and a global topological map, including: The input image is subjected to pixel-level semantic category segmentation processing to obtain a semantic segmentation map; From the semantic segmentation graph, object-level entity nodes, room-level partition nodes, and scene-level structural anchor points are parsed out. The relative spatial arrangement of the object-level entity nodes in a two-dimensional plane is obtained, and the spatial relationship between the object-level entity nodes is established based on the relative spatial arrangement to obtain a local topology graph. Extract the regional connectivity status between the room-level partition nodes, and establish regional spatial connectivity relationships between the room-level partition nodes based on the regional connectivity status to obtain the mid-level topology graph; Extract the macroscopic topological connection state between the scene-level structural anchor points, and establish scene spatial connectivity between the scene-level structural anchor points based on the macroscopic topological connection state to obtain a global topology graph.

4. The hierarchical scene matching method based on topological constraints as described in claim 1, characterized in that, Based on the local topology graph, the intermediate topology graph, and the global topology graph, a local adjacency matrix, an intermediate adjacency matrix, and a global adjacency matrix are generated, respectively, including: Extract the local topology nodes contained in the local topology graph, the middle-level topology nodes contained in the middle-level topology graph, and the global topology nodes contained in the global topology graph, and obtain the node center coordinates of the local topology nodes, the middle-level topology nodes, and the global topology nodes in the two-dimensional plane respectively; Based on the node center coordinates, the local relationship status between the local topology nodes, the intermediate relationship status between the intermediate topology nodes, and the global relationship status between the global topology nodes are determined respectively. Based on the local relation state, the mid-level relation state, the global relation state, and the preset Gaussian kernel decay distribution, the local spatial association weights between local topology nodes, the mid-level spatial association weights between mid-level topology nodes, and the global spatial association weights between global topology nodes are determined respectively. The local topology graph is transformed into a local adjacency matrix based on the local spatial association weight, the mid-level topology graph is transformed into a mid-level adjacency matrix based on the mid-level spatial association weight, and the global topology graph is transformed into a global adjacency matrix based on the global spatial association weight.

5. The hierarchical scene matching method based on topological constraints as described in claim 1, characterized in that, Based on the local adjacency matrix, the mid-level adjacency matrix, and the global adjacency matrix, respectively, topological constraint cross-attention processing is performed on the local visual labels, mid-level visual labels, and global visual labels to obtain local feature embeddings, mid-level feature embeddings, and global feature embeddings, including: A linear mapping process is performed on the local visual markers to generate local query vectors, local key vectors, and local value vectors. A linear mapping process is also performed on the mid-level visual markers to generate mid-level query vectors, mid-level key vectors, and mid-level value vectors. Finally, a linear mapping process is performed on the global visual markers to generate global query vectors, global key vectors, and global value vectors. Matrix multiplication is performed on the local query vector and the transposed local key vector to obtain the local initial attention distribution. Matrix multiplication is also performed on the mid-level query vector and the transposed mid-level key vector to obtain the mid-level initial attention distribution. Finally, matrix multiplication is performed on the global query vector and the transposed global key vector to obtain the global initial attention distribution. Based on the local adjacency matrix, topological constraint modulation processing is performed on the local initial attention distribution, and normalization mapping processing is performed to obtain local topological modulation weights. Based on the mid-layer adjacency matrix, topological constraint modulation processing is performed on the mid-layer initial attention distribution, and normalization mapping processing is performed to obtain mid-layer topological modulation weights. Based on the global adjacency matrix, topological constraint modulation processing is performed on the global initial attention distribution, and normalization mapping processing is performed to obtain global topological modulation weights. The local topology modulation weights and the local value vector are subjected to weighted interaction processing and linear mapping processing to obtain local feature embedding. The mid-layer topology modulation weights and the mid-layer value vector are subjected to weighted interaction processing and linear mapping processing to obtain mid-layer feature embedding. The global topology modulation weights and the global value vector are subjected to weighted interaction processing and linear mapping processing to obtain global feature embedding.

6. The hierarchical scene matching method based on topological constraints as described in claim 1, characterized in that, Hierarchical feature fusion processing is performed on the local feature embedding, the mid-level feature embedding, and the global feature embedding to obtain a unified representation vector, including: Obtain the local fusion weight matrix, the mid-level fusion weight matrix, and the global fusion weight matrix corresponding to the local feature embedding, the mid-level feature embedding, and the global feature embedding, respectively; The local feature embedding is multiplied by the local fusion weight matrix to obtain the local weighted embedding; the mid-level feature embedding is multiplied by the mid-level fusion weight matrix to obtain the mid-level weighted embedding; and the global feature embedding is multiplied by the global fusion weight matrix to obtain the global weighted embedding. Based on the local weighted embedding, the middle-layer weighted embedding, and the global weighted embedding, a hierarchical fusion coefficient generation process is performed to obtain the local hierarchical fusion coefficient, the middle-layer hierarchical fusion coefficient, and the global hierarchical fusion coefficient; Based on the local hierarchical fusion coefficient, perform local hierarchical fusion processing on the local weighted embedding to obtain a local fusion result; based on the mid-level hierarchical fusion coefficient, perform mid-level hierarchical fusion processing on the mid-level weighted embedding to obtain a mid-level fusion result; based on the global hierarchical fusion coefficient, perform global hierarchical fusion processing on the global weighted embedding to obtain a global fusion result. A weighted summation process is performed on the local fusion result, the mid-level fusion result, and the global fusion result to obtain the fusion feature; A linear projection process is performed on the fused features to obtain a unified representation vector.

7. The hierarchical scene matching method based on topological constraints as described in claim 1, characterized in that, Based on the unified representation vector, scene matching processing is performed to obtain matching results, including: Obtain a pre-constructed reference feature database, which includes multiple reference representation vectors and reference scene labels corresponding to the multiple reference representation vectors respectively; A similarity comparison process is performed between the unified representation vector and the multiple reference representation vectors to obtain multiple similarity scores; Based on the multiple similarity scores, the multiple reference representation vectors are sorted and filtered to obtain the target reference representation vector; Extract the target reference scene label corresponding to the target reference representation vector from the reference feature database, and use the target reference scene label as the matching result.

8. A hierarchical scene matching device based on topological constraints, characterized in that, The hierarchical scene matching device based on topology constraints includes: The visual encoding module is used to perform multi-level visual encoding processing on the input image to obtain local visual tags, intermediate visual tags, and global visual tags. The topology parsing module is used to perform semantic segmentation processing on the input image to obtain a semantic segmentation map, and to perform topology parsing processing on the semantic segmentation map to obtain a local topology map, a mid-level topology map, and a global topology map. An adjacency matrix generation module is used to generate a local adjacency matrix, a mid-level adjacency matrix, and a global adjacency matrix based on the local topology graph, the mid-level topology graph, and the global topology graph, respectively. The topological constraint cross-attention module is used to perform topological constraint cross-attention processing on the local visual labels, mid-level visual labels, and global visual labels based on the local adjacency matrix, mid-level adjacency matrix, and global adjacency matrix, respectively, to obtain local feature embedding, mid-level feature embedding, and global feature embedding. The hierarchical feature fusion module is used to perform hierarchical feature fusion processing on the local feature embedding, the intermediate feature embedding, and the global feature embedding to obtain a unified representation vector; The scene matching module is used to perform scene matching processing based on the unified representation vector to obtain the matching result.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and a topology-constrained hierarchical scene matching program stored in the memory and executable on the processor. When executed by the processor, the topology-constrained hierarchical scene matching program implements the steps of the topology-constrained hierarchical scene matching method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a topology-constrained hierarchical scene matching program, which, when executed by a processor, implements the steps of the topology-constrained hierarchical scene matching method as described in any one of claims 1-7.

Images