Topological guidance scenic spot scene recognition method facing end side equipment

Abstract

The invention discloses a topological guidance scenic spot scene recognition method facing end-side equipment, and belongs to the technical field of computer vision and edge computing. The method comprises the following steps: acquiring a scenic spot scene image and performing image block division; constructing lightweight topological representation based on the image blocks; performing sparse structural feature extraction on the lightweight topological representation to obtain scene-level topological structural features; generating a topological guide mask based on the topological structure features and carrying out routing selection; in the reasoning stage, topological branch calculation is triggered in a self-adaptive mode according to the prediction confidence coefficient and topological structure feature changes, so that the average calculation overhead of end-side equipment is reduced. According to the method, through the prototype mask, sparse structure coding and the self-adaptive reasoning strategy, the calculation complexity and the reasoning time delay are effectively reduced while the recognition accuracy is guaranteed, and the method is suitable for deployment of end-side equipment such as a mobile terminal, a vehicle-mounted terminal and an edge calculation node and can be linked with a scenic spot service system in real time.

Description

Technical Field

[0001] This invention relates to the field of computer vision and edge computing technology, and in particular to a topology-guided scenic scene recognition method for edge devices. Background Technology

[0002] With the advancement of smart scenic area construction and digital management of cultural tourism, scenic areas need to quickly understand and judge on-site images or videos in business operations such as visitor services, security patrols, passenger flow scheduling, emergency command, and guided tour recommendations. As an important application of computer vision, scenic area scene recognition typically uses images collected by surveillance cameras, mobile terminals, or vehicle-mounted / drone equipment as input to automatically identify typical scenic area scene categories such as "entrance / ticket gates, boardwalks, viewing platforms, cruise ship docks, parking lots, catering areas, visitor centers, commercial streets, and mountain trails," providing basic information for subsequent business strategies (such as crowd density warnings, congestion diversion, equipment linkage, and guided tour recommendations).

[0003] Existing methods for scenic area scene recognition mostly employ deep models such as convolutional neural networks or visual Transformers, learning from the appearance, texture, color, and semantic region features of images to achieve classification. However, scenic areas often exhibit characteristics of similar appearance between categories and significant variations within categories: for example, walkways, boardwalks, plazas, or viewing platforms at different attractions may appear highly similar, but their spatial layout and facility organization may differ. Furthermore, the scenic environment is affected by changes in season, lighting, occlusion, crowd density, and shooting angle; relying solely on appearance features can easily lead to misjudgments, thus affecting the real-time performance and reliability of scenic area management and services.

[0004] To improve recognition accuracy, some studies have attempted to introduce structured information (such as region adjacency relationships, target layout, and relationship graphs) to characterize the spatial organization and topological relationships between elements in a scene, thereby helping to distinguish scene categories that are similar in appearance but different in structure. However, in practical applications, the above-mentioned structured methods usually face the following problems:

[0005] (1) High cost of constructing structured information and difficult deployment on the edge: Many methods rely on preprocessing steps such as superpixel segmentation, target detection / segmentation, and graph construction, which have problems such as high computational overhead, complex implementation links, and strong dependence on the operating environment, making it difficult to run stably on edge devices commonly found in scenic areas (such as mobile terminals, edge boxes, low-power cameras, or vehicle terminals).

[0006] (2) Uncontrollable computational overhead of structural branches: Some methods use structural learning modules such as graph attention networks to perform multi-layer computation on graph structures, which usually requires sparse adjacency operations and attention normalization, resulting in high memory access and computational overhead; when the graph size changes with the complexity of the scene, the inference latency fluctuates significantly, which is not conducive to real-time applications on the edge.

[0007] (3) Lack of edge-side constraint design in fusion mechanism and inference process: Existing fusion methods mostly perform fixed structural branch calculation and fusion with visual features in each inference, lacking edge-side lightweight and adaptive calculation mechanisms (such as conditional execution, cache reuse, sparsity constraints, quantization friendly design, etc.), resulting in overall latency, power consumption and memory usage that are difficult to meet edge-side deployment requirements.

[0008] (4) Difficulty in balancing accuracy and computational efficiency: In order to improve recognition performance, the common practice is to increase the model size or introduce more complex structural modules, but this will further increase the pressure on edge deployment; conversely, simple pruning or replacement of the backbone network may lead to insufficient utilization of structural information and decreased recognition performance, making it difficult to achieve a comprehensive balance between accuracy and edge efficiency.

[0009] Therefore, there is an urgent need for a scenic scene recognition technology for edge devices that can effectively integrate topological information into the visual recognition process while ensuring recognition performance, with lower computational overhead, more stable inference latency, and easier engineering deployment, thereby meeting the real-time application needs of smart scenic areas. Summary of the Invention

[0010] This invention aims to provide a topology-guided scenic scene recognition method for edge devices. In the application of scenic scene image acquisition and recognition, it introduces topological structure information to help distinguish scenic scene categories that are similar in appearance but have significant differences in layout. While ensuring recognition effect, it reduces computational overhead and inference latency, thereby meeting the real-time deployment requirements of edge devices under conditions of limited computing power, power consumption and storage.

[0011] To achieve the above objectives, the present invention provides a topology-guided scenic area scene recognition method for end-side devices, comprising:

[0012] Step 1: Acquire scenic scene images and divide the images into blocks;

[0013] Step 2: Construct lightweight topological representations of each image patch based on image patch features;

[0014] Step 3: Extract sparse structural features from the lightweight topological representation to obtain scene-level topological structural features;

[0015] Step 4: Generate a topology mask prototype for guiding attention based on scene-level topology features, and determine the final topology guiding mask based on the routing selection mechanism.

[0016] Step 5: Fuse the final topology-guided mask with image patch features to obtain a topology-guided visual representation and output the scenic scene category recognition result;

[0017] Steps 2 to 5 are implemented using a neural network model. The loss function used in the training phase of the neural network model includes scene classification loss and topology guidance related constraint terms. These topology guidance related constraint terms are used to regulate the distribution characteristics and computational cost of the topology guidance mask. In the inference phase of the neural network model, adaptive topology branch calculation and mask update are triggered based on confidence or computational budget, thereby achieving efficient inference for edge devices.

[0018] Furthermore, in step 1, the methods for acquiring scenic scene images include: acquiring them through a fixed image acquisition device deployed at a designated location in the scenic area, or acquiring them through a mobile device equipped with an image acquisition device.

[0019] Furthermore, in step 1, the acquired scenic scene images also include accompanying metadata, including: shooting timestamp, device number, and location information.

[0020] Furthermore, the image patch features are set as follows:

[0021]

[0022] in, For the first Image patch features of an image patch Indicates the first The set of pixels corresponding to each image patch Indicates the first The number of pixels in an image patch Represents pixels The basic feature vector includes color information or brightness information, or a combination of color and brightness information.

[0023] Furthermore, in step 2, the lightweight topological representation of the image patch is specifically as follows:

[0024] Constructing a topology graph using image patches as nodes ,in, For a set of nodes, For the set of edges, if two image patches are spatially adjacent, or the spatial distance between their center coordinates is less than a preset threshold, then an undirected edge is established between the corresponding nodes.

[0025] And calculate the edge weights of each undirected edge based on the appearance difference measure between the image patch features corresponding to the two nodes:

[0026]

[0027]

[0028] in, Representing a topology graph Nodes in With nodes The edge weight of the undirected edge between them. and They are nodes With nodes The corresponding image patch features, For nodes With nodes A measure of the appearance difference between corresponding image patch features. To prevent smoothing constants with denominators of zero.

[0029] Furthermore, in step 3, the scene-level topological structure features are specifically as follows:

[0030] For topology graph The first in Nodes Based on image patch features The initial structural features are obtained by linear mapping, denoted as... Among them, node number ,in The number of image blocks in the scenic area scene image;

[0031] By extracting multi-layer sparse structural features, the structural features of nodes are updated layer by layer to obtain their structural features in the next layer:

[0032]

[0033] in, This is the layer identifier, with an initial value of 0. for The structural characteristics of the layers For nodes The set of adjacent nodes, For nodes The identifier of the adjacent node. For the first The structural characteristics of the layers For the first The learnable weight matrix of the layer, It is a non-linear activation function; The normalized edge weights of undirected edges are used to control the contribution of neighboring nodes to the update of the current node.

[0034] Define L as the number of layers for feature extraction in a multi-layer sparse structure. After extracting the layer structure features, the nodes are obtained. High-rise structural features Then, an aggregation operation is performed to obtain scene-level topological structure features. .

[0035] Furthermore, the normalization process for the edge weights of undirected edges is as follows:

[0036]

[0037] in, Represents a node Its adjacent nodes The edge weights between them.

[0038] Furthermore, in step 4, the final topology guiding mask is specifically as follows:

[0039] Let the set of mask prototypes be represented as ,in, For the number of mask prototypes, and The value is 10%-20% of the number of scenic scene categories; the k-th mask prototype ,and Length and number of image patches Consistent, among which, , For the first Learnable mapping parameters corresponding to each mask prototype This is a non-linear activation function used to constrain the mask weights within a preset range; The Sigmoid function can be used;

[0040] Based on scene-level topology features Calculate the routing weight vector of the mask prototype ,in , For the first Routing weights for each mask prototype; These are learnable mapping parameters; It is a normalized exponential function;

[0041] Based on scene-level topology features Calculate the activation weight vector (i.e., the routing weight vector) of the mask prototype. ,in , For the first Routing weights for each mask prototype; These are learnable mapping parameters;

[0042] based on For the mask prototype Weighted combinations are performed to obtain the final topology-guided mask. Among them, mask Includes P mask values, each mask value It corresponds to one image block.

[0043] Furthermore, step 5 specifically includes:

[0044] The final topology-guided mask consists of P mask values, each mask value corresponding to an image patch;

[0045] For each image patch, the image patch features are weighted block by block with the corresponding mask value to obtain the topology-guided image patch features for each image patch, i.e., the topology-guided visual representation.

[0046] The topology-guided visual representation is then fed into a classifier for scenic scene category recognition, and the scenic scene category recognition result is output.

[0047] Furthermore, the loss function used during the training phase is:

[0048]

[0049] in, For scene classification loss, For topology-guided mask constraint loss, The average activation level constraint loss for the topology-guided mask, and Constraint loss and Weighting coefficients;

[0050] Constraint loss Set to:

[0051]

[0052]

[0053] Constraint loss Set to:

[0054]

[0055] in, For the final topology guiding mask Entropy constraints, The target entropy value is preset. for The mask value corresponding to the p-th image patch is given, where P is the number of image patches. To prevent numerical instability, a smoothing constant is used.

[0056] Furthermore, during the inference phase of the neural network model, adaptive topology branching and mask updates are triggered based on confidence levels, including:

[0057] When the confidence level in the output scenic scene category recognition result is greater than or equal to the preset threshold If the current scenic scene image is in the correct position, the final topology-guided mask corresponding to the current scenic scene image is used to directly identify the scenic scene category for the next / next frame of scenic scene image; otherwise, adaptive topology branch calculation and mask update are triggered, that is, steps 1 to 5 are executed for the next / next frame of scenic scene image.

[0058] Furthermore, during the inference phase of the neural network model, adaptive topology branching and mask updates are triggered based on the computational budget, including:

[0059] Scene-level topological structure features based on the scenic scene images of the current frame and the previous frame , Calculate the magnitude of changes in scene-level topology When the range of change Less than the preset threshold If the previous frame's final topology guiding mask is used, the scenic scene category is directly identified for the current frame's scenic scene image; otherwise, the final topology guiding mask corresponding to the current frame is calculated, and then the scenic scene category is identified based on it.

[0060] The technical solution provided by this invention brings at least the following beneficial effects:

[0061] This invention introduces a lightweight topology guidance mechanism under the premise of edge deployment: the scenic area image is divided into image blocks, a lightweight topology representation is constructed based on the image blocks, and sparse structural feature extraction is used to obtain scene-level topology features; furthermore, a final topology mask is generated through "a limited number of topology guidance mask prototypes + route selection", and it is fused with image block features in a weighted manner to obtain a visual representation of topology guidance, so as to achieve more reliable differentiation of scenes with similar appearances but different layouts; in the inference stage, the topology branch calculation can also be adaptively triggered according to the prediction confidence or computational budget, and the mask cache reuse can be combined to reduce redundant calculations, thereby significantly reducing the average computational overhead and inference latency fluctuations on the edge while ensuring recognition accuracy.

[0062] Compared with existing technologies, this invention designs the topology information acquisition and encoding process as a lightweight process that is user-friendly to the end side, and replaces the fixed category-specific mask generation method with a prototyping mask and routing selection mechanism. At the same time, it introduces optional adaptive computing strategies to reduce unnecessary structural branch calculations, thereby achieving lower computational complexity and more stable inference latency under end-side deployment conditions, and improving the distinguishability and recognition reliability of scenic scenes with similar appearances. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a flowchart of a topology-guided scenic scene recognition method for end-side devices provided in an embodiment of the present invention.

[0065] Figure 2 This is a deployment architecture diagram for a topology-guided scenic scene recognition method for end-side devices provided in an embodiment of the present invention. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be described in detail and completely below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Generally, the components of the embodiments of the present invention described and shown in the accompanying drawings can be arranged and designed using different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present invention.

[0067] This invention provides a topology-guided scenic area scene recognition method for edge devices, applicable to scenarios such as scenic area guidance, visitor flow organization, security inspection and emergency management. It addresses the problem of easy confusion caused by visually similar but spatially different scenes in scenic areas by introducing a lightweight topology guidance mechanism that can be deployed on the edge.

[0068] In one embodiment, see Figure 1The present invention provides a topology-guided scenic scene recognition method for edge devices, comprising: first, acquiring scenic scene images and dividing them into image blocks; then, constructing lightweight topological representations based on image blocks, i.e., constructing lightweight topological representations of each image block based on image block features; next, extracting sparse structural features from the lightweight topological representations to obtain scene-level topological structure features; then, generating a topological guidance mask based on the topological structure features and performing routing selection, i.e., generating a limited number of topological guidance mask prototypes (topological mask prototypes for guiding attention) based on the topological structure features, and determining the final topological guidance mask through a routing selection mechanism; finally, performing adaptive inference based on confidence and outputting the recognition result, i.e., weighted fusion of the topological guidance mask and image block features to obtain a visual representation of topological guidance and complete the scenic scene category recognition; in the inference stage, adaptively triggering topological branch calculation based on the predicted confidence and changes in topological structure features to reduce the average computational overhead of the edge device. This invention effectively reduces computational complexity and inference latency while ensuring recognition accuracy through prototyping masks, sparse structure coding, and adaptive inference strategies. It is suitable for deployment on edge devices such as mobile terminals, vehicle terminals, and edge computing nodes, and can achieve real-time linkage with scenic area business systems.

[0069] In one embodiment, the system implementing the method proposed in this invention can be deployed as follows: a system including an image acquisition module, a computing and communication module, a topology representation construction module, a topology-guided mask generation module, a topology-guided scene recognition module, and a business linkage module. This system can be deployed on scenic area edge devices, edge computing nodes, or cloud servers for real-time or near-real-time scene recognition of scenic area scene images and for linkage with the scenic area's external business systems, such as... Figure 2 As shown. The image acquisition module belongs to the scenic area front-end equipment domain. This module can be a fixed camera, a mobile inspection terminal camera, or a drone camera, used to acquire images or video frames of the scenic area scene. The computing and communication module and data storage and model management belong to the edge-cloud computing communication domain. This module can be an edge computing box, a mobile terminal system-on-a-chip (SoC), or a cloud computing server, used to perform inference calculations for the scene recognition model and communicate the recognition results. The business linkage module and external business systems belong to the scenic area business linkage and application domain. This module interfaces with external business systems such as the scenic area guide system, electronic map system, visitor flow monitoring system, or emergency command system, triggering corresponding business processing or linkage actions based on the recognition results. The topology representation construction module, topology-guided mask generation module, and topology-guided scene recognition module belong to the topology scene recognition algorithm domain and can be implemented through program configuration. Specifically, each module is as follows:

[0070] (1) The image acquisition module is used to acquire images or video frames of scenic spots. It can be composed of fixed cameras deployed at entrances and exits, visitor centers, exhibition halls, key nodes of walkways, etc., or mobile inspection terminals carried by staff, drone cameras, etc. The acquired data can be accompanied by metadata such as shooting timestamps, device numbers, and optional location information to facilitate subsequent identification result traceability and business linkage. To reduce the computing and communication burden on the end side, the image acquisition module can perform lightweight preprocessing operations such as frame extraction, resolution adaptation, brightness correction, or noise reduction locally.

[0071] (2) The computing and communication module is used to carry out the inference execution and result distribution of the scene recognition model. It can be deployed as a scenic area edge computing box, a mobile terminal system-on-a-chip, or a cloud computing server. In a recommended implementation, scene recognition tasks with low latency requirements are preferentially executed on the edge or end-user side, while model updates, centralized training, or cross-regional statistical analysis can be completed in the cloud. The computing and communication module communicates with the image acquisition module and the service linkage module through wired or wireless networks. It supports, but is not limited to, data transmission methods such as RTSP and HTTP(S), and can adopt encrypted transmission, identity authentication, and cache retransmission mechanisms to ensure the reliability and security of data transmission.

[0072] (3) The topology representation construction module, the topology-guided mask generation module, and the topology-guided scene recognition module can run as algorithm inference service processes within the computation and communication module. After receiving an input image or video frame, the inference service sequentially executes lightweight topology representation construction, topology-guided mask generation, and scene recognition calculation, outputting the corresponding scenic scene category and its confidence information. In edge-deployment scenarios, the above modules can be run in a modular or containerized manner to facilitate model updates and system maintenance.

[0073] (4) The business linkage module is used to connect with the scenic area's business system and trigger corresponding business actions. It can interface with the scenic area's tour guide system, electronic map or route planning system, passenger flow monitoring and dispatching system, security patrol and emergency command system, etc. After receiving the scene recognition results, the business linkage module can generate linkage events according to preset rules, such as triggering voice or graphic explanation push, issuing route guidance suggestions, generating congestion warning information or issuing patrol task instructions. It also supports filtering and controlling linkage behaviors based on scene category, recognition confidence, time window and regional strategy.

[0074] In one embodiment, constructing a lightweight topological representation based on image patches specifically involves:

[0075] To adapt to the computational resource constraints of edge devices, this invention no longer relies on complex region segmentation or superpixel generation processes. Instead, it constructs a lightweight topological representation based on image patches, thereby reducing the computational overhead of topological modeling while maintaining scene structural information.

[0076] First, the input scenic scene image is divided into image blocks. Specifically, the input image is divided into several non-overlapping or partially overlapping image blocks of a preset size, with each image block corresponding to a local region in the image. Let the total number of blocks after division be... The image patch, the first The feature representation of each image patch is denoted as . Image patch features can be obtained from the statistical information of pixels within the image patch, and can be represented as:

[0077]

[0078] in, Indicates the first The set of pixels corresponding to each image patch Indicates the first The number of pixels in an image patch Represents pixels The basic feature vector includes color information, brightness information, or a combination of color and brightness information. By default, the mean and standard deviation of the color and the mean and standard deviation of the brightness information are used. Considering actual end-user requirements, the feature dimensions can be reduced as appropriate to further control the computational load.

[0079] Based on this, a topology graph is constructed using image patches as nodes. ,in, For a set of nodes, The set of edges; the set of nodes Each image patch corresponds one-to-one. They represent There are n nodes, each representing an image patch. If two image patches are spatially adjacent, or the spatial distance between their center coordinates is less than a preset threshold, then an undirected edge is established between the corresponding nodes. Let the nth node be an image patch. The image patch and the first The center coordinates of the image blocks are respectively and Its spatial distance can be expressed as:

[0080]

[0081] When spatial distance When the number of nodes is less than the preset spatial adjacency threshold, and Edges (undirected edges) are established between them to form topological relationships that reflect the local spatial structure of the image.

[0082] To enhance the ability of topological representation to characterize local appearance consistency, edges are weighted based on the appearance similarity between adjacent image patches. Let the image patch features... and The difference in appearance between them is measured as follows:

[0083]

[0084] The corresponding edge weight is defined as the appearance difference metric. Decreasing function:

[0085]

[0086] in, To prevent the smoothing constant from having a denominator of zero, a weighted topological graph that simultaneously reflects the spatial adjacency relationship and appearance similarity of image patches can be obtained through the above method.

[0087] The topology graph constructed in this invention features a fixed number of nodes, regular adjacency relationships, and a limited number of edges, facilitating subsequent processing using sparse storage and efficient computation methods, thus making it suitable for the real-time inference needs of edge devices. This lightweight topology representation provides structural information support for subsequent topology-guided feature extraction and scene recognition without significantly increasing computational complexity.

[0088] In one embodiment, the sparse extraction of the lightweight topological representation specifically involves:

[0089] In obtaining a lightweight topological graph based on image patches (i.e., a topological graph) After that, in order to further extract topological information that can characterize the overall structural features of the scenic area, i.e., scene-level topological features, this invention adopts a sparse topological feature extraction method to encode the structure of the constructed topological graph.

[0090] In the constructed topology graph In the middle, the set of nodes is edge set Reflecting the adjacency relationship between image patches, edge weights are used to describe the appearance similarity between adjacent image patches. Let the first edge weight be... Layer nodes The structural features are represented as Its initial structural features Image patch features Obtained through linear mapping.

[0091] The initialization process of the structural features of a node can be represented as follows:

[0092]

[0093] in, It is a learnable linear mapping matrix.

[0094] Subsequently, the structural features of nodes are updated layer by layer through multi-layer sparse structure feature extraction. In each layer, the structural features of a node are updated by weighted aggregation of the features of its neighboring nodes. Specifically, the first... The structural feature update method for layer nodes can be represented as follows:

[0095]

[0096] in, For the first The structural characteristics of the layers Represents a node The set of adjacent nodes, For the first The learnable weight matrix of the layer, It is a non-linear activation function; The normalized edge weights are used to control the contribution of neighboring nodes to the current node's update.

[0097] In this embodiment, to improve numerical stability and limit computational complexity, the edge weights need to be normalized:

[0098]

[0099] in, Represents a node Its adjacent nodes Edge weights between them.

[0100] In this way, the node's structural feature update process only involves sparse weighted summation operations within the local neighborhood, avoiding complex global attention calculations, thereby reducing computational and storage overhead and making it suitable for efficient execution on edge devices. After extracting the (preset) layer structure features, the high-level structure features of each node can be obtained. To obtain a global representation describing the overall scene topology, the high-level structural features of all nodes are aggregated using global average pooling.

[0101]

[0102] in, This refers to the scene-level topological structure features.

[0103] The sparse topology feature extraction method described above can obtain stable scene-level topological representations (i.e., global topological feature representations of scenic scenes) with low computational complexity while maintaining the spatial organization of image patches. This provides effective structural information support for subsequent topology-guided mask generation and scene recognition. Correspondingly, in the implementation system, the topology representation construction module obtains scene-level topological features based on this sparse topology feature extraction method. .

[0104] In one embodiment, generating a topology guidance mask based on the obtained topology features and performing route selection specifically includes:

[0105] After obtaining the global topological structure feature representation of the scenic scene, in order to further guide the subsequent visual features to focus on the scene structure information, this invention adopts a prototyping generation and routing selection mechanism for topological guidance masks, so as to reduce the computational overhead of the edge inference stage while ensuring the structure guidance capability.

[0106] Unlike generating masks separately for each category, this invention pre-constructs a set of mask prototypes consisting of a finite number of mask prototypes to represent common scene structure interest patterns. Let the set of mask prototypes be represented as:

[0107]

[0108] in, For the number of mask prototypes, and Much smaller than the number of scene categories in the scenic area; each mask prototype ( ) length and number of image patches Consistency is used to represent the importance weight of each image patch. The mask prototype can be obtained by linearly mapping topological features and then constraining them with a nonlinear function.

[0109]

[0110] in, For the first Learnable mapping parameters corresponding to each mask prototype, nonlinear activation function The Sigmoid function can be used to constrain the mask weights within a preset range.

[0111] To adaptively select or combine mask prototypes based on specific input scenarios, this invention further introduces a routing mechanism based on topology features. This mechanism is based on the current scene-level topology features. The activation weights of each mask prototype are calculated to determine the final topological mask used to guide visual features.

[0112]

[0113] in, This represents the routing weight of each mask prototype. These are learnable mapping parameters. The mask prototypes are weighted and combined based on the obtained routing weights to obtain the final topology guidance mask:

[0114]

[0115] By employing the prototyping and routing methods described above, this invention only requires combining a limited number of mask prototypes during the inference phase, avoiding the generation of separate masks for each category. This effectively reduces computational complexity and storage overhead, making it particularly suitable for edge device deployment. To further improve edge inference efficiency, the topology-guided mask can be cached and reused when consecutive input images exhibit temporal or spatial consistency; a mask update operation is only triggered when changes in topological features exceed a preset threshold. This approach reduces redundant computation while ensuring the effectiveness of structure guidance, thereby improving overall inference efficiency.

[0116] Through the prototyping generation and routing mechanism of the topology-guided mask described above, this invention can obtain attention guidance information related to scene structure with low computational cost, providing effective support for subsequent topology-guided scene recognition. Corresponding to the implementation system, the topology-guided mask generation module generates the final topology-guided mask based on this prototyping and routing mechanism. .

[0117] In one embodiment, fusing a topology-guided mask with image patch features to complete scene recognition specifically includes: Topology Attention Guided Fusion (TAG) and Transformer recognition. After obtaining the topology-guided mask, in order to effectively integrate the topological structure information of the scene into the visual feature representation, this invention adopts an image patch feature fusion method based on the topology-guided mask, which performs weighted processing on the image patch features to form a topology-guided visual representation that takes into account both appearance and structural information.

[0118] First, the input scenic scene image is divided into image patches, and feature embedding is performed on each image patch. Let the input image be divided into... For each image patch, after linear mapping or embedding layer, an image patch embedding matrix can be obtained:

[0119]

[0120] in, The image patch embedding matrix represents the feature dimension of the image patch embedding. Each element in Indicates the first Embedding vectors of image patches.

[0121] Let the obtained final topology guiding mask be Each mask element corresponds to an image patch, representing the importance of that image patch within the current scene structure. By embedding the topology-guided mask with the image patch and then weighting each patch, the topology-guided image patch feature representation is obtained, which can be expressed as:

[0122]

[0123] Or, equivalently, in matrix form:

[0124]

[0125] in, Indicates the first Topologically guided image patch features of an image patch Indicates the first The final topological guide mask vector for each image patch, This represents the scene-level topology-guided image patch feature matrix, i.e., the image patch features after topology-guided fusion. for Element; For the final topology-guided mask The matrix obtained by expanding along the feature dimension This indicates element-wise multiplication.

[0126] Through the weighted fusion method described above, topology-guided masking can assign higher weights to image patches with structural discriminative significance while preserving the original feature structure of image patches, thereby suppressing regional features that are weakly related to the scene structure. Image patch features after topology-guided fusion. It can be used as input for subsequent scene recognition models for global feature modeling and category discrimination. Topology-guided image patch features are input into a feature encoding module based on a self-attention mechanism, which further extracts discriminative scene representations by modeling global dependencies between image patches.

[0127] By fusing the aforementioned topology-guided mask with image patch features, this invention can explicitly integrate scene topology information into visual feature representation without introducing complex attention calculations. This improves the ability to recognize scenic scenes that are visually similar but structurally distinct, while maintaining edge-side inference efficiency. Corresponding to the implementation system, the topology-guided scene recognition module first acquires image patch features. Then based on the features of this image patch Scene recognition is accomplished through a classifier.

[0128] In this invention, scene recognition based on image patch-based lightweight topological representation, generating a topological guidance mask and performing route selection, and fusing the topological guidance mask with image patch features is achieved through a neural network. In specific implementation, it also includes setting corresponding training objectives and loss functions. To ensure the accuracy of scenic area scene recognition while meeting the requirements of edge devices for computational efficiency and inference stability, this invention employs a multi-objective joint optimization approach during the model training phase, comprehensively constraining scene classification performance, the effectiveness of the topological guidance mask, and edge deployment friendliness.

[0129] The training objective consists of a scene classification loss and a topology guidance constraint term. The scene classification loss is used to improve the ability to distinguish the scenic scene categories, while the topology guidance constraint term is used to regulate the distribution characteristics and computational cost of the topology guidance mask.

[0130] Suppose the model's predicted output for the input sample is a class probability vector. The one-hot encoding corresponding to the real label is The scene classification loss can then be expressed as cross-entropy loss, which is defined as follows:

[0131]

[0132] in, This indicates the total number of scenic scene categories. The model represents the first Predicted probability of class Indicates the first The true label of the class is used to guide the model to learn a discriminative visual and topological joint feature representation by minimizing the cross-entropy loss.

[0133] To ensure that the topology-guided mask maintains good stability and controllability while guiding visual feature attention to the structural discrimination region, this invention improves the final topology-guided mask. A distribution constraint loss term is introduced. In one implementation, the distribution shape of the mask can be standardized by entropy constraints, defined as follows:

[0134]

[0135] in, Indicates the number of image patches. To prevent numerically unstable smoothing constants, constraints are imposed on the mask entropy value. This avoids the mask from becoming overly concentrated in a few image patches or tending towards a uniform distribution, thereby improving the stability of the topology-guided effect. The topology-guided mask constraint loss can be expressed as:

[0136]

[0137] in, The preset target entropy value is used to balance the mask's coverage and distinguishability.

[0138] To further improve the model's deployment friendliness on edge devices, constraints related to computational overhead can be introduced to guide the model to form a more sparse and stable topology-guided pattern during training. For example, the average activation level of the topology-guided mask can be constrained, which can be expressed as:

[0139]

[0140] By applying weighted constraints to this item, the effective computational load in the subsequent inference stage can be reduced without significantly affecting recognition performance.

[0141] In summary, the total training loss function of this invention can be expressed as:

[0142]

[0143] in, and These are weighting coefficients used to balance the relationship between scene classification performance, topology guidance effect, and edge efficiency constraints. During training, these weights can be adjusted according to the specific application scenario and the capabilities of the edge device.

[0144] Through the above-mentioned training objectives and loss function design, this invention can guide the model to learn a topology guidance mechanism that balances recognition performance and computational efficiency during the training phase, thereby providing a basic guarantee for stable and efficient scenic scene recognition on edge devices.

[0145] In one embodiment, the confidence-based adaptive inference and output of recognition results (i.e., the adaptive inference strategy for edge devices) specifically includes:

[0146] To further reduce the average computational overhead of edge devices during actual operation and improve the system's response stability in complex scenarios, this invention introduces an adaptive inference strategy for edge devices during the inference phase, dynamically adjusting the execution mode of topology guidance-related modules based on the discrimination difficulty and confidence level of the input scenario.

[0147] In one implementation, after encoding the basic visual features, the model first outputs a preliminary scene category prediction and its corresponding confidence score. Let the model's prediction probability distribution for the current input sample be... Then its maximum prediction confidence can be expressed as:

[0148]

[0149] in, This indicates the total number of scenic scene categories. This represents the prediction confidence index for the current sample.

[0150] Based on the aforementioned confidence index, this invention sets a preset threshold. This is used to distinguish between easily identifiable scenarios and structurally complex scenarios. When the following conditions are met... When the current scene can be reliably identified using only basic visual features, the system can skip or simplify the topology-guided computation process and directly output the scene recognition result; when the conditions are met... When this occurs, the complete topology-guided inference process is triggered to utilize topological structure information to assist in scene discrimination. The adaptive inference strategy can be formally represented as:

[0151]

[0152] The Fast mode represents the execution of only the basic visual encoding and classification process, while the Topology-guided mode represents the execution of the complete process, including topology representation construction, topology-guided mask generation, and fusion.

[0153] Furthermore, to adapt to scenic video streams or continuous image input scenarios, this invention introduces a caching and reuse mechanism for topology-guided masks during adaptive inference. When the topological features of the input images change little within adjacent time windows, the topology-guided mask generated at the previous time step can be reused, thereby avoiding redundant calculations. Let the topological feature vectors corresponding to two adjacent frames be... and Its range of change can be expressed as:

[0154]

[0155] When the magnitude of change Less than the preset threshold When the change amplitude is... If this happens, the mask generation and routing process will be re-executed.

[0156] By employing the aforementioned adaptive inference strategy based on confidence triggering and feature change detection, this invention can significantly reduce the average computational load of edge devices, reduce inference latency fluctuations, and improve system stability under complex environments and continuous operating conditions, while ensuring recognition accuracy.

[0157] In actual deployment, the above adaptive inference strategy can adjust parameters according to the performance level of the end device, the real-time requirements of the service, or the power status to achieve a dynamic balance between recognition performance and computing efficiency.

[0158] The topology-guided scenic area scene recognition method of the present invention for end-side devices can be integrated with the existing business system of the scenic area to build a business closed loop of "perception-recognition-decision-linkage" to support application scenarios such as scenic area tour services, visitor flow organization, security inspection and emergency response.

[0159] During system operation, after completing scene recognition and inference, the computing and communication module outputs standardized recognition result information to the business linkage module. The recognition result may include, but is not limited to: scene category identifier (scene_id), scene category name (scene_name), recognition confidence score, acquisition device identifier (device_id), timestamp, and optional location information (location) or intermediate result reference identifier (ref). Based on the above recognition results and preset business rules, the business linkage module triggers corresponding business processing flows.

[0160] In the scenic area guide application example, when the recognition result shows that the current scene belongs to a scenic area node that can be explained (such as exhibition hall, scenic spot entrance, visitor center, etc.) and the recognition confidence exceeds the preset threshold, the business linkage module can match the corresponding scenic spot item according to the location information of the collection device, push graphic or audio explanation content to the tourist's mobile terminal, and can generate subsequent tour route suggestions in combination with electronic map.

[0161] In the application embodiment of passenger flow organization and congestion early warning, when the identification result indicates that the scene belongs to areas prone to congestion, such as entrances, ticket gates, narrow passages, or viewing platforms, within a continuous time window, and the identification confidence level remains higher than the threshold, the business linkage module can generate a congestion risk event and report it to the passenger flow monitoring and dispatching system. The passenger flow dispatching system can further link with electronic displays, broadcasting systems, or staff terminals to issue guidance prompts or adjust traffic strategies to tourists.

[0162] In the application example of security inspection and emergency management, the business linkage module can match the scene recognition results with the area rules in the security platform. When an unopened area, dangerous area, or abnormal usage scenario is identified within a limited time period or area, the system can automatically generate an inspection or handling task and send it to the staff terminal. At the same time, it records the time of the event, the collection point, and the recognition confidence level to facilitate subsequent traceability and analysis.

[0163] To facilitate system integration and expansion, the business linkage module can provide standardized interfaces, including but not limited to REST-based interfaces or message queue-based publish-subscribe methods, to support interfacing with heterogeneous systems such as tour guide systems, map systems, visitor flow systems, and security platforms. In one implementation, the recognition results and linkage events can be encapsulated using a unified data structure to improve data compatibility and processing efficiency between systems.

[0164] In abnormal and degraded scenarios, when the network is unavailable or cloud services are unreachable, the computing and communication modules can switch to local inference mode on the edge or device side and cache the recognition results. After the network is restored, the cached results are uploaded to the upper-layer system in batches. When the input image quality is insufficient (such as too dark, blurry, or severely occluded), the system can include quality labeling information in the output and reduce the business linkage level accordingly to avoid erroneous business actions.

[0165] Through the above-mentioned business linkage and system integration methods, the present invention can effectively integrate the edge scene recognition results into the actual business process of the scenic area, and improve the level of intelligent management and service of the scenic area while ensuring the real-time performance and stability of the system.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0167] The above descriptions are merely some embodiments of the present invention. Those skilled in the art can make various modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the scope of protection of the present invention.

Claims

1. A topology-guided scenic scene recognition method for end-side devices, characterized in that, Includes the following steps: Step 1: Acquire scenic scene images and divide the images into blocks; Step 2: Construct lightweight topological representations of each image patch based on image patch features; Step 3: Extract sparse structural features from the lightweight topological representation to obtain scene-level topological structural features; Step 4: Generate a topology mask prototype for guiding attention based on scene-level topology features, and determine the final topology guiding mask based on the routing selection mechanism. Step 5: Fuse the final topology-guided mask with image patch features to obtain a topology-guided visual representation and output the scenic scene category recognition result; Steps 2 to 5 are implemented through a neural network model. The loss function used in the training phase of the neural network model includes scene classification loss and topology guidance related constraint terms. These topology guidance related constraint terms are used to regulate the distribution characteristics and computational cost of the topology guidance mask. In the inference phase of the neural network model, adaptive topology branch calculation and mask update are triggered based on confidence or computational budget.

2. The topology-guided scenic scene recognition method for end-side devices as described in claim 1, characterized in that, In step 1, the acquired scenic scene images also include accompanying metadata, including: shooting timestamp, device number, and location information.

3. The topology-guided scenic scene recognition method for end-side devices as described in claim 1, characterized in that, Image patch features are set as follows: ； in, For the first Image patch features of an image patch Indicates the first The set of pixels corresponding to each image patch Indicates the first The number of pixels in each image patch Represents pixels The basic feature vector includes color information or brightness information, or a combination of color and brightness information.

4. The topology-guided scenic scene recognition method for end-side devices as described in claim 1, characterized in that, In step 2, the lightweight topological representation of the image patch is specifically as follows: Constructing a topology graph using image patches as nodes ,in, For a set of nodes, For the set of edges, if two image patches are spatially adjacent, or the spatial distance between their center coordinates is less than a preset threshold, then an undirected edge is established between the corresponding nodes. And calculate the edge weights of each undirected edge based on the appearance difference measure between the image patch features corresponding to the two nodes: ； ； in, Representing a topology graph Nodes in With nodes The edge weight of the undirected edge between them. and They are nodes With nodes The corresponding image patch features, For nodes With nodes A measure of the appearance difference between corresponding image patch features. To prevent smoothing constants with a denominator of zero.

5. A topology-guided scenic scene recognition method for end-side devices as described in claim 4, characterized in that, In step 3, the scene-level topological structure features are specifically as follows: For topology graph The first in Nodes Based on image patch features The initial structural features are obtained by linear mapping, denoted as Among them, node number ,in The number of image blocks in the scenic area scene image; By extracting multi-layer sparse structural features, the structural features of nodes are updated layer by layer to obtain their structural features in the next layer: ； in, This is the layer identifier, with an initial value of 0. For the first The structural characteristics of the layers For nodes The set of adjacent nodes, For nodes The identifier of the adjacent node. For the first The structural characteristics of the layers For the first The learnable weight matrix of the layer, It is a non-linear activation function; The edge weights of the undirected edges after normalization; Define L as the number of layers for feature extraction in a multi-layer sparse structure. After extracting the layer structure features, the nodes are obtained. High-rise structural features Then, an aggregation operation is performed to obtain scene-level topological structure features. .

6. The topology-guided scenic scene recognition method for end-side devices as described in claim 1, characterized in that, In step 4, the final topology guiding mask is specifically as follows: Let the set of mask prototypes be represented as ,in, For the number of mask prototypes, and The value is 10%-20% of the number of scenic scene categories; the k-th mask prototype ,and Length and number of image patches Consistent, among which, , For the first Learnable mapping parameters corresponding to each mask prototype It is a non-linear activation function; Based on scene-level topology features Calculate the routing weight vector of the mask prototype ,in , For the first Routing weights for each mask prototype; These are learnable mapping parameters; It is a normalized exponential function; based on For the mask prototype Weighted combinations are performed to obtain the final topology-guided mask. Among them, mask Includes P mask values, each mask value It corresponds to one image block.

7. The topology-guided scenic scene recognition method for end-side devices as described in claim 1, characterized in that, Step 5 specifically includes: The final topology-guided mask consists of P mask values, each mask value corresponding to an image patch; For each image patch, the image patch features are weighted block by block with the corresponding mask value to obtain the topology-guided image patch features for each image patch, i.e., the topology-guided visual representation. The topology-guided visual representation is then fed into a classifier for scenic scene category recognition, and the scenic scene category recognition result is output.

8. The topology-guided scenic scene recognition method for end-side devices as described in claim 1, characterized in that, The loss function used during the training phase is: ； in, For scene classification loss, For topology-guided mask constraint loss, The average activation level constraint loss for the topology-guided mask, and Constraint loss and Weighting coefficients; Constraint loss Set to: ； ； Constraint loss Set to: ； in, For the final topology guiding mask Entropy constraints, The target entropy value is preset. for The mask value corresponding to the p-th image patch is given, where P is the number of image patches. To prevent numerical instability, a smoothing constant is used.

9. A topology-guided scenic scene recognition method for end-side devices as described in claim 1, characterized in that, In the inference phase of a neural network model, adaptive topology branching and mask updates are triggered based on confidence levels, including: When the confidence level in the output scenic scene category recognition result is greater than or equal to the preset threshold If the current scenic scene image is in the correct position, the final topology-guided mask corresponding to the current scenic scene image is used to directly identify the scenic scene category for the next / next frame of scenic scene image; otherwise, adaptive topology branch calculation and mask update are triggered, that is, steps 1 to 5 are executed for the next / next frame of scenic scene image.

10. A topology-guided scenic scene recognition method for end-side devices as described in claim 1, characterized in that, During the inference phase of a neural network model, adaptive topology branching and mask updates are triggered based on the computational budget, including: Scene-level topological structure features based on the scenic scene images of the current frame and the previous frame , Calculate the magnitude of changes in scene-level topology When the range of change Less than the preset threshold If the previous frame's final topology guiding mask is used, the scenic scene category is directly identified for the current frame's scenic scene image; otherwise, the final topology guiding mask corresponding to the current frame is calculated, and then the scenic scene category is identified based on it.

Images