A dynamic feature discrimination method and system based on a graph neural network

Abstract

This application discloses a dynamic feature discrimination method and system based on graph neural networks, belonging to the field of robot localization and mapping algorithms. The method includes: acquiring image sequences and motion data; detecting feature points and extracting multi-dimensional feature information; constructing standardized multimodal feature vectors for the feature points; calculating the comprehensive similarity between feature points and constructing an adjacency matrix through adaptive threshold filtering; using the multimodal feature vectors as node feature matrices and the adjacency matrix as edge relationship descriptions, inputting them into a trained graph autoencoder to obtain latent embedding representations; performing cluster analysis on the latent embedding representations, and completing the discrimination of feature points by calculating the comprehensive score of each cluster; outputting the discrimination results as dynamic / static labels, which are integrated with a SLAM system via a standard interface; significantly improving the accuracy and efficiency of dynamic feature recognition, and also possessing good generalization, adaptability, and system compatibility.

Description

Technical Field

[0001] This application belongs to the field of robot localization and mapping algorithms, and specifically relates to a dynamic feature discrimination method and system based on graph neural networks. Background Technology

[0002] Simultaneous Localization and Mapping (SLAM) is the core technology for mobile robots and autonomous driving systems to achieve autonomous navigation and environmental perception. However, most traditional visual SLAM methods are based on the assumption of static scenes, that is, that most feature points in the environment originate from static background structures. In practical applications, the presence of dynamic objects is unavoidable, such as pedestrians, vehicles, and moving furniture in indoor scenes. These dynamic factors can lead to errors in feature extraction and matching, resulting in decreased localization accuracy or even system failure.

[0003] Existing research on dealing with dynamic scene interference mainly adopts two approaches. The first approach relies on a combination of semantic segmentation and geometric constraints. It uses deep learning models to perform semantic segmentation on images, identifying potential dynamic object regions, and then further combines geometric consistency for feature removal. This type of method can eliminate dynamic interference to some extent, but its robustness is often limited by the training data and category coverage of the semantic segmentation model. When new categories of dynamic objects not included in the training set appear in the scene, the detection and removal performance significantly decreases. The second approach mainly relies on motion consistency, such as using visual-inertial measurement constraints to determine the dynamic attributes of feature points. This type of method can work even without semantic priors, but it usually requires a preset threshold to distinguish between dynamic and static features. The selection of the threshold is highly scene-dependent and can easily lead to over-removal in highly dynamic environments, resulting in insufficient effective features in the system and affecting the stability of overall localization and mapping.

[0004] In summary, existing methods still have significant shortcomings in dynamic scenes: semantic segmentation-based schemes lack adaptability to dynamic objects of unknown categories, while schemes based on geometric and motion consistency face threshold sensitivity issues and struggle to maintain stable performance in different scenarios. Summary of the Invention

[0005] To address the above problems, this application provides a dynamic feature discrimination method based on graph neural networks, including: Acquire image sequences and motion data, detect feature points and extract multi-dimensional feature information, and construct standardized multimodal feature vectors of feature points; Calculate the comprehensive similarity between feature points and construct the adjacency matrix by adaptive threshold filtering; The standardized multimodal feature vectors are used as node feature matrices, and the adjacency matrix is ​​used as edge relation description. These are then input into the trained graph attention network encoder to obtain the latent embedding representation. Cluster analysis is performed on the latent embedding representation, and the feature points are identified by calculating the comprehensive score of each cluster. The output results are dynamic / static tags, which are integrated with SLAM systems via a standard interface.

[0006] Optionally, the step of detecting feature points and extracting multi-dimensional feature information to construct a standardized multimodal feature vector of the feature points includes: Obtain the image coordinates of the feature points; The motion information of feature points between adjacent frames is calculated using the LK optical flow method to obtain the first motion amplitude and first motion direction of the feature points. Motion estimation of feature points is calculated based on the IMU pre-integration method to obtain the second motion amplitude and second motion direction of the feature points; The depth values ​​of feature points are obtained using a depth sensor; The semantic weights of feature points are obtained using the DeepLabV3+ semantic segmentation model; The image coordinates, first motion amplitude, first motion direction, second motion amplitude, second motion direction, depth value, and semantic weight are combined to form a multimodal feature vector of the feature point, and then standardized to obtain the standardized multimodal feature vector of the feature point.

[0007] Optionally, the calculation of the comprehensive similarity between feature points includes: Spatial distance similarity is calculated based on feature point coordinates and distance scale parameters; motion consistency similarity is calculated based on the first and second motion directions; and semantic similarity is calculated based on the semantic weights of each feature point and other feature points. The spatial distance similarity, motion consistency similarity, and semantic similarity are assigned their respective weight coefficients and then summed to obtain the comprehensive similarity. The sum of the weight coefficients for spatial distance similarity, motion consistency similarity, and semantic similarity is 1.

[0008] Optionally, the step of constructing the adjacency matrix through adaptive threshold filtering includes: Based on the comprehensive similarity, feature points with a similarity value greater than or equal to the similarity threshold are retained, while feature points with a similarity value less than the threshold are set to 0. An adjacency matrix is ​​then established and symmetric processing is performed.

[0009] Optionally, the training process of the graph attention network encoder includes: Construct a training dataset, including image sequences, motion data, and corresponding dynamic / static labels; Multimodal feature extraction and adjacency matrix construction are performed on the training data to obtain the node feature matrix and adjacency matrix used for training; A graph attention network encoder is used to aggregate the node feature matrix and adjacency matrix to obtain the latent embedding representation. A multi-head attention mechanism is used to calculate the attention coefficients, and the adjacency matrix is ​​reconstructed through the decoder. The reconstruction loss is then calculated. A contrastive learning strategy is adopted, with feature points within the same cluster as positive sample pairs and feature points between different clusters as negative sample pairs, and the normalized temperature-scaled cross-entropy loss is calculated. The total loss function is obtained based on reconstruction loss, normalized temperature-scaling cross-entropy loss, and balanced hyperparameters. The graph attention network encoder is optimized by minimizing the total loss function until the learning rate reaches a predetermined value or the training cycle is completed, resulting in a trained graph attention network encoder.

[0010] Optionally, the clustering analysis of the latent embedding representation, by calculating the comprehensive score of each cluster, completes the discrimination of feature points. The discrimination result is dynamic / static, including: Cluster analysis is performed on the latent embedding representation. A comprehensive score for each cluster is calculated using a comprehensive scoring expression. If the comprehensive score is greater than a preset threshold, a dynamic discrimination result is output; otherwise, a static discrimination result is output.

[0011] Optionally, the comprehensive scoring expression is:

[0012] in, The overall score for the k-th cluster is given. As the first weighting coefficient, This is the second weighting coefficient. The number of feature points To normalize the motion amplitude, For clustering, Let be the semantic weight of the i-th feature point.

[0013] Based on the same inventive concept, this application provides a dynamic feature discrimination system based on graph neural networks, comprising: The multimodal feature construction unit is used to acquire image sequences and motion data, detect feature points and extract multi-dimensional feature information, and construct standardized multimodal feature vectors of feature points; The graph structure building unit is used to calculate the comprehensive similarity between feature points and to build the adjacency matrix through adaptive threshold filtering. The graph neural network learning unit is used to input the multimodal feature vector as a node feature matrix and the adjacency matrix as an edge relationship description into the trained graph attention network encoder to obtain the latent embedding representation. The dynamic / static feature discrimination unit is used to perform cluster analysis on the latent embedding representation and completes the discrimination of feature points by calculating the comprehensive score of each cluster. The output integration unit is used to output labels with dynamic / static discrimination results, and integrates with the SLAM system through a standard interface.

[0014] Optionally, the step of detecting feature points and extracting multi-dimensional feature information to construct a standardized multimodal feature vector of the feature points includes: Obtain the image coordinates of the feature points; The motion information of feature points between adjacent frames is calculated using the LK optical flow method to obtain the first motion amplitude and first motion direction of the feature points. Motion estimation of feature points is calculated based on the IMU pre-integration method to obtain the second motion amplitude and second motion direction of the feature points; The depth values ​​of feature points are obtained using a depth sensor; The semantic weights of feature points are obtained using the DeepLabV3+ semantic segmentation model; The image coordinates, first motion amplitude, first motion direction, second motion amplitude, second motion direction, depth value, and semantic weight are combined to form a multimodal feature vector of the feature point, and then standardized to obtain the standardized multimodal feature vector of the feature point.

[0015] Optionally, the calculation of the comprehensive similarity between feature points includes: Spatial distance similarity is calculated based on feature point coordinates and distance scale parameters; motion consistency similarity is calculated based on the first and second motion directions; and semantic similarity is calculated based on the semantic weights of each feature point and other feature points. The spatial distance similarity, motion consistency similarity, and semantic similarity are assigned their respective weight coefficients and then summed to obtain the comprehensive similarity. The sum of the weight coefficients for spatial distance similarity, motion consistency similarity, and semantic similarity is 1.

[0016] Optionally, the step of constructing the adjacency matrix through adaptive threshold filtering includes: Based on the comprehensive similarity, feature points with a similarity value greater than or equal to the similarity threshold are retained, while feature points with a similarity value less than the threshold are set to 0. An adjacency matrix is ​​then established and symmetric processing is performed.

[0017] Optionally, the training process of the graph attention network encoder includes: Construct a training dataset, including image sequences, motion data, and corresponding dynamic / static labels; Multimodal feature extraction and adjacency matrix construction are performed on the training data to obtain the node feature matrix and adjacency matrix used for training; A graph attention network encoder is used to aggregate the node feature matrix and adjacency matrix to obtain the latent embedding representation. A multi-head attention mechanism is used to calculate the attention coefficients, and the adjacency matrix is ​​reconstructed through the decoder. The reconstruction loss is then calculated. A contrastive learning strategy is adopted, with feature points within the same cluster as positive sample pairs and feature points between different clusters as negative sample pairs, and the normalized temperature-scaled cross-entropy loss is calculated. The total loss function is obtained based on reconstruction loss, normalized temperature-scaling cross-entropy loss, and balanced hyperparameters. The graph attention network encoder is optimized by minimizing the total loss function until the learning rate reaches a predetermined value or the training cycle is completed, resulting in a trained graph attention network encoder.

[0018] Optionally, the clustering analysis of the latent embedding representation, by calculating the comprehensive score of each cluster, completes the discrimination of feature points. The discrimination result is dynamic / static, including: Cluster analysis is performed on the latent embedding representation. A comprehensive score for each cluster is calculated using a comprehensive scoring expression. If the comprehensive score is greater than a preset threshold, a dynamic discrimination result is output; otherwise, a static discrimination result is output.

[0019] Optionally, the comprehensive scoring expression is:

[0020] in, The overall score for the k-th cluster is given. As the first weighting coefficient, This is the second weighting coefficient. The number of feature points To normalize the motion amplitude, For clustering, Let be the semantic weight of the i-th feature point.

[0021] Furthermore, this application also provides a computing device, comprising: at least one processor and a memory; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, a dynamic feature discrimination method based on a graph neural network as described above is implemented.

[0022] In another aspect, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the dynamic feature discrimination method based on a graph neural network as described above.

[0023] Compared with the prior art, this application has the following advantages: This application provides a dynamic feature discrimination method and system based on graph neural networks, comprising: acquiring image sequences and motion data, detecting feature points and extracting multi-dimensional feature information, and constructing standardized multimodal feature vectors of feature points; calculating the comprehensive similarity between feature points and constructing an adjacency matrix through adaptive threshold filtering; using the standardized multimodal feature vectors as node feature matrices and the adjacency matrix as edge relation descriptions, inputting them into a trained graph attention network encoder to obtain latent embedding representations; performing cluster analysis on the latent embedding representations, and completing the discrimination of feature points by calculating the comprehensive score of each cluster; outputting the discrimination results as dynamic / static labels, and integrating with a SLAM system through a standard interface.

[0024] It significantly improves the accuracy and efficiency of dynamic feature recognition, and also has good generalization, adaptability and system compatibility, making it suitable for visual SLAM applications in a variety of high-dynamic scenarios such as inspection robots, service robots, autonomous driving, and augmented reality.

[0025] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A flowchart illustrating the implementation of a dynamic feature discrimination method based on graph neural networks provided in this application is shown. Figure 2 The diagram illustrates the construction of the dynamic feature discrimination method based on graph neural networks provided in this application. Figure 3 The graph neural network learning process provided in this application is illustrated. Figure 4 The dynamic features (red) and static features (green) detected by DSF-Net are shown. Figure 5 The diagram shows dynamic features (red) and static features (green) detected based on semantic segmentation methods. Figure 6 The localization results of the SLAM system integrating the dynamic feature judgment network of this application are shown. Figure 7 This paper shows the localization results of a SLAM system that does not integrate the dynamic feature judgment network of this application; Figure 8 The system organization diagram of a dynamic feature discrimination device based on a graph neural network provided in this application is shown. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] Example 1 This application provides a dynamic feature discrimination method based on graph neural networks, such as... Figure 1 ,include: Acquire image sequences and motion data, detect feature points and extract multi-dimensional feature information, and construct standardized multimodal feature vectors of feature points; Calculate the comprehensive similarity between feature points and construct the adjacency matrix by adaptive threshold filtering; The standardized multimodal feature vectors are used as node feature matrices, and the adjacency matrix is ​​used as edge relation description. These are then input into the trained graph attention network encoder to obtain the latent embedding representation. Cluster analysis is performed on the latent embedding representation, and the feature points are identified by calculating the comprehensive score of each cluster. The output results are dynamic / static tags, which are integrated with SLAM systems via a standard interface.

[0030] This application embodiment can be implemented on a mobile computing platform equipped with multiple sensors. The hardware platform uses an Intel Core i7-11800H processor and an NVIDIA RTX 3060 GPU, with 32GB of DDR4 system memory. The sensor system includes an Intel RealSense D455 RGB-D camera (providing 640×480 resolution color and depth images) and a BMI088 inertial measurement unit (IMU). The software environment is an Ubuntu 20.04 operating system, and the deep learning module is implemented based on PyTorch 1.9.0.

[0031] The structural diagram of the method in the embodiments of this application is as follows: Figure 2 The input image frames and IMU data are processed by a multimodal feature construction module, which extracts feature points, calculates optical flow, uses IMU pre-integration methods, incorporates depth information, and performs semantic segmentation to construct multimodal feature vectors. A graph structure construction module then constructs a graph using nodes as feature points, calculates comprehensive similarity, and uses adjacency matrices as edge relationships. This graph is input into a graph neural network learning unit, passes through a GAT encoder and decoder, and undergoes contrastive learning optimization to obtain a latent embedding representation. A dynamic / static feature discrimination unit performs clustering based on the latent embedding representation, calculates clustering scores, and completes the category determination of feature points, resulting in dynamic / static labels that form a binary mask output.

[0032] Optionally, obtaining the standardized multimodal feature vector for each feature point includes: Obtain the i Image coordinates of feature points ( ); The motion information of feature points between adjacent frames is calculated using the LK optical flow method to obtain the first... i The first motion amplitude of each feature point and the first direction of motion ; Motion estimation of feature points is calculated based on the IMU pre-integration method to obtain the first... i The second motion amplitude of each feature point Second direction of motion ; Obtain the first through the depth sensor i Depth value of each feature point ; Using the DeepLabV3+ semantic segmentation model to obtain the first i Semantic weights of each feature point In this embodiment, the weight of the dynamic object (such as pedestrians and vehicles) region is set to 0.8, and the weight of the static region is set to 0.2. The image coordinates, first motion amplitude, first motion direction, second motion amplitude, second motion direction, depth value, and semantic weight are combined to form the first... i Multimodal feature vectors of 1 feature point And after standardization, the first... i Standardized multimodal feature vectors of 1 feature point .

[0033]

[0034]

[0035] in, This represents the mean of the multimodal feature vectors of all nodes. This represents the standard deviation of the multimodal feature vectors of all nodes. and It is obtained by calculating the feature mean and standard deviation of the training dataset.

[0036] The set of feature points is modeled as a graph structure G=(V,E), where the set of nodes V represents the feature points and the set of edges E represents the relationships between the feature points.

[0037] Node features are represented by standardized multimodal feature vectors, where each component is a standardized feature value, as shown in the following equation:

[0038] in, Represents the standardized first i Image coordinates of feature points Represents the standardized first i The first motion amplitude and first motion direction of each feature point Represents the standardized first i The second motion amplitude and second motion direction of each feature point Represents the standardized first i Depth values ​​of each feature point Represents the standardized first i The semantic weights of each feature point.

[0039] Optionally, the comprehensive similarity between the computing nodes includes: Spatial distance similarity is calculated based on feature point coordinates and distance scale parameters;

[0040] in, Indicates the first i The feature point and the first j Spatial distance similarity between feature points =[ ] indicates the first i The coordinates of the feature points Indicates the first j The coordinates of the feature points This represents the distance scale parameter, which is 10 (pixel units) in this embodiment.

[0041] The motion consistency similarity is calculated based on the first and second motion directions using the cosine value of the motion direction.

[0042] in, Indicates the first i The feature point and the first j Motion consistency similarity between feature points Indicates the first j The first direction of motion of each feature point Indicates the first j The second direction of motion of each feature point.

[0043] Based on the i The first feature point and the second j The semantic similarity is calculated by the semantic weights of each feature point. The closer the semantic weights are, the higher the similarity.

[0044] in, Indicates the first i The feature point and the first j Semantic similarity between feature points Indicates the first j The semantic weights of each feature point.

[0045] The spatial distance similarity, motion consistency similarity, and semantic similarity are assigned their respective weight coefficients and then summed to obtain the comprehensive similarity.

[0046] in, This represents the overall similarity between the i-th feature point and the j-th feature point. Weight parameters representing spatial distance similarity, The weighting coefficients representing the similarity of motion consistency Weight parameters representing semantic similarity.

[0047] The sum of the weighting coefficients for spatial distance similarity, motion consistency similarity, and semantic similarity is 1 (in this embodiment, the weighting coefficients are set to 1). =0.4, =0.3, =0.3).

[0048] Optionally, the construction of the adjacency matrix and the symmetry-based processing include: Based on comprehensive similarity, feature points with similarity values ​​greater than or equal to a similarity threshold are retained, while feature points with similarity values ​​less than the threshold are set to 0, and an adjacency matrix is ​​established. , N This represents the total number of nodes in the graph (i.e., the number of feature points), and is then processed to be symmetrical.

[0049]

[0050] in, Represents the adjacency matrix of the nth element. One element, This represents the similarity threshold, used to preserve significant connections and reduce noisy edges in the graph (set to 0.6 in this embodiment).

[0051] To ensure the symmetry of the adjacency matrix, symmetry processing is performed to guarantee the undirectedness of the graph structure, including:

[0052] in, express The transpose of .

[0053] Optionally, the training process of the graph autoencoder includes: Construct a training dataset, including image sequences, motion data, and corresponding dynamic / static labels; Multimodal feature extraction and adjacency matrix construction are performed on the training data to obtain the node feature matrix and adjacency matrix used for training; A graph attention network encoder is used to aggregate the node feature matrix and adjacency matrix to obtain the latent embedding representation, such as... Figure 3 , First, feature aggregation is performed using a graph attention network encoder:

[0054] in, Represents the potential embedding representation. Represents the node feature matrix, N This represents the total number of nodes in the graph (i.e., the number of feature points). D This represents the dimension of the input features at each node. This represents the adjacency matrix.

[0055] Then, the attention coefficients are calculated using a multi-head attention mechanism, the adjacency matrix is ​​reconstructed through the decoder, and the reconstruction loss is calculated. This embodiment uses a three-layer GAT encoder, with 4 attention heads per layer and a hidden layer dimension of 128. The formula for calculating the attention coefficient is:

[0056] in, Indicates the attention coefficient. Represents a node i The set of neighboring nodes, Represents the weight matrix. Represents the attention vector. , , Representing nodes respectively i , j , k The eigenvectors are represented by LeakyReLU, which indicates a negative slope. In this embodiment, it is set to 0.2.

[0057] Next, the adjacency matrix is ​​reconstructed using the decoder as follows:

[0058] in, express The transpose of the matrix, It is the sigmoid activation function.

[0059] The formula for calculating the reconstruction loss is:

[0060] in, This represents the reconstruction loss.

[0061] A contrastive learning strategy is adopted, with feature points within the same cluster as positive sample pairs and feature points between different clusters as negative sample pairs, and the normalized temperature-scaled cross-entropy loss is calculated. The formula for calculating the normalized temperature-scaled cross-entropy loss is:

[0062] in, This represents the normalized temperature-scaled cross-entropy loss. N Represents the total number of samples. and These represent the embedding representations of two feature points in the same positive sample pair. Indicates the first j Embedded representation of each feature point This represents the temperature parameter (set to 0.1 in this embodiment).

[0063] The total loss function is obtained based on reconstruction loss, normalized temperature-scaling cross-entropy loss, and balanced hyperparameters. The graph autoencoder is optimized by minimizing the total loss function until the learning rate reaches a predetermined value or the training cycle is completed.

[0064] The formula for calculating the total loss function is:

[0065] in, Indicates the total loss. This represents the balancing hyperparameter.

[0066] This embodiment uses the Adam optimizer for training, with a learning rate of 0.001 and a training cycle of 200 rounds.

[0067] Optionally, the clustering analysis of the latent embedding representation, by calculating the comprehensive score of each cluster, completes the discrimination of feature points. The discrimination result is dynamic / static, including: Cluster analysis is performed on the latent embedding representation. K-means clustering (number of clusters K=2) is used to calculate the comprehensive score of each cluster through a comprehensive scoring expression. If the comprehensive score is greater than a preset threshold, a dynamic discrimination result is output; otherwise, a static discrimination result is output.

[0068] Optionally, cluster analysis can be performed on the latent embedding representation Z to obtain several clusters. Each cluster contains If there are 1 feature point, then the comprehensive scoring expression is:

[0069] in, The overall score for the k-th cluster is given. and The weighting coefficients in this step (as set in this embodiment) , ), For the first i The semantic weight of each feature point The normalized motion amplitude is defined as follows:

[0070]

[0071] In the formula, and The motion amplitudes obtained by optical flow and IMU pre-integration are respectively. and Original motion amplitude for all feature points The mean and standard deviation.

[0072] The feature point category is determined as follows:

[0073] in, Indicates the first i The category of each feature point This represents the scoring threshold (set to 0.5 in this example).

[0074] The discrimination results are published to a ROS topic in a standard message format for the SLAM system to subscribe to. Dynamic features are marked as 0, and static features are marked as 1, forming a binary mask output.

[0075] Compared with existing technologies, the dynamic feature discrimination network and method based on graph neural networks provided in this application have the following significant advantages: It has high accuracy in dynamic feature recognition and good generalization ability for unknown dynamic objects. By fusing multimodal features (including image coordinates, optical flow motion, IMU integral motion, depth, and semantic information) and employing a graph attention network (GAT) for feature relationship modeling, this application can fully capture the spatial and motion consistency between feature points, significantly improving the discrimination accuracy of dynamic and static features. Traditional geometric-based methods for detecting dynamic features, such as... Figure 4 Dynamic features detected based on semantic rule methods, such as Figure 5 On public datasets such as TUM RGB-D, the dynamic feature recognition accuracy of this application reaches 98.0%, such as... Figure 6 This method achieves an improvement of approximately 36.1% compared to traditional geometric or semantic rule-based methods. It maintains a high recall rate, particularly in identifying semantically undefined, dynamic objects (such as unknown pedestrians or moving furniture). In SLAM systems that do not integrate the dynamic feature judgment network of this application, the error mapped onto the trajectory in the localization results is significantly reduced. Figure 7 The accuracy is insufficient.

[0076] 2) Strong adaptive discrimination capability, reducing reliance on manually set thresholds. Traditional methods often rely on manually setting thresholds for dynamic feature filtering. This application combines graph structure learning and contrastive learning to achieve end-to-end feature relationship modeling and adaptive discrimination, avoiding over- or under-rejection problems caused by fixed thresholds, and improving the system's adaptability and stability in environments with different levels of dynamics.

[0077] 3) Modular design, easy to integrate and highly compatible This application can be seamlessly integrated as an independent module into various SLAM systems (such as ORB-SLAM3, VINS-Mono, etc.), outputting standardized dynamic / static binary labels without changing the architecture and parameters of the original SLAM system, and has good system compatibility and engineering reusability.

[0078] 4) Excellent robustness, adaptable to complex dynamic environments Under the joint learning framework of multimodal feature fusion and graph neural network, this application shows strong robustness to interference factors such as illumination changes, motion blur, fast camera movement and partial occlusion. It can still maintain a recognition accuracy of more than 85% in high-speed dynamic scenes, significantly improving the availability and reliability of SLAM system in complex real-world environments.

[0079] Example 2 Based on the same inventive concept, this application also provides a dynamic feature discrimination device based on graph neural networks, such as... Figure 8 ,include: The multimodal feature construction unit is used to acquire image sequences and motion data, detect feature points and extract multi-dimensional feature information, and construct standardized multimodal feature vectors of feature points; The graph structure building unit is used to calculate the comprehensive similarity between feature points and to build the adjacency matrix through adaptive threshold filtering. The graph neural network learning unit is used to input the multimodal feature vector as a node feature matrix and the adjacency matrix as an edge relationship description into the trained graph autoencoder to obtain the latent embedding representation. The dynamic / static feature discrimination unit is used to perform cluster analysis on the latent embedding representation and completes the discrimination of feature points by calculating the comprehensive score of each cluster. The output integration unit is used to output labels with dynamic / static discrimination results, and integrates with the SLAM system through a standard interface.

[0080] Optionally, the step of detecting feature points and extracting multi-dimensional feature information to construct a standardized multimodal feature vector of the feature points includes: Obtain the image coordinates of the feature points; The motion information of feature points between adjacent frames is calculated using the LK optical flow method to obtain the first motion amplitude and first motion direction of the feature points. Motion estimation of feature points is calculated based on the IMU pre-integration method to obtain the second motion amplitude and second motion direction of the feature points; The depth values ​​of feature points are obtained using a depth sensor; The semantic weights of feature points are obtained using the DeepLabV3+ semantic segmentation model; The image coordinates, first motion amplitude, first motion direction, second motion amplitude, second motion direction, depth value, and semantic weight are combined to form a multimodal feature vector of the feature point, and then standardized to obtain the standardized multimodal feature vector of the feature point.

[0081] Optionally, the calculation of the comprehensive similarity between feature points includes: Spatial distance similarity is calculated based on feature point coordinates and distance scale parameters; motion consistency similarity is calculated based on the first and second motion directions; and semantic similarity is calculated based on the semantic weights of each feature point and other feature points. The spatial distance similarity, motion consistency similarity, and semantic similarity are assigned their respective weight coefficients and then summed to obtain the comprehensive similarity. The sum of the weight coefficients for spatial distance similarity, motion consistency similarity, and semantic similarity is 1.

[0082] Optionally, the step of constructing the adjacency matrix through adaptive threshold filtering includes: Based on the comprehensive similarity, feature points with a similarity value greater than or equal to the similarity threshold are retained, while feature points with a similarity value less than the threshold are set to 0. An adjacency matrix is ​​then established and symmetric processing is performed.

[0083] Optionally, the training process of the graph attention network encoder includes: Construct a training dataset, including image sequences, motion data, and corresponding dynamic / static labels; Multimodal feature extraction and adjacency matrix construction are performed on the training data to obtain the node feature matrix and adjacency matrix used for training; A graph attention network encoder is used to aggregate the node feature matrix and adjacency matrix to obtain the latent embedding representation. A multi-head attention mechanism is used to calculate the attention coefficients, and the adjacency matrix is ​​reconstructed through the decoder. The reconstruction loss is then calculated. A contrastive learning strategy is adopted, with feature points within the same cluster as positive sample pairs and feature points between different clusters as negative sample pairs, and the normalized temperature-scaled cross-entropy loss is calculated. The total loss function is obtained based on reconstruction loss, normalized temperature-scaling cross-entropy loss, and balanced hyperparameters. The graph attention network encoder is optimized by minimizing the total loss function until the learning rate reaches a predetermined value or the training cycle is completed, resulting in a trained graph attention network encoder.

[0084] Optionally, the clustering analysis of the latent embedding representation, by calculating the comprehensive score of each cluster, completes the discrimination of feature points. The discrimination result is dynamic / static, including: Cluster analysis is performed on the latent embedding representation. A comprehensive score for each cluster is calculated using a comprehensive scoring expression. If the comprehensive score is greater than a preset threshold, a dynamic discrimination result is output; otherwise, a static discrimination result is output.

[0085] Optionally, the comprehensive scoring expression is:

[0086] in, The overall score for the k-th cluster is given. As the first weighting coefficient, This is the second weighting coefficient. The number of feature points To normalize the motion amplitude, For clustering, Let be the semantic weight of the i-th feature point.

[0087] Example 3 Based on the same inventive concept, this application also provides an electronic device. The electronic device of this application includes at least one processor and at least one storage medium electrically connected to the processor. The storage medium is electrically connected to the processor, wherein the storage medium stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described above.

[0088] Example 4 Based on the same inventive concept, this application also provides a storage medium storing instructions executable by at least one processor, the instructions being executed by at least one processor to enable at least one processor to perform the method described above.

[0089] Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A dynamic feature discrimination method based on graph neural networks, characterized in that, include: Acquire image sequences and motion data, detect feature points and extract multi-dimensional feature information, and construct standardized multimodal feature vectors of feature points; Calculate the comprehensive similarity between feature points and construct the adjacency matrix by adaptive threshold filtering; The standardized multimodal feature vectors are used as node feature matrices, and the adjacency matrix is ​​used as edge relation description. These are then input into the trained graph attention network encoder to obtain the latent embedding representation. Cluster analysis is performed on the latent embedding representation, and the feature points are identified by calculating the comprehensive score of each cluster. The output results are dynamic / static tags, which are integrated with SLAM systems via a standard interface.

2. The method according to claim 1, characterized in that, The process of detecting feature points and extracting multi-dimensional feature information to construct a standardized multimodal feature vector for the feature points includes: Obtain the image coordinates of the feature points; The motion information of feature points between adjacent frames is calculated using the LK optical flow method to obtain the first motion amplitude and first motion direction of the feature points. Motion estimation of feature points is calculated based on the IMU pre-integration method to obtain the second motion amplitude and second motion direction of the feature points; The depth values ​​of feature points are obtained using a depth sensor; The semantic weights of feature points are obtained using the DeepLabV3+ semantic segmentation model; The image coordinates, first motion amplitude, first motion direction, second motion amplitude, second motion direction, depth value, and semantic weight are combined to form a multimodal feature vector of the feature point, and then standardized to obtain the standardized multimodal feature vector of the feature point.

3. The method according to claim 2, characterized in that, The calculation of the comprehensive similarity between feature points includes: Spatial distance similarity is calculated based on feature point coordinates and distance scale parameters; motion consistency similarity is calculated based on the first and second motion directions; and semantic similarity is calculated based on the semantic weights of each feature point and other feature points. The spatial distance similarity, motion consistency similarity, and semantic similarity are assigned their respective weight coefficients and then summed to obtain the comprehensive similarity. The sum of the weight coefficients for spatial distance similarity, motion consistency similarity, and semantic similarity is 1.

4. The method according to claim 3, characterized in that, The construction of the adjacency matrix through adaptive threshold filtering includes: Based on the comprehensive similarity, feature points with a similarity value greater than or equal to the similarity threshold are retained, while feature points with a similarity value less than the threshold are set to 0. An adjacency matrix is ​​then established and symmetric processing is performed.

5. A method according to claim 1, characterized in that, The training process of the graph attention network encoder includes: Construct a training dataset, including image sequences, motion data, and corresponding dynamic / static labels; Multimodal feature extraction and adjacency matrix construction are performed on the training data to obtain the node feature matrix and adjacency matrix used for training; A graph attention network encoder is used to aggregate the node feature matrix and adjacency matrix to obtain the latent embedding representation. A multi-head attention mechanism is used to calculate the attention coefficients, and the adjacency matrix is ​​reconstructed through the decoder. The reconstruction loss is then calculated. A contrastive learning strategy is adopted, with feature points within the same cluster as positive sample pairs and feature points between different clusters as negative sample pairs, and the normalized temperature-scaled cross-entropy loss is calculated. The total loss function is obtained based on reconstruction loss, normalized temperature-scaling cross-entropy loss, and balanced hyperparameters. The graph attention network encoder is optimized by minimizing the total loss function until the learning rate reaches a predetermined value or the training cycle is completed, resulting in a trained graph attention network encoder.

6. The method according to claim 1, characterized in that, The clustering analysis of the latent embedding representation, by calculating the comprehensive score of each cluster, completes the discrimination of feature points. The discrimination result is dynamic / static, including: Cluster analysis is performed on the latent embedding representation. A comprehensive score for each cluster is calculated using a comprehensive scoring expression. If the comprehensive score is greater than a preset threshold, a dynamic discrimination result is output; otherwise, a static discrimination result is output.

7. A method as described in claim 6, characterized in that, The comprehensive scoring expression is as follows: in, The overall score for the k-th cluster is given. As the first weighting coefficient, This is the second weighting coefficient. The number of feature points To normalize the motion amplitude, For the number of clusters, Let be the semantic weight of the i-th feature point.

8. A dynamic feature discrimination system based on graph neural networks, characterized in that, include: The multimodal feature construction unit is used to acquire image sequences and motion data, detect feature points and extract multi-dimensional feature information, and construct standardized multimodal feature vectors of feature points; The graph structure building unit is used to calculate the comprehensive similarity between feature points and to build the adjacency matrix through adaptive threshold filtering. The graph neural network learning unit is used to input the multimodal feature vector as a node feature matrix and the adjacency matrix as an edge relationship description into the trained graph attention network encoder to obtain the latent embedding representation. The dynamic / static feature discrimination unit is used to perform cluster analysis on the latent embedding representation and completes the discrimination of feature points by calculating the comprehensive score of each cluster. The output integration unit is used to output labels with dynamic / static discrimination results, and integrates with the SLAM system through a standard interface.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the dynamic feature discrimination method based on a graph neural network as described in any one of claims 1-7.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements the steps of the dynamic feature discrimination method based on a graph neural network as described in any one of claims 1-7.

Images