A SDF vector high-precision map change detection method

Abstract

The present invention discloses a change detection method for an SDF vector high-precision map, comprising: taking a surround image as input, processing it into a bird's-eye view map and outputting a hierarchical semantic bird's-eye view map through a semantic segmentation network, constructing an SDF subgraph on it and detecting key points and descriptors, locating the SDF subgraph in the SDF vector high-precision map through the key points and descriptors, retaining the SDF subgraph that meets the threshold requirement and then optimizing the coordinates of the SDF subgraph. Training an improved Siamese neural network and using a shared spatial attention mechanism to learn the difference between the SDF subgraph and the SDF vector high-precision map. Detecting changes in the SDF vector high-precision map through the relationship between the similarity d output by the network and the change detection threshold T. The present invention introduces a shared spatial attention mechanism into the Siamese neural network, so that the network can better capture the structured areas in the SDF subgraph and the SDF vector high-precision map, thereby improving the accuracy of change detection in the SDF vector high-precision map.

Description

Technical Field

[0001] The present invention relates to the technical field of high-precision map change detection, and in particular to a high-precision map change detection method based on an improved Siamese neural network using an SDF vector. Background Art

[0002] Change detection in HD maps remains an unresolved research challenge in the autonomous driving field. Existing HD map change detection methods are mostly based on point cloud HD maps. However, the storage efficiency of point cloud HD maps is far lower than that of vector HD maps. Therefore, the application of vector HD maps is one of the key future development directions in the autonomous driving field, and change detection methods for vector HD maps are an urgent task.

[0003] Current HD map change detection methods can be categorized into three categories: 1. CNN-based binary classification and feature fusion methods; 2. Deep metric learning-based methods; and 3. Deep neural network-based methods. The main drawbacks of these methods are that, on the one hand, they are independent modules of the autonomous vehicle's positioning system and fail to fully utilize the relationship between HD maps and front-end visual features. On the other hand, these methods lack the ability to extract semantic information about the changed road elements. Summary of the Invention

[0004] The purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology and propose an SDF vector high-precision map change detection method based on an improved Siamese neural network, which can better capture the structured areas in the SDF subgraphs and vector high-precision maps, and improve the accuracy of SDF vector high-precision map change detection.

[0005] To achieve the above objectives, the present invention provides a technical solution: a method for detecting changes in SDF vector high-precision maps based on an improved Siamese neural network, comprising the following steps:

[0006] S1: Take the surround image as input and process it into a bird's-eye view. Train a semantic segmentation neural network with the bird's-eye view as input and output the semantic segmentation result of the bird's-eye view.

[0007] S2: The semantic segmentation results of the bird's-eye view are layered according to semantic categories, and a layered bird's-eye view semantic map is output. For the layered bird's-eye view semantic map, an SDF subgraph is constructed for each semantic layer, key points of the SDF subgraph are detected, and descriptors are constructed for the key points.

[0008] S3: Use key points and their descriptors to locate the SDF subgraph in the SDF vector HD map, record the positioning fit score, and retain the SDF subgraphs with a fit score higher than the threshold;

[0009] S4: For the SDF subgraphs whose fitting scores are higher than the threshold, select the SDF subgraphs adjacent to them in time and space from the SDF vector HD map, and adjust the coordinates of the current SDF subgraph using the ICP algorithm;

[0010] S5: For the SDF subgraph after coordinate adjustment, the changes in the SDF vector high-precision map are detected through the pre-trained improved Siamese neural network. The improvement of the Siamese neural network is to introduce a shared spatial attention mechanism, so that the network can better capture the structured areas in the SDF subgraph and the SDF vector high-precision map, so as to improve the accuracy of change detection; in the network inference stage, the relationship between the similarity d between the SDF subgraph and the SDF vector high-precision map and the change detection threshold T is used to judge whether the SDF vector high-precision map has changed. If d<T, the SDF vector high-precision map has not changed; if d≥T, the SDF vector high-precision map has changed.

[0011] Furthermore, in step S1, the surround view image is processed into a bird's-eye view using an inverse perspective transformation algorithm. The inverse perspective transformation algorithm formula is expressed as follows:

[0012]

[0013] Where, Indicates pixel coordinates, u is the horizontal coordinate, v is the vertical coordinate, is the homogeneous form of pixel coordinates; Represents the coordinates of the two-dimensional plane in the Cartesian coordinate system after the inverse perspective transformation, where x is the vertical coordinate and y is the horizontal coordinate. is the homogeneous form of the coordinate; Represents the intrinsic parameter matrix of the camera, K -1 represents the inverse matrix of K; Is a 4×4 transformation matrix, representing the transformation from the world coordinate system to the camera coordinate system, T -1 is the inverse matrix of T; is a 3×3 rotation matrix, R ij is the rotation component of the rotation matrix, 1≤i,j≤3; is a 3×1 vector, indicating translation, t k is the translation component, 1≤k≤3; is a constant representing the normalized distance; the bird's-eye view image is used as input and YOLOP is used as the semantic segmentation neural network.

[0014] Furthermore, in step S2, the specific steps of constructing the SDF subgraph are as follows: taking the bird's-eye view semantic image as input, first filtering outliers in the bird's-eye view semantic image, then converting it into a grayscale image and dividing the image into grayscale images with different heights according to the grayscale value, and then applying the SDF algorithm to convert the grayscale images with different heights into SDF subgraphs. The formula of the SDF subgraph is defined as follows:

[0015] f(p)=sign(p)·min q∈S ||pq||

[0016] Where f(p) represents the SDF subgraph space, S represents a closed surface in the SDF subgraph space, p represents a point in the SDF subgraph space, q represents a point on S, the function min represents taking the minimum value in the set, and the function sign represents taking the positive or negative sign.

[0017] Furthermore, in step S2, the specific steps of constructing the key points of the SDF subgraph are: taking the SDF subgraph as input, first applying Gaussian blur to the SDF subgraph to obtain S g , then S g Apply the first-order Sobel operator in the x and y directions respectively to obtain the approximate first-order derivative image S of the SDF subgraph dx and S dy , again to S dx and S dy Apply the first-order Sobel operator in the x and y directions respectively to obtain the second-order derivative image S of the SDF subgraph dxx 、S dxy 、S dyx and S dyy Then traverse all the coordinate points on the SDF subgraph and calculate the Hessian matrix and the determinant DoH, eigenvalue a and eigenvector of the Hessian matrix corresponding to each coordinate point The formula for calculating the determinant DoH of the Hessian matrix is ​​as follows:

[0018]

[0019] Then traverse DoH to record the coordinates of the local maximum points. The local maximum points are the key points of the SDF subgraph, called SDF key points. Finally, according to the key point coordinates and the corresponding eigenvalues ​​a and eigenvectors Classify the coordinate points as local maximum key points P max and the local minimum key point P min ;

[0020] The specific steps of constructing the descriptor are: traverse all SDF key points, construct a region of interest with the key point coordinates as the center and r as the radius for each key point, use a 36-bin histogram to count the gradient direction in the region of interest, record the main gradient direction, and then calculate the average SDF distance in the region of interest. Finally, use the main gradient direction, 36-bin histogram, average SDF distance and key point type as the descriptor of the current key point.

[0021] Furthermore, in step S3, the specific steps of locating the SDF subgraph in the SDF vector high-precision map are as follows: the SDF vector high-precision map is stored in a KD-Tree data structure, first, the SDF subgraph coordinates p c As the initial value, find the 5 coordinates closest to the coordinate in the KD-Tree, then traverse the 5 nearest coordinates and calculate the fitting score between the SDF subgraph coordinate and the 5 nearest coordinates through the ICP algorithm. Take the nearest coordinate with the smallest fitting score as the optimal match of the SDF subgraph coordinate, and then calculate the transformation matrix T corresponding to the optimal match. b , through the formula p w =T b p c Calculate the coordinates p of the SDF subgraph in the SDF vector high-precision map w ,in, It is the coordinate of the world coordinate system in three-dimensional space, w represents the world coordinate system, is the coordinate of the camera coordinate system in three-dimensional space, c represents the camera coordinate system, It is a 4×4 matrix that represents the coordinate transformation from the camera coordinate system to the world coordinate system in three-dimensional space, where R ii represents the components of the rotation matrix, t k Represents the components of the translation vector, 1≤i, j, k≤3; for the best match and its fitting score, when the fitting score is higher than the fitting threshold, the best match is retained for the subsequent change detection process.

[0022] Furthermore, in step S4, the specific steps of adjusting the coordinates of the SDF subgraphs with fitting scores higher than the threshold are as follows: for the SDF subgraphs with fitting scores higher than the threshold, the SDF subgraph coordinates are used as initial values, and the SDF vector high-precision map segments within a physical distance of 30 meters from the coordinates are searched from the SDF vector high-precision map in the forward and reverse directions respectively, and then the SDF subgraphs and the SDF vector high-precision map segments are fitted using the ICP algorithm to obtain the fitting scores and the transformation matrix T′. b , through the formula p′ w =T′ b p w Get the adjusted SDF subgraph coordinates p′w, where Indicates the adjusted SDF subgraph coordinates, which are the coordinates in the world coordinate system in three-dimensional space. is a 4×4 transformation matrix, where R ij represents the components of the rotation matrix, t k Represents the components of the translation vector, 1≤i,j,k≤3.

[0023] Furthermore, in step S5, the improved Siamese neural network introduces a shared spatial attention mechanism on the basis of the original Siamese neural network, so that the network can pay more attention to the areas that are more critical for change detection in the SDF subgraph and the SDF vector high-precision map. The shared spatial attention mechanism weights the key areas of the feature map, thereby enhancing the feature representation of these areas, thereby improving the sensitivity and accuracy of change detection;

[0024] The improved Siamese neural network consists of four layers: input layer, feature extraction layer, spatial attention layer, and feature difference measurement layer. The details are as follows:

[0025] ① Input layer: accepts SDF subgraph x1 and SDF vector high-precision map x2 as input, and the size of x1 and x2 are both 224×224;

[0026] ② Feature extraction layer: A custom convolutional neural network is used to extract feature maps F from each input. i ′;F i′ =MP(Conv 3×3 (MP(Conv 3×3 (MP(Conv 3×3 (x i′ ))))))

[0027] Where x i′ Represents an SDF subgraph or a high-precision map of an SDF vector, i′∈{1,2}; MP represents the maximum pooling layer, Conv 3×3 Represents a 3×3 convolution kernel; three sets of convolution and pooling operations are used to extract features. The number of channels after convolution is 32, 64, and 128 respectively. A 3×3 convolution kernel is used, the stride is set to 1, and the padding is set to 1. The SDF subgraph contains information about the surface distance of the object. This information is highly structured in spatial distribution. The convolution layer can effectively capture this spatial structure information and extract features useful for change detection tasks. The pooling layer uses maximum pooling with a stride of 2 to obtain the original input feature maps F1 and F2.

[0028] ③ Spatial attention layer: Use the shared spatial attention mechanism to capture the key areas in the SDF subgraph and the SDF vector HD map, and weight the key areas:

[0029] Si′ =Sigmoid(Conv 1×1 (Relu(Conv 3×3 (F i′ ))))

[0030] Where, Conv 1×1 Represents a 1×1 convolution kernel, Relu represents a Relu activation function, and Sigmoid represents a Sigmoid activation function. First, a 3×3 convolution is used to capture local spatial features. The number of output channels is set to 8 to balance feature representation capability and computational efficiency. The step size is set to 1 and the padding is set to keep the same size as the input. Then, the Relu activation function is connected to enhance the network expression capability. The convolution layer for generating the spatial attention map is connected, with the convolution kernel size set to 1×1, the number of output channels set to 1, and the step size set to 1. Then, the Sigmoid activation layer is connected to obtain the spatial attention weight map S. i′ ;Use spatial attention weight map S i′ Original input feature map F i′ Weighted, get the weighted feature map, that is, the feature vector Where ⊙ represents element-wise multiplication;

[0031] ④ Feature difference measurement layer: use Euclidean distance to calculate two feature vectors The difference, Where n represents the dimension of the feature vector, and They are feature maps and The value in the i″th dimension, i″∈{1,2,3,...,n};

[0032] Improve the Siamese neural network to use the contrast loss function during training. The contrast loss function is defined as: Where W is the network parameter, x1 and x2 are the SDF subgraph and SDF vector high-precision map respectively, and y′ is a label, which is 1 when x1 and x2 are similar and 0 when x1 and x2 are not similar. Indicates the similarity between x1 and x2. m is a boundary value used to distinguish whether x1 and x2 are similar. max(0,md) means taking the larger value between 0 and md.

[0033] Furthermore, in step S5, the steps of using the improved Siamese neural network to detect changes in the SDF vector high-precision map are as follows: In the change detection stage, first calculate the feature vector F1 of the SDF subgraph x1 and the SDF vector high-precision map x2 att and Then the similarity d is calculated and compared with the change detection threshold T to determine whether the SDF vector high-precision map x2 has changed. If d < T, it means that x1 and x2 are similar, and the SDF vector high-precision map has not changed. If d ≥ T, it means that x1 and x2 are not similar, and the SDF vector high-precision map has changed.

[0034] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0035] 1. The present invention automatically learns the complex features of SDF vector high-precision maps and SDF subgraphs through deep learning, eliminating the need to manually design feature extractors and improving the efficiency and accuracy of feature extraction.

[0036] 2. The shared spatial attention mechanism in this invention enhances the network's sensitivity to subtle changes in the SDF vector high-precision map by weighting important spatial regions. The shared parameters reduce the complexity of the model and the consumption of computing resources.

[0037] 3. The present invention can adapt to different map features and change patterns, has good generalization ability, and is suitable for SDF vector high-precision map change detection tasks of various types and scales. It also shows strong robustness to common problems such as image noise and illumination changes, ensuring the reliability of change detection results.

[0038] 4. The present invention can directly provide similarity measurement between SDF vector high-precision maps and SDF subgraphs, simplifying the change detection process. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 Schematic diagram of the logic flow of the method of the present invention.

[0040] Figure 2 Schematic diagram of the SDF subgraph preprocessing logic flow.

[0041] Figure 3 This is a schematic diagram of the logical flow of SDF vector high-precision map change detection.

[0042] Figure 4 Schematic diagram of the feature extraction layer.

[0043] Figure 5 Schematic diagram of the structure of the spatial attention layer. DETAILED DESCRIPTION

[0044] The present invention will be described in further detail below with reference to the embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0045] like Figures 1 to 5As shown, this embodiment provides a method for detecting changes in SDF vector high-precision maps based on an improved Siamese neural network, comprising the following steps:

[0046] S1: Take the surround view image as input and process it into a bird's-eye view. Train a semantic segmentation neural network using the bird's-eye view as input and output the semantic segmentation result of the bird's-eye view.

[0047] The inverse perspective transformation algorithm is used to process the surround view image into a bird's-eye view. The inverse perspective transformation algorithm formula is as follows:

[0048]

[0049] Where, Indicates pixel coordinates, u is the horizontal coordinate, v is the vertical coordinate, is the homogeneous form of pixel coordinates; Represents the coordinates of the two-dimensional plane in the Cartesian coordinate system after the inverse perspective transformation, where x is the vertical coordinate and y is the horizontal coordinate. is the homogeneous form of the coordinate; Represents the intrinsic parameter matrix of the camera, K -1 represents the inverse matrix of K; Is a 4×4 transformation matrix, representing the transformation from the world coordinate system to the camera coordinate system, T -1 is the inverse matrix of T; is a 3×3 rotation matrix, R ij is the rotation component of the rotation matrix, 1≤i,j≤3; is a 3×1 vector, indicating translation, t k is the translation component, 1≤k≤3; is a constant representing the normalized distance; the bird's-eye view image is used as input and YOLOP is used as the semantic segmentation neural network.

[0050] S2: The semantic segmentation results of the bird's-eye view image are layered according to semantic categories, and the layered bird's-eye view semantic map is output; for the layered bird's-eye view semantic map, an SDF subgraph is constructed for each semantic layer, the key points of the SDF subgraph are detected, and descriptors are constructed for the key points.

[0051] The specific steps of constructing the SDF subgraph are as follows: take the bird's-eye view semantic image as input, first filter outliers in the bird's-eye view semantic image, then convert it into a grayscale image and divide the image into grayscale images with different heights according to the grayscale value, and then apply the SDF algorithm to convert the grayscale images with different heights into SDF subgraphs. The formula of the SDF subgraph is defined as follows:

[0052] f(p)=sign(p)·min q∈S ||pq||

[0053] Where f(p) represents the SDF subgraph space, S represents a closed surface in the SDF subgraph space, p represents a point in the SDF subgraph space, q represents a point on S, the function min represents taking the minimum value in the set, and the function sign represents taking the positive or negative sign.

[0054] The specific steps of constructing the key points of the SDF subgraph are: taking the SDF subgraph as input, first apply Gaussian blur to the SDF subgraph to obtain S g , then S g Apply the first-order Sobel operator in the x and y directions respectively to obtain the approximate first-order derivative image S of the SDF subgraph dx and S dy , again to S dx and S dy Apply the first-order Sobel operator in the x and y directions respectively to obtain the second-order derivative image S of the SDF subgraph dxx 、S dxy 、S dyx and S dyy Then traverse all the coordinate points on the SDF subgraph and calculate the Hessian matrix and the determinant DoH, eigenvalue a and eigenvector of the Hessian matrix corresponding to each coordinate point The formula for calculating the determinant DoH of the Hessian matrix is ​​as follows:

[0055]

[0056] Then traverse DoH to record the coordinates of the local maximum points. The local maximum points are the key points of the SDF subgraph, called SDF key points. Finally, according to the key point coordinates and the corresponding eigenvalues ​​a and eigenvectors Classify the coordinate points as local maximum key points P max and the local minimum key point P min ;

[0057] The specific steps of constructing the descriptor are: traverse all SDF key points, construct a region of interest with the key point coordinates as the center and r as the radius for each key point, use a 36-bin histogram to count the gradient direction in the region of interest, record the main gradient direction, and then calculate the average SDF distance in the region of interest. Finally, use the main gradient direction, 36-bin histogram, average SDF distance and key point type as the descriptor of the current key point.

[0058] S3: Use key points and their descriptors to locate the SDF subgraph in the SDF vector HD map, record the positioning fit score, and retain the SDF subgraphs with a fit score higher than the threshold.

[0059] The specific steps of locating the SDF subgraph in the SDF vector high-precision map are as follows: the SDF vector high-precision map is stored in a KD-Tree data structure. First, the SDF subgraph coordinates p c As the initial value, find the 5 coordinates closest to the coordinate in the KD-Tree, then traverse the 5 nearest coordinates and calculate the fitting score between the SDF subgraph coordinate and the 5 nearest coordinates through the ICP algorithm. Take the nearest coordinate with the smallest fitting score as the optimal match of the SDF subgraph coordinate, and then calculate the transformation matrix T corresponding to the optimal match. b , through the formula p w =T b p c Calculate the coordinates p of the SDF subgraph in the SDF vector high-precision map w ,in, It is the coordinate of the world coordinate system in three-dimensional space, w represents the world coordinate system, is the coordinate of the camera coordinate system in three-dimensional space, c represents the camera coordinate system, is a 4×4 matrix representing the coordinate transformation from the camera coordinate system to the world coordinate system in three-dimensional space, where R ij represents the components of the rotation matrix, t k Represents the components of the translation vector, 1≤i,j,k≤3; for the best match and its fitting score, when the fitting score is higher than the fitting threshold, the best match is retained for the subsequent change detection process, and the fitting threshold is set to 0.32.

[0060] S4: For the SDF subgraphs whose fitting scores are higher than the threshold, select the SDF subgraphs adjacent to them in time and space from the SDF vector HD map, and adjust the coordinates of the current SDF subgraph using the ICP algorithm.

[0061] The specific steps for adjusting the coordinates of the SDF subgraphs with fitting scores higher than the threshold are as follows: for the SDF subgraphs with fitting scores higher than the threshold, use the SDF subgraph coordinates as the initial value, search for the SDF vector high-precision map segments within a physical distance of 30 meters from the coordinates from the SDF vector high-precision map in the forward and reverse directions respectively, and then use the ICP algorithm to fit the SDF subgraph and the SDF vector high-precision map segments to obtain the fitting score and transformation matrix T b ′, through the formula p′ w =T b ′p w Get the adjusted SDF subgraph coordinates p′ w ,in, Indicates the adjusted SDF subgraph coordinates, which are the coordinates in the world coordinate system in three-dimensional space. is a 4×4 transformation matrix, where R ij represents the components of the rotation matrix, t kRepresents the components of the translation vector, 1≤i,j,k≤3.

[0062] S5: For the SDF subgraph after coordinate adjustment, the changes in the SDF vector high-precision map are detected through the pre-trained improved Siamese neural network. The improvement of the Siamese neural network is to introduce a shared spatial attention mechanism, so that the network can better capture the structured areas in the SDF subgraph and the SDF vector high-precision map, so as to improve the accuracy of change detection; in the network inference stage, the relationship between the similarity d between the SDF subgraph and the SDF vector high-precision map and the change detection threshold T is used to judge whether the SDF vector high-precision map has changed. If d<T, the SDF vector high-precision map has not changed; if d≥T, the SDF vector high-precision map has changed.

[0063] The improved Siamese neural network introduces a shared spatial attention mechanism on the basis of the original Siamese neural network, allowing the network to pay more attention to the areas that are more critical for change detection in the SDF subgraph and SDF vector high-precision map. The shared spatial attention mechanism weights the key areas of the feature map, thereby enhancing the feature representation of these areas, thereby improving the sensitivity and accuracy of change detection.

[0064] The improved Siamese neural network consists of four layers: input layer, feature extraction layer, spatial attention layer, and feature difference measurement layer. The details are as follows:

[0065] ① Input layer: accepts SDF subgraph x1 and SDF vector high-precision map x2 as input, and the size of x1 and x2 are both 224×224;

[0066] ② Feature extraction layer: A custom convolutional neural network is used to extract feature maps F from each input. i′ ;

[0067] F i ′=MP(Conv 3×3 (MP(Conv 3×3 (MP(Conv 3×3 (x i′ ))))))

[0068] Where x i′ Represents an SDF subgraph or a high-precision map of an SDF vector, i′∈{1,2}; MP represents the maximum pooling layer, Conv 3×3Represents a 3×3 convolution kernel; three sets of convolution and pooling operations are used to extract features. The number of channels after convolution is 32, 64, and 128 respectively. A 3×3 convolution kernel is used, the stride is set to 1, and the padding is set to 1. The SDF subgraph contains information about the surface distance of the object. This information is highly structured in spatial distribution. The convolution layer can effectively capture this spatial structure information and extract features useful for change detection tasks. The pooling layer uses maximum pooling with a stride of 2 to obtain the original input feature maps F1 and F2.

[0069] ③ Spatial attention layer: Use the shared spatial attention mechanism to capture the key areas in the SDF subgraph and the SDF vector HD map, and weight the key areas:

[0070] S i′ =Sigmoid(Conv 1×1 (Relu(Conv 3×3 (F i′ ))))

[0071] Where, Conv 1×1 Represents a 1×1 convolution kernel, Relu represents a Relu activation function, and Sigmoid represents a Sigmoid activation function. First, a 3×3 convolution is used to capture local spatial features. The number of output channels is set to 8 to balance feature representation capability and computational efficiency. The step size is set to 1 and the padding is set to keep the same size as the input. Then, the Relu activation function is connected to enhance the network expression capability. The convolution layer for generating the spatial attention map is connected, with the convolution kernel size set to 1×1, the number of output channels set to 1, and the step size set to 1. Then, the Sigmoid activation layer is connected to obtain the spatial attention weight map S. i′ ;Use spatial attention weight map S i′ Original input feature map F i′ Weighted, get the weighted feature map, that is, the feature vector Where ⊙ represents element-wise multiplication;

[0072] ④ Feature difference measurement layer: use Euclidean distance to calculate two feature vectors F1 att 、 The difference, Where n represents the dimension of the feature vector, and They are feature maps F1 att and The value in the i″th dimension, i″∈{1,2,3,...,n};

[0073] Improve the Siamese neural network to use the contrast loss function during training. The contrast loss function is defined as: Where W is the network parameter, x1 and x2 are the SDF subgraph and SDF vector high-precision map respectively, and y′ is a label, which is 1 when x1 and x2 are similar and 0 when x1 and x2 are not similar. Indicates the similarity between x1 and x2. m is a boundary value used to distinguish whether x1 and x2 are similar. max(0,md) means taking the larger value between 0 and md.

[0074] The steps of using the improved Siamese neural network to detect changes in the SDF vector high-precision map are: In the change detection stage, first calculate the feature vector F1 of the SDF subgraph x1 and the SDF vector high-precision map x2 att and Then the similarity d is calculated and compared with the change detection threshold T to determine whether the SDF vector high-precision map x2 has changed. If d < T, it means that x1 and x2 are similar and the SDF vector high-precision map has not changed. If d ≥ T, it means that x1 and x2 are not similar and the SDF vector high-precision map has changed. T is set to 5.25.

[0075] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. The SDF vector high-precision map change detection method based on the improved Siamese neural network is characterized by: The following steps are involved: S1: Take the surround image as input and process it into a bird's-eye view. Train a semantic segmentation neural network using the bird's-eye view as input and output the semantic segmentation result of the bird's-eye view. S2: The semantic segmentation results of the bird's-eye view are layered according to semantic categories, and a layered bird's-eye view semantic map is output. For the layered bird's-eye view semantic map, an SDF subgraph is constructed for each semantic layer, key points of the SDF subgraph are detected, and descriptors are constructed for the key points. S3: Use key points and their descriptors to locate the SDF subgraph in the SDF vector HD map, record the positioning fit score, and retain the SDF subgraphs with a fit score higher than the threshold; S4: For the SDF subgraphs whose fitting scores are higher than the threshold, select the SDF subgraphs adjacent to them in time and space from the SDF vector HD map, and adjust the coordinates of the current SDF subgraph using the ICP algorithm; S5: For the SDF subgraph after coordinate adjustment, the changes in the SDF vector high-precision map are detected through the pre-trained improved Siamese neural network. The improvement of the Siamese neural network is to introduce a shared spatial attention mechanism, so that the network can better capture the structured areas in the SDF subgraph and the SDF vector high-precision map, so as to improve the accuracy of change detection; in the network inference stage, the relationship between the similarity d between the SDF subgraph and the SDF vector high-precision map and the change detection threshold T is used to judge whether the SDF vector high-precision map has changed. If d<T, the SDF vector high-precision map has not changed; if d≥T, the SDF vector high-precision map has changed.

2. The SDF vector high-precision map change detection method based on the improved Siamese neural network according to claim 1 is characterized in that: In step S1, the surround view image is processed into a bird's-eye view using an inverse perspective transformation algorithm. The inverse perspective transformation algorithm formula is expressed as follows: Where, Indicates pixel coordinates, u is the horizontal coordinate, v is the vertical coordinate, is the homogeneous form of pixel coordinates; Represents the coordinates of the two-dimensional plane in the Cartesian coordinate system after the inverse perspective transformation, where x is the vertical coordinate and y is the horizontal coordinate. is the homogeneous form of the coordinate; Represents the intrinsic parameter matrix of the camera, K -1 represents the inverse matrix of K; Is a 4×4 transformation matrix, representing the transformation from the world coordinate system to the camera coordinate system, T -1 is the inverse matrix of T; is a 3×3 rotation matrix, R ij is the rotation component of the rotation matrix, 1≤i,j≤3; is a 3×1 vector, indicating translation, t k is the translation component, 1≤k≤3; is a constant representing the normalized distance; the bird's-eye view image is used as input and YOLOP is used as the semantic segmentation neural network.

3. The SDF vector high-precision map change detection method based on the improved Siamese neural network according to claim 2 is characterized in that: In step S2, the specific steps of constructing the SDF subgraph are as follows: taking the bird's-eye view semantic image as input, first filtering outliers in the bird's-eye view semantic image, then converting it into a grayscale image and dividing the image into grayscale images with different heights according to the grayscale value, and then applying the SDF algorithm to convert the grayscale images with different heights into SDF subgraphs. The formula of the SDF subgraph is defined as follows: f(p)=sign(p)·min q∈S ||p-q|| Where f(p) represents the SDF subgraph space, S represents a closed surface in the SDF subgraph space, p represents a point in the SDF subgraph space, q represents a point on S, the function min represents taking the minimum value in the set, and the function sign represents taking the positive or negative sign.

4. The SDF vector high-precision map change detection method based on the improved Siamese neural network according to claim 3 is characterized in that: In step S2, the specific steps of constructing the key points of the SDF subgraph are: taking the SDF subgraph as input, first apply Gaussian blur to the SDF subgraph to obtain S g , then S g Apply the first-order Sobel operator in the x and y directions respectively to obtain the approximate first-order derivative image S of the SDF subgraph dx and S dy , again to S dx and S dy Apply the first-order Sobel operator in the x and y directions respectively to obtain the second-order derivative image S of the SDF subgraph dxx 、S dxy 、S dyx and S dyy Then traverse all the coordinate points on the SDF subgraph and calculate the Hessian matrix and the determinant DoH, eigenvalue a and eigenvector of the Hessian matrix corresponding to each coordinate point The formula for calculating the determinant DoH of the Hessian matrix is ​​as follows: Then traverse DoH to record the coordinates of the local maximum points. The local maximum points are the key points of the SDF subgraph, called SDF key points. Finally, according to the key point coordinates and the corresponding eigenvalues ​​a and eigenvectors Classify the coordinate points as local maximum key points P max and the local minimum key point P min ; The specific steps of constructing the descriptor are: traverse all SDF key points, construct a region of interest with the key point coordinates as the center and r as the radius for each key point, use a 36-bin histogram to count the gradient direction in the region of interest, record the main gradient direction, and then calculate the average SDF distance in the region of interest. Finally, use the main gradient direction, 36-bin histogram, average SDF distance and key point type as the descriptor of the current key point.

5. The SDF vector high-precision map change detection method based on the improved Siamese neural network according to claim 4 is characterized in that: In step S3, the specific steps of locating the SDF subgraph in the SDF vector high-precision map are as follows: the SDF vector high-precision map is stored in a KD-Tree data structure. First, the SDF subgraph coordinates p c As the initial value, find the 5 coordinates closest to the coordinate in the KD-Tree, then traverse the 5 nearest coordinates and calculate the fitting score between the SDF subgraph coordinate and the 5 nearest coordinates through the ICP algorithm. Take the nearest coordinate with the smallest fitting score as the optimal match of the SDF subgraph coordinate, and then calculate the transformation matrix T corresponding to the optimal match. b , through the formula p w =T b p c Calculate the coordinates p of the SDF subgraph in the SDF vector high-precision map w ,in, It is the coordinate of the world coordinate system in three-dimensional space, w represents the world coordinate system, is the coordinate of the camera coordinate system in three-dimensional space, c represents the camera coordinate system, It is a 4×4 matrix that represents the coordinate transformation from the camera coordinate system to the world coordinate system in three-dimensional space, where R ij represents the components of the rotation matrix, t k Represents the components of the translation vector, 1≤i,j,k≤3; for the best match and its fitting score, when the fitting score is higher than the fitting threshold, the best match is retained for subsequent change detection process.

6. The SDF vector high-precision map change detection method based on the improved Siamese neural network according to claim 5 is characterized in that: In step S4, the specific steps of adjusting the coordinates of the SDF subgraph with a fitting score higher than the threshold are as follows: for the SDF subgraph with a fitting score higher than the threshold, the SDF subgraph coordinates are used as the initial value, and the SDF vector high-precision map fragments within a physical distance of 30 meters from the coordinates are searched from the SDF vector high-precision map in the forward and reverse directions respectively, and then the SDF subgraph and the SDF vector high-precision map fragments are fitted using the ICP algorithm to obtain the fitting score and the transformation matrix T′. b , through the formula p′ w =T′ b p w Get the adjusted SDF subgraph coordinates p′ w ,in, Indicates the adjusted SDF subgraph coordinates, which are the coordinates in the world coordinate system in three-dimensional space. is a 4×4 transformation matrix, where R ij represents the components of the rotation matrix, t k Represents the components of the translation vector, 1≤i,j,k≤3.

7. The SDF vector high-precision map change detection method based on the improved Siamese neural network according to claim 6 is characterized in that: In step S5, the improved Siamese neural network introduces a shared spatial attention mechanism on the basis of the original Siamese neural network, so that the network can pay more attention to the areas that are more critical for change detection in the SDF subgraph and the SDF vector high-precision map. The shared spatial attention mechanism weights the key areas of the feature map, thereby enhancing the feature representation of these areas, thereby improving the sensitivity and accuracy of change detection; The improved Siamese neural network consists of four layers: input layer, feature extraction layer, spatial attention layer, and feature difference measurement layer. The details are as follows: ① Input layer: accepts SDF subgraph x1 and SDF vector high-precision map x2 as input, and the size of x1 and x2 are both 224×224; ② Feature extraction layer: A custom convolutional neural network is used to extract feature maps F from each input. i′ ; F i′ =MP(Conv 3×3 (MP(Conv 3×3 (MP(Conv 3×3 (x i′ )))))) Where x i′ Represents an SDF subgraph or a high-precision map of an SDF vector, i′∈{1,2}; MP represents the maximum pooling layer, Conv 3×3 Represents a 3×3 convolution kernel; three sets of convolution and pooling operations are used to extract features. The number of channels after convolution is 32, 64, and 128 respectively. A 3×3 convolution kernel is used, the stride is set to 1, and the padding is set to 1. The SDF subgraph contains information about the surface distance of the object. This information is highly structured in spatial distribution. The convolution layer can effectively capture this spatial structure information and extract features useful for change detection tasks. The pooling layer uses maximum pooling with a stride of 2 to obtain the original input feature maps F1 and F2. ③ Spatial attention layer: Use the shared spatial attention mechanism to capture the key areas in the SDF subgraph and the SDF vector HD map, and weight the key areas: S i′ =Sigmoid(Conv 1×1 (Relu(Conv 3×3 (F i′ )))) Where, Conv 1×1 Represents a 1×1 convolution kernel, Relu represents a Relu activation function, and Sigmoid represents a Sigmoid activation function. First, a 3×3 convolution is used to capture local spatial features. The number of output channels is set to 8 to balance feature representation capability and computational efficiency. The step size is set to 1 and the padding is set to keep the same size as the input. Then, the Relu activation function is connected to enhance the network expression capability. The convolution layer for generating the spatial attention map is connected, with the convolution kernel size set to 1×1, the number of output channels set to 1, and the step size set to 1. Then, the Sigmoid activation layer is connected to obtain the spatial attention weight map S. i′ ;Use spatial attention weight map S i′ Original input feature map F i′ Perform weighting to obtain a weighted feature map, i.e., the feature vector F i′ att =F i′ ⊙S i′ , where ⊙ represents element-wise multiplication; ④ Feature difference measurement layer: use Euclidean distance to calculate two feature vectors F1 att 、F2 att The difference, Where n represents the dimension of the feature vector, and They are feature maps F1 att and F2 att The value in the i″th dimension, i″∈{1,2,3,...,n}; Improve the Siamese neural network to use the contrast loss function during training. The contrast loss function is defined as: Where W is the network parameter, x1 and x2 are the SDF subgraph and SDF vector high-precision map respectively, y′ is a label, which is 1 when x1 and x2 are similar and 0 when x1 and x2 are not similar, d=d(F1 att ,F2 att ) represents the similarity between x1 and x2, m is a boundary value used to distinguish whether x1 and x2 are similar, and max(0,md) means taking the larger value between 0 and md.

8. The SDF vector high-precision map change detection method based on the improved Siamese neural network according to claim 7 is characterized in that: In step S5, the steps of using the improved Siamese neural network to detect changes in the SDF vector high-precision map are as follows: In the change detection stage, first calculate the feature vector F1 of the SDF subgraph x1 and the SDF vector high-precision map x2 att and F2 att , then calculate the similarity d and compare it with the change detection threshold T to determine whether the SDF vector high-precision map x2 has changed. If d < T, it means that x1 and x2 are similar, and the SDF vector high-precision map has not changed. If d ≥ T, it means that x1 and x2 are not similar, and the SDF vector high-precision map has changed.

Images