Stream query denoising method for vector high-definition map construction

Abstract

The invention relates to the technical field of automatic driving, and discloses a stream query denoising method for constructing a vector high-definition map, which comprises the following steps of: matching real information of a previous frame with real information of a current frame on an instance level, establishing a one-to-one correspondence relationship between an instance of the previous frame and an instance of the current frame, calculating a chamfering distance between each instance in the current frame and each instance in the previous frame to obtain a minimum distance value; dynamically adjusting the noise attenuation rate according to the transformation deviation of the real information of the previous frame in the current frame and the minimum distance value, and randomly adding noise for the curve through rectangular frame shifting and rectangular frame scaling based on the noise attenuation rate to obtain a curve with noise; the content embedding and the position embedding are fused and then input into a multi-layer perceptron to obtain noise query of the current frame; according to the method, the time consistency of map elements among different frames is improved through a stream query de-noising strategy.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a stream query denoising method for constructing vector high-definition maps. Background Art

[0002] Modern computer vision technology based on artificial intelligence and deep learning methods has made significant progress in the past 10 years. Today, it is widely used in image classification, object recognition, video analysis, image processing of autonomous vehicles, etc. As an emerging industry, autonomous driving is a hot topic in today's science and technology field and a key task in the field of computer vision. This technology hopes to achieve complete autonomous control of the car by the algorithm through various on-board peripherals, so as to complete the driving task safely and efficiently without the intervention of human drivers. Autonomous driving technology mainly includes perception module, prediction module and control module. In the perception module, according to the different modes of on-board peripherals, that is, the input data of the model, it can be divided into image-based, radar-based, and mixed-mode autonomous driving technology. At present, autonomous driving technology mainly relies on the Bird's-Eye-View (BEV) algorithm to convert the data collected by multiple sensors (such as cameras, radars, lidars, etc.) into images from a bird's-eye view, so that the autonomous driving system can more intuitively understand the surrounding roads, vehicles, pedestrians and other obstacles. It can be divided into perspective-based BEV and deep learning-based BEV. The transformer-based deep learning BEV technology can be divided into sparse query-based methods and dense query-based methods.

[0003] In recent years, there has been a great deal of interest in leveraging onboard sensors in autonomous vehicles to build vectorized local HD maps by connecting a decoder behind the BEV features. Compared to traditional offline SLAM methods, these online methods eliminate the need for a fleet of surveying vehicles and reduce manpower while maintaining the ability to adapt to new environments or potential map changes, thereby saving costs. These maps are critical for autonomous vehicles because they contain rich semantic information about the road, enabling effective navigation.

[0004] At present, researchers at home and abroad are committed to improving the performance of models on public datasets such as nuScenes and Argoverse2, and have made significant progress, such as BeMapNet, PivotNet, StreamMapNet, etc. However, most existing methods ignore the integration of temporal information and cannot fully utilize temporal information to improve the accuracy and efficiency of map construction. Even StreamMapNet only directly applies the stream paradigm to the construction of vectorized high-definition maps (HD-maps), failing to fully utilize the inherent potential of temporal information.

[0005] Therefore, it is of great theoretical significance and application value to carry out work on the construction of vectorized high-definition maps in autonomous driving scenarios and study how to perform more effective time modeling in the flow paradigm. Summary of the invention

[0006] In view of the problem of insufficient utilization of time series information in the existing vectorized HD map construction process, the present invention provides a stream query denoising (SQD) strategy for enhancing time modeling in HD map construction and improving the time consistency of map elements, thereby improving the construction performance of vectorized HD maps.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0008] A stream query denoising method for vector HD map construction is implemented based on the StreamMapNet model and includes the following steps:

[0009] Step 1: For the curves in the data frame used for map construction, encapsulate the curves by considering the minimum circumscribed rectangular box of the curves as instance representation; match the real information of the previous frame with the real information of the current frame at the instance level, establish a one-to-one correspondence between the instances of the previous frame and the instances of the current frame, and calculate the chamfer distance between each instance in the current frame and each instance in the previous frame to obtain the minimum distance value;

[0010] Step 2: dynamically adjust the noise attenuation rate according to the transformation deviation of the real information of the previous frame in the current frame and the minimum distance value, and randomly add noise η to the curve by shifting and scaling the rectangular frame based on the noise attenuation rate to obtain a noisy curve;

[0011] Step 3: Input the horizontal and vertical coordinates of each point in the point set of the instance corresponding to the noisy curve into the position encoding function respectively, and input the obtained position encoding results into the multi-layer perceptron to obtain the position embedding of the instance; secondly, set a learnable embedding for each category, namely, content embedding; fuse the content embedding with the position embedding and input it into the multi-layer perceptron to obtain the noise query of the current frame;

[0012] Step 4: Pass the multi-view image of the current frame through the BEV encoder to obtain the BEV features of the current frame, fuse the BEV features of the current frame with the BEV features of the previous frame to obtain the fused BEV features; input the fused BEV features, map query, noise query of the previous frame and noise query of the current frame into the Transformer decoder, and output a description of a set of points. As the prediction of the next frame map, M represents the number of instances of the next frame map, D mRepresents the description of the Mth instance in the next frame of the map.

[0013] Furthermore, in step 1, matching the real information of the previous frame with the real information of the current frame at the instance level, establishing a one-to-one correspondence between the instance of the previous frame and the instance of the current frame, and calculating the chamfer distance between each instance in the current frame and each instance in the previous frame to obtain the minimum distance value specifically includes:

[0014] make and They respectively represent the real information of the previous frame, i.e., the t-1 frame, and the real information of the current frame, i.e., the t-th frame; and denotes the coordinate sets of the i-th instance in the previous frame and the k-th instance in the current frame, respectively; m′ and n′ denote the number of instances in the previous frame and the current frame, respectively; the transformation matrix T between the coordinate systems of the previous frame and the current frame is used to transform Convert to the current frame and get the real information after conversion Represents the coordinate set of the i-th instance after the previous frame coordinate system is transformed; calculate the current frame The chamfer distance between each instance in the previous frame and each instance in the previous frame:

[0015]

[0016] Among them, CD Dir (S1, S2) is the chamfer distance in the direction from S1 to S2, CD is the bidirectional chamfer distance between S1 and S2; S1 and S2 are two point sets on the curve; p is a point in the point set S1, and q is a point in the point set S2; idx represents the index closest to the curve itself, and D is the minimum chamfer distance between the instance in the current frame and all instances in the previous frame, that is, the minimum distance value.

[0017] Furthermore, step 2 specifically includes:

[0018] According to the transformation deviation of the real information of the previous frame in the current frame, the noise attenuation rate is dynamically adjusted: the noise attenuation rate R of each instance dacay :

[0019]

[0020] Where γ is the predefined attenuation scale, w and h represent the width and height of the rectangular box;

[0021] When a noisy curve is obtained by shifting and scaling a rectangular frame, let the randomly added noise η = {Δx, Δy, Δw, Δh}, Δx and Δy are the horizontal coordinate Δx and vertical coordinate Δy of the random shift of the rectangular frame, Δw and Δh are the changes in the width and height of the rectangular frame, then the noise added example B ins Expressed as

[0022] B ins ={x,y,w,h}+{Δx,Δy,Δw,Δh}·R decay

[0023] Where x, y, w, and h are the horizontal and vertical coordinates of the center of the rectangular box corresponding to the original instance, and the width and height of the rectangular box respectively;

[0024] The noise adding example B ins This is a noisy curve.

[0025] Furthermore, step three specifically includes:

[0026] The point set of the mth noisy curve is Corresponding to the mth instance, n is the number of points in the point set, and the i-th point in the point set Position embedding for:

[0027]

[0028] For point The horizontal and vertical coordinates of , PE is the position encoding function, which is used to map floating point numbers to dimensional vector, MLP (pt) Indicates that dimensional vector projected onto Dimensional operation; D represents the number of dimensions after the coordinate point is positionally encoded;

[0029] After obtaining the position embedding of each point, the position embeddings of all points on the curve are concatenated and fused to obtain the instance position embedding Pos q :

[0030]

[0031] MLP (pos) Indicates the operation of fusing the position information of all points on the curve;

[0032] Through multi-layer perceptron MLP (fuse) The content embedding is fused with the position embedding to obtain the noisy query Q denoise :

[0033] Q denoise =MLP(fuse) (Concat(C q ,Pos q ));

[0034] C q Indicates that content is embedded.

[0035] Compared with the prior art, the beneficial technical effects of the present invention are:

[0036] 1. Improve temporal consistency. The stream query denoising strategy significantly improves the temporal consistency of map elements between different frames.

[0037] 2. Enhanced prediction accuracy. The accuracy of map element prediction is improved through adaptive time matching and dynamic query denoising.

[0038] 3. Extensive experiments on the nuScenes dataset and the Argoverse2 dataset show that our method outperforms existing methods in both close-range and long-range settings:

[0039] On the nuScenes dataset, the model SQD-MapNet used in this paper achieved an average precision (mAP) of 63.9 and 64.0 in the short-range and long-range evaluation settings, respectively, within 24 training cycles, which is more than 3.0 mAP higher than the previous SOTA method StreamMapNet. In addition, using the powerful V2-99 backbone network pre-trained on DD3D, SQD-MapNet achieved mAP of 74.0 and 75.2, refreshing the record of the nuScenes benchmark.

[0040] On the Argoverse2 dataset, SQD-MapNet outperforms StreamMapNet by about 2.0 mAP in evaluations within the 60×30 m and 100×50 m ranges, verifying the generalization ability and superiority of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 The decoding process of the StreamMapNet model and the improvement of the model SQD-MapNet adopted by the present invention are demonstrated.

[0042] Figure 2 It is a schematic diagram of the main processing process of the method of the present invention.

[0043] Figure 3 Schematic diagram of the strategy of adding noise to a curve in the present invention; wherein (a) represents the original curve, surrounded by a minimum circumscribed rectangular frame; (b) represents applying a random displacement to the rectangular frame; (c) represents scaling the rectangular frame, which simultaneously realizes an angular rotation and a scale transformation of the curve. DETAILED DESCRIPTION

[0044] A preferred embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0045] The present invention provides a stream query denoising method for constructing vector high-definition maps, which is implemented based on StreamMapNet. The method comprises: (1) designing curve noise and acquiring common query noise; and (2) denoising stream queries.

[0046] The specific technical solution of the present invention is described in two parts below.

[0047] 1. Curve noise design and acquisition of common query noise

[0048] Noise query is a query input obtained by adding a certain amount of noise to the ground truth (GT). These noises can include the center coordinate offset of the target, scale transformation, angle rotation, and random replacement of the category label. In DN-DETR, the noisy query is used in the denoising part of the Transformer decoder and processed in parallel with the original query (matching part). The output of the denoising part aims to reconstruct the noisy target, while the output of the matching part is responsible for bipartite graph matching with the real target. In this way, DN-DETR can accelerate the training process and improve performance by introducing noisy queries without changing the model structure. Therefore, noisy query is a key component in DN-DETR, which accelerates the training process and improves the performance of DETR by introducing noise and training the model to reconstruct these noisy targets. Noise training is a common data augmentation method. By adding noise during the training process, the model can learn more robust feature representations, thereby improving its performance and anti-interference ability in the actual environment. We introduce noise during the model training process by randomly perturbing the real instance and obtaining noisy queries according to the process of step 2.

[0049] Inspired by the query denoising strategy introduced by DN-DETR to alleviate the slow convergence speed of the DETR series and the tendency to converge to the local optimal solution, this paper proposes a new method called normal query denoising. First, this paper explores the noise strategy tailored for curves and introduces in detail the method of obtaining noise queries for HD map construction.

[0050] Specifically, this paper first explores a noise strategy tailored for curves. So far, no previous work has studied noise strategies tailored for curves in depth. The curve is encapsulated by considering its minimum bounding rectangle as an instance representation, and noise is added to the curve through three different curve noise strategies: line displacement, angle rotation, and scale transformation. The noisy curve is obtained, and the representation of the curve and the rectangular box is unified for the first time.

[0051] Assuming x and y represent the center coordinates of the rectangular box, and w and h represent the corresponding width and height, the noise strategy of the curve can be implemented in two different ways.

[0052] 1) Rectangular box shift: introduce random shift (Δx, Δy) into the rectangular box, and the random shift (Δx, Δy) is constrained and Among them, λ1 and λ2 are the maximum scales of the noise, thus achieving the effect of shifting.

[0053] 2) Rectangular frame scaling: The present invention randomly samples height and width in the intervals [(1-λ3)h, (1+λ3)h] and [(1-λ4)w, (1+λ4)w], respectively, where λ3 and λ4 represent the maximum scale of the noise, thereby achieving angle rotation and scale transformation at the same time.

[0054] Given a noisy curve, its categories and point sets are denoted as cls and Then for the point Its position embedding can be generated as follows:

[0055]

[0056] Where m is the index of the curve and n is the number of points that form the instance.

[0057] In the implementation of the present invention, the position encoding function PE maps the floating point number to dimensional vector, MLP (pt) The function will dimensional vector projected onto dimension.

[0058] Since the present invention uses deformable attention to interact the query with the information reference point, the present invention can use MLP (fuse) Fuse content information with location information to obtain denoised queries:

[0059] Q denoise =MLP (fuse) (Concat(C q ,Pos q )).

[0060] 2. Stream query denoising

[0061] The complexity of stream query prediction brings challenges to the current GT when adding noise. To solve this problem, the present invention designs two modules, namely adaptive time matching (ATM) and dynamic query noise (SQD), to effectively supplement and enhance the stream query denoising strategy.

[0062] In order to avoid the instability of bipartite graph matching, this paper proposes an adaptive time matching algorithm (ATM) to match historical real information with current real information at the instance level and establish an explicit one-to-one correspondence between the two. The real data is given by the nuScenes dataset and the Argoverse2 dataset. The data is in the form of a vector map, which is the raw data collected by the vehicle-mounted sensor through the vehicle-mounted system real-time localization and mapping (SLAM) technology.

[0063] make and Represents the GT of the previous frame and the current frame respectively. Due to the movement of the self, the transformation matrix T between the two coordinate systems is used to transform Convert to For the current frame For each curve in the frame, the present invention calculates the chamfer distance (CD) between it and each instance in the previous frame.

[0064]

[0065] CD(S1, S2) = CD Dir (S1, S2)+CD Dir (S2, S1);

[0066]

[0067] CD Dir is the Chamfer distance in the direction, CD is the bidirectional Chamfer distance; S1 and S2 are two sets of points on the curve; idx represents the index closest to the curve itself, and D is the minimum distance value.

[0068] After the basic facts of the previous frame are converted to the current frame based on self-motion, there will be a natural deviation between it and the basic facts in the current frame. To solve this problem, the present invention introduces dynamic query noise, which dynamically adjusts the noise attenuation rate according to the transformation deviation of the real information of the previous frame in the current frame, and optimizes the generation of noise queries. The noise attenuation rate of each instance can be calculated as follows:

[0069]

[0070] where γ is a predefined attenuation scale.

[0071] If the present invention assumes that the randomly added noise is η = {Δx, Δy, Δw, Δh}, the final example is expressed as

[0072] B ins ={x,y,w,h}+{Δx,Δy,Δw,Δh}·R decay ;

[0073] where {x, y, w, h} is the original instance. After obtaining the noise category and point set, the noise query can be derived.

[0074] Example

[0075] The present invention is applied to the field of autonomous driving technology and relates to a stream query denoising method, which is applied to high-definition map (HD-map) construction technology in the field of autonomous driving and is used to enhance the temporal modeling capability of the model in the process of vectorized HD map construction, specifically, to obtain noise queries when constructing vector HD maps.

[0076] Specifically, if Figure 2 As shown, firstly, the datasets in the field of autonomous driving are obtained: the nuScenes dataset and the Argoverse2 dataset, and training is performed on these two datasets. The nuScenes dataset is annotated at 2Hz, and each sample contains 6 synchronized cameras. The Argoverse2 dataset is annotated at 10Hz, and each frame contains 7 ring cameras and 2 stereo cameras. The present invention only uses images from the ring camera, and the frame rate of the dataset is unified to 2Hz.

[0077] The six images captured by the camera sensor at different angles at the same time are first passed through a BEV encoder to obtain their BEV features, and then a GRU gating unit is performed with the BEV features of the previous frame in the cache to obtain the fused BEV features. Secondly, the fused BEV features, map query, and noise query of the current frame and the previous frame are input into the Transformer decoder, and finally a description of a set of points is output. As the prediction of the next frame map, M represents the number of instances of the next frame map.

[0078] In particular, in order to solve the problem that after the ground truth (GT) of the previous frame is converted to the current frame, there will be a natural deviation between it and the GT in the current frame, the present invention introduces dynamic query noise, dynamically adjusts the noise attenuation rate according to the transformation deviation of the ground truth of the previous frame in the current frame, and optimizes the generation of noise queries. In order to avoid the instability of bipartite graph matching, the present invention proposes an adaptive time matching algorithm (ATM), which matches the historical real information with the current real information at the instance level and establishes an explicit one-to-one correspondence between the two. For the current frame For each curve in the image, the present invention calculates the chamfer distance (CD) between it and each instance in the previous frame, and matches the converted vector map with the vector map at the current time from the instance level, avoiding the discreteness of point-to-point matching in bipartite graph matching in StreamMapNet. It should be noted that, like DN-DETR, this denoising strategy is only used during training, and this model branch needs to be deleted during reasoning.

[0079] The model of this embodiment adopts an end-to-end training method. During the training process, the present invention adopts the same map loss function as the StreamMapNet model, including the classification loss L Focal , Line loss L line and translation loss L trans .

[0080]

[0081] Where λ1, λ2 and λ3 are hyperparameters. In addition, for the prediction results of denoised queries, the present invention uses the same type of classification loss and linear loss to construct

[0082]

[0083] in and is the linear loss for classification loss and denoising prediction, λ4 and λ5 are hyperparameters. Finally, the overall loss is defined as:

[0084]

[0085] In terms of implementation details, the present invention adopts ResNet-50 as the backbone network, and uses BEVFormer and a single encoder layer for BEV feature extraction. For small perception ranges, the size of the BEV feature map is 100×50m, and for larger ranges, the size of the BEV feature map is 200×100m. The streaming training strategy is consistent with the StreamMapNet model. Ordinary query denoising is used in the single-frame training stage. All models are trained for 24 rounds on the nuScenes dataset and 30 rounds on the Argoverse2 dataset. All experiments were performed on 8 NVIDIA Telsa V100 GPUs, and the batch size was set to 32.

[0086] In summary, the present invention aims at the problem of insufficient utilization of time series information in the existing vectorized HD map construction process, and provides a stream query denoising (SQD) strategy for enhancing temporal modeling in HD map construction. Specifically, the present invention proposes two core modules, a dynamic noise query module and an adaptive time matching module. The stream query denoising strategy improves the temporal consistency of map elements. Finally, a large number of experiments on nuScenes and Argoverse2 show that the stream query denoising strategy proposed in this paper greatly improves the performance of the stream model. The resulting SQD MapNet framework outperforms existing SOTA methods in all settings of close and long distances.

[0087] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention, and any reference numerals in the claims should not be regarded as limiting the claims involved.

[0088] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A stream query denoising method for vector high-definition map construction, based on the StreamMapNet model, characterized in that: The following steps are involved: Step 1: For the curves in the data frame used for map construction, encapsulate the curves by considering the minimum circumscribed rectangular box of the curves as instance representation; match the real information of the previous frame with the real information of the current frame at the instance level, establish a one-to-one correspondence between the instances of the previous frame and the instances of the current frame, and calculate the chamfer distance between each instance in the current frame and each instance in the previous frame to obtain the minimum distance value; Step 2: dynamically adjust the noise attenuation rate according to the transformation deviation of the real information of the previous frame in the current frame and the minimum distance value, and randomly add noise η to the curve by shifting and scaling the rectangular frame based on the noise attenuation rate to obtain a noisy curve; Step 3: Input the horizontal coordinate and vertical coordinate of each point in the point set of the instance corresponding to the noisy curve into the position encoding function respectively, and concatenate the position encoding results and input them into the multi-layer perceptron to obtain the position embedding of the instance; Secondly, a learnable embedding, i.e., content embedding, is set for each category. The content embedding is fused with the position embedding and then input into a multi-layer perceptron to obtain the noise query of the current frame. Step 4: Pass the multi-view image of the current frame through the BEV encoder to obtain the BEV features of the current frame, fuse the BEV features of the current frame with the BEV features of the previous frame to obtain the fused BEV features; input the fused BEV features, map query, noise query of the previous frame and noise query of the current frame into the Transformer decoder, and output a description of a set of points. As the prediction of the next frame map, M represents the number of instances of the next frame map, D m Represents the description of the Mth instance in the next frame of the map.

2. The stream query denoising method for vector high-definition map construction according to claim 1 is characterized in that: In step 1, the real information of the previous frame is matched with the real information of the current frame at the instance level, a one-to-one correspondence between the instance of the previous frame and the instance of the current frame is established, and the chamfer distance between each instance in the current frame and each instance in the previous frame is calculated to obtain the minimum distance value, specifically including: make and They respectively represent the real information of the previous frame, i.e., the t-1 frame, and the real information of the current frame, i.e., the t-th frame; and Respectively represent the coordinate sets of the i-th instance in the previous frame and the k-th instance in the current frame, m ′ and n ′ Represent the number of instances in the previous frame and the current frame respectively; use the transformation matrix T between the coordinate systems of the previous frame and the current frame to transform Convert to the current frame and get the real information after conversion Represents the coordinate set of the i-th instance after the previous frame coordinate system is transformed; calculate the current frame The chamfer distance between each instance in the previous frame and each instance in the previous frame: <h2 style=";text-align:left;direction:ltr">CD(S1,S2)=CD<h2 style=";text-align:left;direction:ltr"> Dir <h2 style=";text-align:left;direction:ltr"> (S1,S2)+CD<h2 style=";text-align:left;direction:ltr"> Dir <h2 style=";text-align:left;direction:ltr"> (S2,S1); Among them, CD Dir (S1, S2) is the chamfer distance in the direction from S1 to S2, CD is the bidirectional chamfer distance between S1 and S2; S1 and S2 are two point sets on the curve; p is a point in point set S1, and q is a point in point set S2; idx represents the index closest to the curve itself, and D is the minimum chamfer distance between the instance in the current frame and all instances in the previous frame, that is, the minimum distance value.

3. The stream query denoising method for vector high-definition map construction according to claim 1 is characterized in that: Step 2 specifically includes: According to the transformation deviation of the real information of the previous frame in the current frame, the noise attenuation rate is dynamically adjusted: the noise attenuation rate R of each instance dacay : Where γ is the predefined attenuation scale, w and h represent the width and height of the rectangular box; When a noisy curve is obtained by shifting and scaling a rectangular frame, let the randomly added noise η = {Δx, Δy, Δw, Δh}, Δx and Δy are the horizontal coordinates Δx and vertical coordinates Δy of the random shift of the rectangular frame, Δw and Δh are the changes in the width and height of the rectangular frame, then the noise added example B ins Expressed as B ins ={x,y,w,h}+{Δx,Δy,Δw,Δh}·R decay Where x, y, w, and h are the horizontal and vertical coordinates of the center of the rectangular box corresponding to the original instance, and the width and height of the rectangular box respectively; The noise adding example B ins This is a noisy curve.

4. The stream query denoising method for vector high-definition map construction according to claim 1 is characterized in that: Step three specifically includes: The point set of the mth noisy curve is Corresponding to the mth instance, n is the number of points in the point set, and the i-th point in the point set Position embedding for: For point The horizontal and vertical coordinates of , PE is the position encoding function, which is used to map floating point numbers to dimensional vector, MLP (pt) Indicates that dimensional vector projected onto Dimensional operation; D represents the number of dimensions after the coordinate point is positionally encoded; After obtaining the position embedding of each point, the position embeddings of all points on the curve are concatenated and fused to obtain the instance position embedding Pos q : MLP (pos) Indicates the operation of fusing the position information of all points on the curve; Through multi-layer perceptron MLP (fuse) The content embedding is fused with the position embedding to obtain the noisy query Q denoise : Q denoise =MLP (fuse) (Concat(C q ,Pos q )); C q Indicates that content is embedded.

Images