Cross-domain point cloud completion method and system based on sequence-level feature alignment

Abstract

The application discloses a cross-domain point cloud completion method and system based on sequence-level feature alignment, and belongs to the technical field of three-dimensional point cloud processing. The method comprises the following steps: acquiring point cloud sequences of a source domain and a target domain; extracting sequence-level feature representation from the point cloud sequences by a time sequence feature extractor through deformable time sequence attention; performing domain alignment on the sequence-level feature representation in an adversarial training manner through a domain discriminator, and simultaneously predicting complete point clouds by a complete point cloud completion decoder trained by complete labeled point clouds of the source domain; and performing self-supervised optimization on the completion results on the target domain by time sequence cyclic consistency constraint. The method further comprises feature decoupling of the sequence-level feature representation into domain-shared geometric features and domain-specific style features. The application replaces single-frame alignment with sequence-level feature alignment, enhances the robustness of domain alignment by using inter-frame time sequence information, and guarantees the motion continuity of the completion results on the target domain by time sequence consistency self-supervised constraint, thereby significantly improving the point cloud completion accuracy of cross-sensor configurations.

Description

Technical Field

[0001] This invention relates to the field of point cloud processing technology, and in particular to a cross-domain point cloud completion method and system based on sequence-level feature alignment. Background Technology

[0002] Point cloud completion aims to recover the complete geometry of an object or scene from partially observed 3D point clouds, and is a key preprocessing step in autonomous driving perception and robot navigation. However, existing point cloud completion methods typically assume that training and testing data come from the same sensor configuration (e.g., the same number of LiDAR lines and resolution) and scene distribution. When the model is deployed on different sensor platforms or application scenarios, completion performance degrades significantly due to domain shift. For example, when a completion model trained on 64-line LiDAR data is directly applied to 16-line LiDAR data, the chamfer distance (CD) of the completion result typically increases by 50% to 150%.

[0003] In existing technologies, cross-domain point cloud completion methods mainly fall into the following two categories:

[0004] The first category is based on single-frame feature alignment methods. These methods align the single-frame point cloud feature distributions of the source and target domains through adversarial training or minimizing the maximum mean difference (MMD). However, single-frame feature alignment has the following shortcomings: First, the spatial structure information in a single-frame point cloud is limited, lacking sufficient geometric constraints during feature alignment, resulting in aligned features containing significant domain-specific noise; second, supervised temporal motion information is not utilized, and the alignment process cannot distinguish between "domain shifts caused by sensor differences" and "temporal changes caused by object motion"; third, the temporal consistency of the completion results cannot be guaranteed, and geometric structural jumps may occur between adjacent frames.

[0005] The second category is based on cycle consistency constraints. This type of method utilizes CycleGAN or cycle consistency loss to perform inter-domain style transfer without paired data. However, this method is prone to losing geometric details during the transformation between the two domains and lacks direct completion supervision signals in the target domain, resulting in limited completion accuracy.

[0006] The information disclosed in this background section is intended only to enhance the understanding of the general background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0007] This invention provides a cross-domain point cloud completion method and system based on sequence-level feature alignment, thereby effectively solving the problems in the background technology.

[0008] To achieve the above objectives, the technical solution adopted by this invention is: a cross-domain point cloud completion method and system based on sequence-level feature alignment, comprising the following steps:

[0009] Obtain point cloud sequences of a source domain and a target domain. The source domain point cloud sequence contains complete point cloud annotations and corresponding partial point cloud data. The target domain point cloud sequence contains only partial point cloud data and has a domain offset from the source domain due to sensor configuration or scene distribution.

[0010] A temporal feature extractor extracts sequence-level features from the point cloud sequence. The temporal feature extractor takes the point cloud features of each frame in the sequence as input, models the consistency of spatial structure between frames and the continuity of motion between frames through a temporal attention mechanism, and outputs a sequence-level feature representation that integrates temporal information.

[0011] The sequence-level feature representations are aligned using an adversarial training method through a domain alignment discriminator, making the distribution of the sequence-level feature representations of the source domain and the target domain indistinguishable in the feature space. At the same time, the complete point cloud annotation of the source domain is used as a supervision signal to train the point cloud completion decoder to predict the complete point cloud coordinates from the sequence-level feature representations.

[0012] In the target domain, the output of the point cloud completion decoder is self-supervised and optimized by a temporal cyclic consistency constraint. The temporal cyclic consistency constraint requires that the optical flow distortion error between the completion results of adjacent frames be less than a preset threshold, so as to ensure the temporal continuity of the completion results.

[0013] Furthermore, the temporal feature extractor includes a frame-by-frame encoding module and a temporal aggregation module:

[0014] The frame-by-frame encoding module uses PointNet++ or Transformer as the backbone network to encode each frame of point cloud into a frame-level feature vector.

[0015] The temporal aggregation module takes the sequence of frame-level feature vectors of each frame as input, calculates the spatiotemporal correlation weights between frames through deformable temporal attention, and outputs the aggregated sequence-level feature representation.

[0016] Furthermore, the deformable temporal attention learns a set of offsets within the range of adjacent frames for each query frame, and adjusts the position of attention sampling with the offsets so that the attention weights are focused on the neighboring frame region that is consistent with the spatial structure of the query frame and has coherent motion.

[0017] Furthermore, the domain alignment discriminator takes the sequence-level feature representation as input and outputs a domain label prediction;

[0018] The adversarial training is implemented using a gradient inversion layer, which does not perform any transformations during forward propagation, but multiplies the gradient by a negative coefficient during backward propagation.

[0019] The point cloud completion decoder adopts a fractal decoding structure to predict the sparse skeleton, local details and global surface of the complete point cloud in a coarse-to-fine manner.

[0020] Furthermore, the timing cyclic consistency constraint is established as follows:

[0021] Completed with t frames and Frame completion results The scene flow estimation network predicts the motion vector for each point, and pairs the motion vectors together. By performing a forward mapping, we can obtain ,calculate and The chamfer distance between them is used as the cycle consistency loss.

[0022] Furthermore, the method also includes:

[0023] The sequence-level feature representation is decomposed into domain-shared geometric features and domain-specific style features;

[0024] The domain alignment discriminator performs adversarial domain alignment only on the geometric features shared by the domains, while the domain-specific style features are not subject to alignment constraints, so as to preserve the point cloud density and noise distribution information unique to each domain for completing the decoding.

[0025] Furthermore, the feature decomposition is achieved through a dual-channel encoder:

[0026] One path is a domain-shared encoder, trained using samples from all domains;

[0027] The other path is a domain-specific encoder, which is trained separately with samples from the current domain; the two output features are forced to be independent of each other through orthogonality constraints, and the orthogonality loss is the absolute value of the cosine similarity between the two output features.

[0028] Furthermore, the method also includes:

[0029] The point cloud completion decoder is trained by inputting partial point cloud data of the target domain. The completion results output by the decoder are evaluated for temporal consistency by the scene flow estimation network. The completion results with the highest evaluation scores in the top 10% are added to the training set as pseudo-labels. The point cloud completion decoder is iteratively fine-tuned in a semi-supervised manner.

[0030] The present invention also includes a cross-domain point cloud completion system based on sequence-level feature alignment, using the method described above, wherein the system comprises:

[0031] The acquisition unit is used to acquire point cloud sequences of the source domain and the target domain. The source domain point cloud sequence contains complete point cloud annotations and corresponding partial point cloud data, while the target domain point cloud sequence contains only partial point cloud data and has a domain offset from the source domain due to sensor configuration or scene distribution.

[0032] The temporal feature extraction unit is used to extract sequence-level features from the point cloud sequence using a temporal feature extractor. The temporal feature extractor takes the point cloud features of each frame in the sequence as input, models the consistency of the spatial structure between frames and the continuity of motion between frames through a temporal attention mechanism, and outputs a sequence-level feature representation that integrates temporal information.

[0033] The domain alignment unit is used to perform domain alignment on the sequence-level feature representation through a domain alignment discriminator in an adversarial training manner, so that the distribution of the sequence-level feature representations of the source domain and the target domain in the feature space is indistinguishable. At the same time, the complete point cloud annotation of the source domain is used as a supervision signal to train the point cloud completion decoder to predict the complete point cloud coordinates from the sequence-level feature representation.

[0034] An optimization unit is used to perform self-supervised optimization of the output of the point cloud completion decoder on the target domain with a temporal cyclic consistency constraint. The temporal cyclic consistency constraint requires that the optical flow distortion error between the completion results of adjacent frames be less than a preset threshold, so as to ensure the temporal continuity of the completion results.

[0035] The present invention also includes a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as described above.

[0036] The present invention also includes a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described above.

[0037] The beneficial effects of this invention are as follows: By replacing single-frame-level alignment with sequence-level feature alignment, the robustness of cross-domain point cloud completion is significantly improved. The temporal attention mechanism utilizes inter-frame spatial structure consistency and motion continuity as prior cues, enabling the domain alignment process to distinguish between domain offsets caused by sensor differences and temporal changes caused by object motion, thus avoiding domain-specific motion noise in the aligned features. The temporal cyclic consistency constraint ensures that the completion result maintains temporal motion continuity. After adding the temporal consistency constraint, the average optical flow distortion error between adjacent frame completion results is significantly reduced, and the inter-frame geometric jumps in the completion result are significantly reduced. By decoupling domain-shared geometric features from domain-specific style features, information loss during the domain alignment process is avoided, and the completion accuracy in the source domain remains stable. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention;

[0040] Figure 2 This is a schematic diagram of the system structure in Embodiment 1 of the present invention;

[0041] Figure 3 This is a flowchart of the method in Embodiment 2 of the present invention;

[0042] Figure 4 This is a schematic diagram of the computer device in Embodiment 3 of the present invention. Detailed Implementation

[0043] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0044] Example 1:

[0045] like Figure 1 As shown: A cross-domain point cloud completion method based on sequence-level feature alignment includes the following steps:

[0046] Obtain point cloud sequences of the source and target domains. The source domain point cloud sequence contains complete point cloud annotations and corresponding partial point cloud data, while the target domain point cloud sequence contains only partial point cloud data and has a domain offset from the source domain due to sensor configuration or scene distribution.

[0047] A temporal feature extractor is used to extract sequence-level features from point cloud sequences. The temporal feature extractor takes the point cloud features of each frame in the sequence as input, models the consistency of spatial structure and motion continuity between frames through a temporal attention mechanism, and outputs a sequence-level feature representation that integrates temporal information.

[0048] The domain alignment discriminator is used to perform domain alignment on the sequence-level feature representation in an adversarial training manner, so that the distribution of the sequence-level feature representations of the source domain and the target domain in the feature space is indistinguishable. At the same time, the complete point cloud annotation of the source domain is used as a supervision signal to train the point cloud completion decoder to predict the complete point cloud coordinates from the sequence-level feature representation.

[0049] In the target domain, the output of the point cloud completion decoder is self-supervised by temporal cyclic consistency constraints. The temporal cyclic consistency constraints require that the optical flow distortion error between the completion results of adjacent frames be less than a preset threshold, so as to ensure the temporal continuity of the completion results.

[0050] Replacing single-frame alignment with sequence-level feature alignment significantly improves the robustness of cross-domain point cloud completion. The temporal attention mechanism utilizes inter-frame spatial structure consistency and motion continuity as prior cues, enabling the domain alignment process to distinguish between domain offsets caused by sensor differences and temporal variations caused by object motion, thus avoiding domain-specific motion noise in the aligned features. Temporal cyclic consistency constraints maintain temporal motion continuity in the completion results. After incorporating these constraints, the average optical flow distortion error between adjacent frame completion results is significantly reduced, and inter-frame geometric jumps in the completion results are significantly decreased. By decoupling domain-shared geometric features from domain-specific style features, information loss during domain alignment is avoided, and the completion accuracy in the source domain remains stable.

[0051] In this embodiment, the temporal feature extractor includes a frame-by-frame encoding module and a temporal aggregation module:

[0052] The frame-by-frame encoding module uses PointNet++ or Transformer as the backbone network to encode each frame of point cloud into a frame-level feature vector;

[0053] The temporal aggregation module takes the sequence of frame-level feature vectors of each frame as input, calculates the spatiotemporal correlation weights between frames through deformable temporal attention, and outputs the aggregated sequence-level feature representation.

[0054] Among them, deformable temporal attention learns a set of offsets in the range of adjacent frames for each query frame, and adjusts the position of attention sampling with the offsets so that the attention weights are focused on the neighboring frame region that is consistent with the spatial structure of the query frame and has coherent motion.

[0055] As a preferred embodiment of the above, the domain alignment discriminator takes sequence-level feature representation as input and outputs domain label prediction;

[0056] Adversarial training is implemented using a gradient inversion layer, which does not perform any transformations during forward propagation, but multiplies the gradient by a negative coefficient during backward propagation.

[0057] The point cloud completion decoder adopts a fractal decoding structure to predict the sparse skeleton, local details and global surface of the complete point cloud in a coarse-to-fine manner.

[0058] The method for establishing time-series cyclic consistency constraints is as follows:

[0059] Completed with t frames and Frame completion results The scene flow estimation network predicts the motion vector of each point, and the motion vector pairs By performing a forward mapping, we can obtain ,calculate and The chamfer distance between them is used as the cycle consistency loss.

[0060] As a preferred embodiment of the above, the method further includes:

[0061] Sequence-level feature representations are decomposed into domain-shared geometric features and domain-specific style features;

[0062] The domain alignment discriminator performs adversarial domain alignment only on domain-shared geometric features. Domain-specific style features are not subject to alignment constraints, so as to preserve the domain-specific point cloud density and noise distribution information for completing the decoding.

[0063] In this embodiment, feature decomposition is achieved through a dual-channel encoder:

[0064] One path is a domain-shared encoder, trained using samples from all domains;

[0065] The other path is a domain-specific encoder, which is trained separately with samples from the current domain; the two output features are forced to be independent of each other through orthogonality constraints, and the orthogonality loss is the absolute value of the cosine similarity between the two output features.

[0066] As a preferred embodiment of the above, the method further includes:

[0067] The point cloud completion decoder is trained by inputting partial point cloud data of the target domain. The completion results output by the decoder are evaluated for temporal consistency by passing them through a scene flow estimation network. The completion results with the highest evaluation scores in the top 10% are added to the training set as pseudo-labels. The point cloud completion decoder is then fine-tuned iteratively in a semi-supervised manner.

[0068] like Figure 2 As shown, this embodiment also includes a cross-domain point cloud completion system based on sequence-level feature alignment. Using the method described above, the system includes:

[0069] The acquisition unit is used to acquire point cloud sequences of the source domain and the target domain. The source domain point cloud sequence contains complete point cloud annotations and corresponding partial point cloud data, while the target domain point cloud sequence contains only partial point cloud data and has a domain offset from the source domain due to sensor configuration or scene distribution.

[0070] The temporal feature extraction unit is used to extract sequence-level features from the point cloud sequence using a temporal feature extractor. The temporal feature extractor takes the point cloud features of each frame in the sequence as input, models the consistency of the spatial structure between frames and the continuity of motion between frames through a temporal attention mechanism, and outputs a sequence-level feature representation that integrates temporal information.

[0071] The domain alignment unit is used to perform domain alignment on the sequence-level feature representation through the domain alignment discriminator in an adversarial training manner, so that the distribution of the sequence-level feature representations of the source domain and the target domain in the feature space is indistinguishable. At the same time, the complete point cloud annotation of the source domain is used as a supervision signal to train the point cloud completion decoder to predict the complete point cloud coordinates from the sequence-level feature representation.

[0072] The optimization unit is used to perform self-supervised optimization of the output of the point cloud completion decoder in the target domain with temporal cyclic consistency constraints. The temporal cyclic consistency constraints require that the optical flow distortion error between the completion results of adjacent frames be less than a preset threshold, so as to ensure the temporal continuity of the completion results.

[0073] Example 2:

[0074] like Figure 3 As shown, this embodiment includes a cross-domain point cloud completion method based on sequence feature alignment, specifically including:

[0075] I. Dataset and Task Setup

[0076] This embodiment uses the SemanticKITTI dataset as the source domain (64-line Velodyne HDL-64E LiDAR, complete scene point cloud and frame-by-frame partial point cloud), and a self-acquired low-beam dataset as the target domain (16-line Velodyne VLP-16 LiDAR, only partial point cloud, no complete annotation). The source domain contains 22 sequences totaling 43,000 frames, and the target domain contains 6 sequences totaling 5,200 frames. The goal of cross-domain completion is to complete the sparse 16-line point cloud into a dense and complete point cloud equivalent to the 64-line LiDAR.

[0077] II. Temporal Feature Extraction

[0078] The frame-by-frame encoding module employs the PointNet++ backbone network, using the SA layer to extract hierarchical features from the point cloud of each frame, and outputs a 256-dimensional frame-level feature vector after global max pooling. The temporal aggregation module takes a sequence of feature vectors from eight consecutive frames (8×256 dimensions) as input and applies deformable temporal attention (4-head attention, 256 hidden layer dimensions) to calculate spatiotemporal relevance weights. The deformable attention learns a 2D offset (4 offset positions) within ±2 frames for each query frame, focusing attention on spatial regions with consistent structure and coherent motion. The sequence-level features are represented as a 512-dimensional vector after temporal pooling from the 8×256-dimensional vector.

[0079] III. Field Alignment and Completion Decoding

[0080] The domain discriminator consists of a 3-layer fully connected network (512→256→128→1), taking sequence-level feature representations as input and outputting domain label predictions. The gradient inversion layer coefficients initially increase linearly from 0 to 0.5. The completion decoder employs a 3-level fractal decoding: the first level predicts 64 sparse skeleton points, the second level predicts 256 local detail points conditioned on the skeleton points, and the third level predicts 1024 complete points conditioned on the local details. Bevel distance and bulldozer distance are used as supervised losses in the source domain. In the 64-line → 16-line domain alignment ablation experiment, the target domain CD = 0.082 without domain alignment, 0.058 with single-frame alignment, and 0.042 with sequence-level alignment.

[0081] IV. Cyclic Consistency Constraints

[0082] Using FlowNet3D as the scene flow estimation network, the completion results for frames t and t+1 are... and Calculate the 3D motion vector for each point. Pair the motion vectors together. By performing a forward mapping, we can obtain ,calculate and The chamfer distance between the left and right corners is used as the cycle consistency loss. After adding timing constraints, the average timing jitter of the completion result decreases from 0.021 to 0.009.

[0083] V. Feature Decoupling Experiment

[0084] The dual encoders output 256-dimensional domain-shared features and 256-dimensional domain-specific features, respectively. The orthogonality constraint loss weight is set to 0.05. The domain alignment discriminator performs alignment only on the domain-shared features. After feature decoupling, the target domain completion CD decreases from 0.044 to 0.042 (an improvement of 5.2%), verifying the contribution of preserving domain-specific information to completion accuracy.

[0085] VI. Overall Effect Comparison

[0086] method CD EMD Timing jitter No domain alignment (source domain training → target domain direct testing) 0.082 0.047 0.028 Single-frame adversarial domain alignment 0.058 0.033 0.024 Single-frame alignment + timing consistency constraints 0.052 0.030 0.012 This invention (sequence-level alignment + temporal constraints + feature decoupling) 0.042 0.025 0.009

[0087] This embodiment has the following beneficial effects:

[0088] First, replacing single-frame alignment with sequence-level feature alignment significantly improves the robustness of cross-domain point cloud completion. In the cross-domain completion test of 64-line to 16-line LiDAR, the chamfer distance of sequence-level alignment is 0.042, which is 27.6% lower than that of single-frame alignment (0.058). The advantage of sequence-level alignment is that the temporal attention mechanism uses the consistency of inter-frame spatial structure and motion continuity as prior cues, enabling the domain alignment process to distinguish between "domain offset caused by sensor differences" and "temporal changes caused by object motion," thus avoiding the inclusion of domain-specific motion noise in the aligned features.

[0089] Second, the temporal cyclic consistency constraint ensures that the completion results maintain temporal continuity. After adding the temporal consistency constraint, the average optical flow distortion error between adjacent frame completion results decreased from 0.021 to 0.009 (a reduction of 57.1%), and the inter-frame geometric jumps of the completion results were significantly reduced.

[0090] Third, by decoupling domain-shared geometric features from domain-specific style features, information loss during domain alignment is avoided. Ablation experiments show that after adding the feature decoupling mechanism, the completion accuracy in the target domain is improved by 5.2% (CD decreases from 0.044 to 0.042), while the completion accuracy in the source domain remains stable.

[0091] Example 3:

[0092] Please see Figure 4 The diagram shows a structural schematic of a computer device provided in an embodiment of this application. An embodiment of this application provides a computer device 400, including a processor 410 and a memory 420. The memory 420 stores a computer program executable by the processor 410. When the computer program is executed by the processor 410, it performs the method described above.

[0093] This application embodiment also provides a storage medium 430, on which a computer program is stored, and the computer program is executed by a processor 410 to perform the above method.

[0094] The storage medium 430 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0095] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. "A plurality of" means two or more, unless otherwise explicitly specified.

[0096] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0097] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0098] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0099] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0100] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0101] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0102] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A cross-domain point cloud completion method based on sequence-level feature alignment, characterized in that, Includes the following steps: Obtain point cloud sequences of a source domain and a target domain. The source domain point cloud sequence contains complete point cloud annotations and corresponding partial point cloud data. The target domain point cloud sequence contains only partial point cloud data and has a domain offset from the source domain due to sensor configuration or scene distribution. A temporal feature extractor extracts sequence-level features from the point cloud sequence. The temporal feature extractor takes the point cloud features of each frame in the sequence as input, models the consistency of spatial structure between frames and the continuity of motion between frames through a temporal attention mechanism, and outputs a sequence-level feature representation that integrates temporal information. The sequence-level feature representations are aligned using an adversarial training method through a domain alignment discriminator, making the distribution of the sequence-level feature representations of the source domain and the target domain indistinguishable in the feature space. At the same time, the complete point cloud annotation of the source domain is used as a supervision signal to train the point cloud completion decoder to predict the complete point cloud coordinates from the sequence-level feature representations. In the target domain, the output of the point cloud completion decoder is self-supervised and optimized by a temporal cyclic consistency constraint. The temporal cyclic consistency constraint requires that the optical flow distortion error between the completion results of adjacent frames be less than a preset threshold, so as to ensure the temporal continuity of the completion results.

2. The method according to claim 1, characterized in that, The temporal feature extractor includes a frame-by-frame encoding module and a temporal aggregation module: The frame-by-frame encoding module uses PointNet++ or Transformer as the backbone network to encode each frame of point cloud into a frame-level feature vector. The temporal aggregation module takes the sequence of frame-level feature vectors of each frame as input, calculates the spatiotemporal correlation weights between frames through deformable temporal attention, and outputs the aggregated sequence-level feature representation.

3. The method according to claim 2, characterized in that, The deformable temporal attention learns a set of offsets within the range of adjacent frames for each query frame, and adjusts the position of attention sampling with the offsets so that the attention weights are focused on the neighboring frame region that is consistent with the spatial structure of the query frame and has coherent motion.

4. The method according to claim 1, characterized in that, The domain alignment discriminator takes the sequence-level feature representation as input and outputs a domain label prediction. The adversarial training is implemented using a gradient inversion layer, which does not perform any transformations during forward propagation, but multiplies the gradient by a negative coefficient during backward propagation. The point cloud completion decoder adopts a fractal decoding structure to predict the sparse skeleton, local details and global surface of the complete point cloud in a coarse-to-fine manner.

5. The method according to claim 1, characterized in that, The method for establishing the time-series cyclic consistency constraint is as follows: Completed with t frames and Frame completion results The scene flow estimation network predicts the motion vector of each point, and the motion vector pairs... By performing a forward mapping, we can obtain ,calculate and The chamfer distance between them is used as the cycle consistency loss.

6. The method according to claim 1, characterized in that, The method further includes: The sequence-level feature representation is decomposed into domain-shared geometric features and domain-specific style features; The domain alignment discriminator performs adversarial domain alignment only on the geometric features shared by the domains, while the domain-specific style features are not subject to alignment constraints, so as to preserve the point cloud density and noise distribution information unique to each domain for completing the decoding.

7. The method according to claim 6, characterized in that, The feature decomposition is achieved through a dual-channel encoder: One path is a domain-shared encoder, trained using samples from all domains; The other path is a domain-specific encoder, which is trained separately with samples from the current domain; the two output features are forced to be independent of each other through orthogonality constraints, and the orthogonality loss is the absolute value of the cosine similarity between the two output features.

8. The method according to claim 1, characterized in that, The method further includes: The point cloud completion decoder is trained by inputting partial point cloud data of the target domain. The completion results output by the decoder are evaluated for temporal consistency by the scene flow estimation network. The completion results with the highest evaluation scores in the top 10% are added to the training set as pseudo-labels. The point cloud completion decoder is iteratively fine-tuned in a semi-supervised manner.

9. A cross-domain point cloud completion system based on sequence-level feature alignment, characterized in that, Using the method of any one of claims 1 to 8, the system comprises: The acquisition unit is used to acquire point cloud sequences of the source domain and the target domain. The source domain point cloud sequence contains complete point cloud annotations and corresponding partial point cloud data, while the target domain point cloud sequence contains only partial point cloud data and has a domain offset from the source domain due to sensor configuration or scene distribution. The temporal feature extraction unit is used to extract sequence-level features from the point cloud sequence using a temporal feature extractor. The temporal feature extractor takes the point cloud features of each frame in the sequence as input, models the consistency of the spatial structure between frames and the continuity of motion between frames through a temporal attention mechanism, and outputs a sequence-level feature representation that integrates temporal information. The domain alignment unit is used to perform domain alignment on the sequence-level feature representation through a domain alignment discriminator in an adversarial training manner, so that the distribution of the sequence-level feature representations of the source domain and the target domain in the feature space is indistinguishable. At the same time, the complete point cloud annotation of the source domain is used as a supervision signal to train the point cloud completion decoder to predict the complete point cloud coordinates from the sequence-level feature representation. An optimization unit is used to perform self-supervised optimization of the output of the point cloud completion decoder on the target domain with a temporal cyclic consistency constraint. The temporal cyclic consistency constraint requires that the optical flow distortion error between the completion results of adjacent frames be less than a preset threshold, so as to ensure the temporal continuity of the completion results.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1-8.

11. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-8.

Images