Three-dimensional visual point cloud completion method and device based on human brain nuclear magnetic cognitive distillation

Abstract

The application provides a three-dimensional visual point cloud completion method and device based on human brain nuclear magnetic cognitive distillation, relates to the technical field of three-dimensional point cloud completion and brain cognitive signal fusion, and the method comprises the following steps: acquiring a three-dimensional incomplete point cloud to be completed; inputting the three-dimensional incomplete point cloud into a point cloud completion model to obtain a completed three-dimensional point cloud; wherein a geometric encoder in the model is used for extracting geometric features of the three-dimensional incomplete point cloud to obtain point cloud features; a global semantic bottleneck layer in the model is used for extracting global semantics of the point cloud features to obtain global features; and a reconstruction decoder in the model is used for fine reconstruction of the global features to output the completed point cloud. In this way, by internalizing the human brain three-dimensional visual cognitive prior into the point cloud completion model through hierarchical distillation, the model can have robust structure reasoning capability without relying on additional prior input in the reasoning stage, and the completion generalization in the severe occlusion and cross-domain scene is significantly improved.

Description

Technical Field

[0001] This application relates to the field of 3D point cloud completion and brain cognitive signal fusion technology, and in particular to a 3D visual point cloud completion method and device based on human brain nuclear magnetic resonance cognitive distillation. Background Technology

[0002] 3D point cloud completion refers to the task of recovering a complete 3D shape from partially observed point cloud data. It is a fundamental technical step in applications such as autonomous driving, robot navigation, and augmented reality. Most 3D point cloud completion methods in related technologies are based on a data-driven paradigm, completing the process by learning the geometric mapping relationship between partial observations and the complete shape from large-scale point cloud datasets.

[0003] However, under conditions of severe occlusion or highly sparse input point clouds, the geometric information provided by some observations is extremely limited. Existing data-driven methods, due to their over-reliance on the inherent statistical geometric correlations in the training dataset, often fail to infer semantically reasonable and structurally complete missing parts when faced with regions of significant geometric ambiguity, resulting in blurry, distorted, or structurally incomplete completion results.

[0004] Therefore, how to internalize high-level structural priors into the point cloud completion model to improve its robustness and generalization, without requiring additional prior inputs during the inference stage and while maintaining high computational efficiency, is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] The purpose of this application is to provide a method and device for three-dimensional visual point cloud completion based on human brain nuclear magnetic resonance cognitive distillation. By internalizing the human brain's three-dimensional visual cognitive priors into the point cloud completion model through hierarchical distillation, the model can have robust structural reasoning ability without relying on additional prior inputs during the reasoning stage. This significantly improves the completion generalization in severely occluded and cross-domain scenarios while maintaining real-time reasoning efficiency.

[0006] This application provides a method for 3D visual point cloud completion based on human brain nuclear magnetic resonance cognitive distillation, including: A three-dimensional incomplete point cloud to be completed is obtained; the three-dimensional incomplete point cloud is input into a point cloud completion model to obtain a completed three-dimensional point cloud; wherein, the geometric encoder in the point cloud completion model is used to extract geometric features from the three-dimensional incomplete point cloud to obtain point cloud features; the global semantic bottleneck layer in the point cloud completion model is used to extract global semantics from the point cloud features to obtain global features; the reconstruction decoder in the point cloud completion model is used to perform fine reconstruction of the global features and output the completed point cloud.

[0007] Optionally, the training steps of the point cloud completion model include: constructing a teacher network and an initial student network; the teacher network includes a cognitive fusion module, and the initial student network is a network with the same architecture after removing the cognitive fusion module; acquiring training data, the training data including incomplete point cloud samples and corresponding functional magnetic resonance imaging (fMRI) signals; inputting the training data into the teacher network, extracting cognitive feature vectors from the fMRI signals using a spatiotemporal cognitive encoder, and fusing the cognitive feature vectors with the geometric features of the incomplete point cloud samples at multiple scales using the cognitive fusion module, and training the teacher network until convergence; freezing the parameters of the teacher network, inputting the incomplete point cloud samples into the initial student network, performing hierarchical alignment based on the intermediate representations of the teacher network and the initial student network at multiple levels, calculating distillation loss, and training the initial student network based on the distillation loss to obtain the point cloud completion model.

[0008] Optionally, the step of extracting the cognitive feature vector from the functional magnetic resonance imaging (fMRI) signal using a spatiotemporal cognitive encoder includes: preprocessing the fMRI signal to obtain a spatiotemporal tensor form of a cortical plane map; extracting spatial features from each frame of the cortical plane map using a spatial extractor to obtain a spatial embedding sequence; performing temporal dependency modeling on the spatial embedding sequence using a long short-term memory (LSTM) network to obtain a context-aware feature sequence; and using a global aggregation module to perform attention-weighted aggregation of the context-aware feature sequence using a learnable query vector to output the cognitive feature vector.

[0009] Optionally, the step of multi-scale fusion of the cognitive feature vector with the geometric features of the incomplete point cloud sample through the cognitive fusion module includes: at the front end of the geometric encoder, adaptively weighting the geometric features and the cognitive feature vector through a gated fusion block to obtain modulated input features; at the global semantic bottleneck layer, mapping the cognitive feature vector to a first affine parameter through a first cognitive feature linear modulation module, and performing feature modulation on the modulated input features based on the first affine parameter to obtain semantically calibrated global features; at the reconstruction decoder end, mapping the cognitive feature vector to a second affine parameter through a second cognitive feature linear modulation module, and performing feature modulation on the semantically calibrated global features based on the second affine parameter to obtain reconstructed features after detail restoration.

[0010] Optionally, the step of adaptively weighting and fusing the geometric features and the cognitive feature vector through a gating fusion block to obtain the modulated input features includes: jointly processing the geometric features and the cognitive feature vector through a multilayer perceptron to generate a learnable gating mask; and performing a weighted summation of the geometric features and the cognitive feature vector based on the gating mask and a preset scaling factor to obtain the modulated input features.

[0011] Optionally, the feature modulation based on the first affine parameter, or the feature modulation based on the second affine parameter, includes: mapping the cognitive feature vector to an affine parameter containing a scaling vector and a bias vector through a target cognitive feature linear modulation module; the target cognitive feature linear modulation module is either the first cognitive feature linear modulation module or the second cognitive feature linear modulation module; scaling the feature to be modulated in the current layer using the scaling vector, and superimposing the scaled feature with the bias vector and the feature to be modulated in the current layer; wherein, when performing feature modulation based on the first affine parameter, the feature to be modulated in the current layer is the modulated input feature; when performing feature modulation based on the second affine parameter, the feature to be modulated in the current layer is the semantically calibrated global feature.

[0012] Optionally, the step of performing hierarchical alignment based on intermediate representations of the teacher network and the initial student network at multiple levels and calculating distillation loss includes: for spatially structured features at the input level and fine reconstruction level, obtaining a spatial attention map by normalizing activation amplitude through temperature scaling, and calculating the divergence loss of the spatial attention maps of the teacher network and the initial student network as the attention transfer loss; for compact global features at the global semantic level, calculating the mean square error between the global features of the teacher network and the initial student network as the direct feature alignment loss; and combining the attention transfer loss, the direct feature alignment loss, and the point cloud reconstruction task loss to determine the distillation loss.

[0013] This application also provides a three-dimensional visual point cloud completion device based on human brain nuclear magnetic resonance cognitive distillation, including: The acquisition module is used to acquire the incomplete 3D point cloud to be completed; the point cloud completion module is used to input the incomplete 3D point cloud into the point cloud completion model to obtain the completed 3D point cloud; wherein, the geometric encoder in the point cloud completion model is used to extract geometric features from the incomplete 3D point cloud to obtain point cloud features; the global semantic bottleneck layer in the point cloud completion model is used to extract global semantics from the point cloud features to obtain global features; the reconstruction decoder in the point cloud completion model is used to perform fine reconstruction of the global features and output the completed point cloud.

[0014] Optionally, the device further includes: a model building module and a model training module; the model building module is used to build a teacher network and an initial student network; the teacher network includes a cognitive fusion module, and the initial student network is a network with the same architecture after removing the cognitive fusion module; the acquisition module is further used to acquire training data; the training data includes incomplete point cloud samples and corresponding functional magnetic resonance imaging (fMRI) signals; the model training module is specifically used to input the training data into the teacher network, extract cognitive feature vectors from the fMRI signals through a spatiotemporal cognitive encoder, and perform multi-scale fusion of the cognitive feature vectors with the geometric features of the incomplete point cloud samples through the cognitive fusion module, training the teacher network until convergence; the model training module is further used to freeze the parameters of the teacher network, input the incomplete point cloud samples into the initial student network, perform hierarchical alignment based on the intermediate representations of the teacher network and the initial student network at multiple levels, calculate distillation loss, and train the initial student network based on the distillation loss to obtain the point cloud completion model.

[0015] Optionally, the device further includes: a feature extraction module; the feature extraction module is used to preprocess the functional magnetic resonance imaging signal to obtain a spatiotemporal tensor form of a cortical plane mapping map; the feature extraction module is also used to extract spatial features from each frame of the cortical plane mapping map using a spatial extractor to obtain a spatial embedding sequence; the feature extraction module is also used to perform temporal dependency modeling on the spatial embedding sequence using a long short-term memory network to obtain a context-aware feature sequence; the feature extraction module is also used to perform attention-weighted aggregation of the context-aware feature sequence using a learnable query vector through a global aggregation module to output the cognitive feature vector.

[0016] Optionally, the apparatus further includes: a feature fusion module; the feature fusion module is configured to adaptively weight and fuse the geometric features and the cognitive feature vector at the front end of the geometric encoder using a gated fusion block to obtain modulated input features; the feature fusion module is further configured to map the cognitive feature vector to a first affine parameter at the global semantic bottleneck layer using a first cognitive feature linear modulation module, and perform feature modulation on the modulated input features based on the first affine parameter to obtain semantically calibrated global features; the feature fusion module is further configured to map the cognitive feature vector to a second affine parameter at the reconstruction decoder using a second cognitive feature linear modulation module, and perform feature modulation on the semantically calibrated global features based on the second affine parameter to obtain reconstructed features after detail recovery.

[0017] Optionally, the device further includes: a feature modulation module; the feature modulation module is configured to jointly process the geometric features and the cognitive feature vector through a multilayer perceptron to generate a learnable gating mask; the feature modulation module is further configured to perform a weighted summation of the geometric features and the cognitive feature vector based on the gating mask and a preset scaling factor to obtain the modulated input features.

[0018] Optionally, the feature fusion module is specifically used to map the cognitive feature vector into an affine parameter containing a scaling vector and a bias vector through a target cognitive feature linear modulation module; the target cognitive feature linear modulation module is either the first cognitive feature linear modulation module or the second cognitive feature linear modulation module; the feature fusion module is further used to scale the feature to be modulated in the current layer using the scaling vector, and to superimpose the scaled feature with the bias vector and the feature to be modulated in the current layer; wherein, when performing feature modulation based on the first affine parameter, the feature to be modulated in the current layer is the modulated input feature; when performing feature modulation based on the second affine parameter, the feature to be modulated in the current layer is the semantically calibrated global feature.

[0019] Optionally, the module training module is specifically used to obtain a spatial attention map by temperature scaling and normalizing the activation amplitude for the spatial structured features at the input level and the fine reconstruction level, and to calculate the divergence loss of the spatial attention maps of the teacher network and the initial student network as the attention transfer loss; the module training module is also specifically used to calculate the mean square error between the global features of the teacher network and the initial student network for the compact global features at the global semantic level as the direct feature alignment loss; the module training module is also specifically used to combine the attention transfer loss, the direct feature alignment loss, and the point cloud reconstruction task loss to determine the distillation loss.

[0020] This application also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the three-dimensional visual point cloud completion method based on human brain nuclear magnetic resonance cognitive distillation as described above.

[0021] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the three-dimensional visual point cloud completion method based on human brain nuclear magnetic resonance cognitive distillation as described above.

[0022] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the three-dimensional visual point cloud completion method based on human brain nuclear magnetic resonance cognitive distillation as described above.

[0023] The method and apparatus for 3D visual point cloud completion based on human brain MRI cognitive distillation provided in this application first acquire an incomplete 3D point cloud to be completed; then, the incomplete 3D point cloud is input into a point cloud completion model to obtain a completed 3D point cloud. The point cloud completion model includes a geometric encoder for extracting geometric features from the incomplete 3D point cloud to obtain point cloud features; a global semantic bottleneck layer for extracting global semantics from the point cloud features to obtain global features; and a reconstruction decoder for finely reconstructing the global features to output the completed point cloud. By internalizing the prior knowledge of human brain 3D visual cognition through hierarchical distillation into the point cloud completion model, the model can possess robust structural reasoning capabilities during the inference stage without relying on additional prior input, significantly improving completion generalization in severely occluded and cross-domain scenarios while maintaining real-time inference efficiency. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating the three-dimensional visual point cloud completion method based on human brain nuclear magnetic resonance cognitive distillation provided in this application; Figure 2 This is a schematic diagram of the overall framework of the three-dimensional visual point cloud completion method based on human brain nuclear magnetic resonance cognitive distillation provided in this application; Figure 3 This is a schematic diagram of the spatiotemporal cognitive encoder provided in this application; Figure 4 This is a schematic diagram of the gating fusion module provided in this application; Figure 5 This is a schematic diagram of the three-dimensional visual point cloud completion device based on human brain nuclear magnetic resonance cognitive distillation provided in this application; Figure 6 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. All actions involving the acquisition of signal information or data in this application are performed in accordance with the relevant data protection laws and policies of the country where the application is located and with authorization from the owner of the relevant device.

[0028] The technical solutions in related technologies mainly have the following shortcomings: First, mainstream point cloud completion methods heavily rely on the statistical geometric correlation of the training set and lack transferable, high-level structural priors. Under conditions of severe occlusion or semantic ambiguity, they are prone to producing ambiguous or unstructured outputs, resulting in limited generalization. Second, methods that introduce explicit structural priors require providing prior inputs (such as axes of symmetry or reference images) or performing complex iterative reasoning (such as diffusion steps) during the inference stage, which are often unavailable or computationally expensive in practical deployments. Third, although functional magnetic resonance imaging (fMRI) signals have been shown to encode high-level cognitive representations of human perception of 3D shapes, there is currently a lack of methods to incorporate such cognitive representations as transferable knowledge into point cloud completion models.

[0029] To address the aforementioned shortcomings, this application provides a 3D visual point cloud completion method based on human brain fMRI cognitive distillation. First, the method designs a spatiotemporal cognitive encoder to extract compact cognitive representations from fMRI signals that reflect how the human brain organizes and completes 3D geometry. Second, a multi-scale cognitive prior fusion strategy is proposed, injecting fMRI cognitive priors into three key stages: input layer gating, global semantic calibration, and detail reconstruction, guiding the completion process from coarse topology to fine surface generation. Next, a hierarchical teacher-student distillation framework is constructed, aligning teachers and students in both the feature space and attention map dimensions, enabling the student network to inherit the teacher's cognitive reasoning ability and attention patterns with only point clouds as input during the inference stage. Finally, the method achieves significantly better performance than existing methods under various settings, including intra-domain (ShapeNet-13), unseen categories (ShapeNet-55's 42 classes), and cross-domain (PCN), while maintaining high computational efficiency (>120 FPS) to adapt to real-time applications.

[0030] The following description, in conjunction with the accompanying drawings, details the three-dimensional visual point cloud completion method based on human brain nuclear magnetic resonance cognitive distillation provided in this application through specific embodiments and application scenarios.

[0031] like Figure 1 As shown in the embodiment of this application, a three-dimensional visual point cloud completion method based on human brain nuclear magnetic resonance cognitive distillation is provided. This method may include the following steps 101 and 102: Step 101: Obtain the incomplete 3D point cloud to be completed.

[0032] For example, the aforementioned incomplete 3D point cloud to be completed can be acquired using 3D sensing devices such as depth cameras and LiDAR. Due to mutual occlusion between objects in real-world scenes, limited sensor field of view, and the reflective properties of object surfaces, the acquired point cloud often contains large areas of missing data and voids. Structural restoration of the incomplete 3D point cloud to be completed is a crucial step in 3D visual understanding and downstream task execution.

[0033] Step 102: Input the incomplete 3D point cloud into the point cloud completion model to obtain the completed 3D point cloud.

[0034] The geometric encoder in the point cloud completion model is used to extract geometric features from the incomplete 3D point cloud to obtain point cloud features; the global semantic bottleneck layer in the point cloud completion model is used to extract global semantics from the point cloud features to obtain global features; and the reconstruction decoder in the point cloud completion model is used to perform fine reconstruction of the global features and output the completed point cloud.

[0035] For example, the backbone network of the point cloud completion model is a Transformer-based point cloud completion network. The backbone network is either AdaPoinTr or SnowFlakeNet.

[0036] For example, the point cloud completion model described above relies solely on point cloud data as input during the inference phase, without introducing additional modal data or prior constraints, thus achieving high inference speed. Specifically, the geometric encoder extracts the geometric shape and spatial distribution of the local neighborhood of the point cloud through structures such as multilayer perceptrons or graph convolutions; the global semantic bottleneck layer aggregates the scattered point cloud features into a compact global representation through max pooling or attention mechanisms, forcing the model to capture the overall structural topology of the object; the reconstruction decoder then uses folding operations or upsampling networks to gradually expand the global features into dense three-dimensional coordinates, thereby restoring fine surface details.

[0037] For example, such as Figure 2 The diagram shown illustrates the overall framework of the 3D visual point cloud completion method based on human brain MRI cognitive distillation provided in this application embodiment. It mainly consists of three core components: 1. A cognitive encoder for extracting spatiotemporal cognitive representations; 2. A multi-scale cognitive prior fusion strategy for integrating cross-modal information; and 3. A hierarchical cognitive distillation architecture for transferring high-level cognitive features. During the training phase, the teacher network simultaneously receives incomplete point clouds and fMRI cognitive signals, injecting cognitive priors into the geometric feature learning process through the fusion strategy. Subsequently, the teacher network parameters are frozen, and the cognitive knowledge is transferred to the student network, which receives only point clouds as input, through hierarchical distillation—the aforementioned point cloud completion model.

[0038] For example, during the inference phase after training is complete, the student network is completely decoupled from the fMRI data, improving the performance of various existing point cloud-completed backbone networks in a plug-and-play manner.

[0039] For example, the aforementioned spatiotemporal cognitive encoder consists of a three-stage cascaded structure of a spatial extractor, a temporal modeler, and a global aggregation module, used to extract compact cognitive feature vectors from fMRI image sequences that reflect the global perceptual structure of the human brain. .

[0040] For example, embodiments of this application provide two dedicated modules to realize multi-scale cognitive fusion: a gated fusion block performs adaptive cognitive regulation on the input at the front end of the geometric encoder; and a cognitive feature linear modulation module (CFLM) is deployed at both the global semantic bottleneck (Semantic Fusion) and the reconstructing head (Recon Fusion) to calibrate geometric features with semantically guided affine transformations to ensure that the generated three-dimensional structure is semantically consistent with human perception.

[0041] For example, the teacher network is a complete cognitive conditional model containing a cognitive fusion module, while the student network is a model with the same architecture but without the cognitive fusion module. Distillation is performed at three levels (Pre-Fusion input level, Global semantic level, and Refine-Decoder level) for hierarchical alignment, employing two distillation objectives: attention transfer (KL divergence constrained spatial attention map) and direct feature alignment (MSE constrained global feature vector), enabling the student network to inherit the teacher's low-level attention focusing and high-level semantic calibration capabilities.

[0042] For example, in the embodiments of this application, the training steps of the above-mentioned point cloud completion model include the following steps 103 to 106: Step 103: Construct a teacher network and an initial student network; the teacher network includes a cognitive fusion module, and the initial student network is a homogeneous network with the cognitive fusion module removed.

[0043] Specifically, the teacher network, by introducing human brain cognitive signals as priors, is able to perceive a more robust and global three-dimensional structure; the initial student network is a simplified architecture that removes the cognitive fusion module, aiming to internalize the cognitive abilities in the teacher network into a network with pure point cloud input through knowledge distillation, thereby getting rid of dependence on brain signals during reasoning.

[0044] Step 104: Obtain training data.

[0045] The training data includes incomplete point cloud samples and corresponding functional magnetic resonance imaging signals.

[0046] For example, the aforementioned incomplete point cloud samples can be obtained by simulating virtual perspective cropping of the complete 3D model or random discarding of some point clouds; the corresponding functional magnetic resonance imaging signal is the brain blood oxygen level dependent signal collected by MRI equipment when the subject observes the same 3D visual stimulus.

[0047] Step 105: Input the training data into the teacher network, extract the cognitive feature vector from the functional magnetic resonance imaging signal through the spatiotemporal cognitive encoder, and perform multi-scale fusion of the cognitive feature vector with the geometric features of the incomplete point cloud sample through the cognitive fusion module, and train the teacher network until convergence.

[0048] Specifically, during training, the teacher network simultaneously receives bimodal inputs. The spatiotemporal cognitive encoder extracts implicit vectors reflecting three-dimensional shape cognition from complex brain signals. The cognitive fusion module then injects these cognitive vectors into various stages of point cloud feature processing, guiding geometric features to evolve in a direction that conforms to the cognitive laws of the human brain, until the teacher network converges and obtains feature representation capabilities with strong cognitive priors.

[0049] For example, in this embodiment of the application, fMRI image sequences are used as input data. The raw fMRI scans are preprocessed and converted into cortical flat maps. The input data is formalized into spatiotemporal tensors. .in, Indicates the number of consecutive frames within a time window. This indicates the spatial resolution of each frame.

[0050] like Figure 3 The diagram shown is a structural schematic of the spatiotemporal cognitive encoder provided in an embodiment of this application. The encoder consists of three stages, specifically including: Spatial extractor: A spatial Transformer

[25] is used as the backbone to capture global spatial dependencies. Assume For fMRI frames at time step t, the spatial Transformer It is applied in a time-distributed manner, sharing parameters across all time steps. To ensure consistent feature mapping: ; Temporal modeler: uses an LSTM network to process spatially embedded sequences. Recursively based on the current input and the previous state Update hidden status: ; This process produces a series of context-aware features. It encodes the temporal evolution of cognitive response.

[0051] Global aggregation module: Introduces an attention-based global aggregation mechanism, utilizing a learnable query. LSTM output sequence (Using both as Key and Value) a weighted aggregation, the final global cognitive feature vector is obtained. : ; in, This is the scaling factor. This mechanism allows the model to dynamically focus on the most critical time frames containing significant activations.

[0052] Specifically, in step 105 above, the extraction of cognitive feature vectors from the functional magnetic resonance imaging signal using a spatiotemporal cognitive encoder may include the following steps 105a1 to 105a4: Step 105a1: Preprocess the functional magnetic resonance imaging signal to obtain a planar mapping of the cerebral cortex in the form of a spatiotemporal tensor.

[0053] For example, the original brain signals contain noise and are dimensionally complex. The preprocessing process includes operations such as temporal correction, spatial smoothing and head movement correction. Then, the three-dimensional voxel data is projected onto the two-dimensional cortical surface and stacked in time sequence to form a two-dimensional image sequence that preserves the spatiotemporal continuity, namely, a planar mapping of the cerebral cortex.

[0054] Step 105a2: Extract spatial features from each frame of the cerebral cortex planar mapping using a spatial extractor to obtain a spatial embedding sequence.

[0055] For example, the spatial extractor can use a two-dimensional convolutional network to perform feature mapping on each frame of the cerebral cortex planar map and extract the spatial distribution features of the activated areas of the cerebral cortex at each time point.

[0056] Step 105a3: Perform temporal dependency modeling on the spatial embedding sequence using a long short-term memory network to obtain a context-aware feature sequence.

[0057] For example, human visual cognition has significant temporal dynamic characteristics. Long short-term memory networks can effectively capture the temporal evolution of spatial embedding sequences and fuse the independent spatial features of each frame into a feature sequence containing temporal context information.

[0058] Step 105a4: Use the learnable query vector to perform attention-weighted aggregation on the context-aware feature sequence through the global aggregation module, and output the cognitive feature vector.

[0059] For example, the learnable query vector can be used as a global information probe. The relevance weights to the context-aware feature sequence are calculated through an attention mechanism. The key information most relevant to the current 3D shape cognition is filtered out from the lengthy temporal features and compressed into a fixed-length cognitive feature vector.

[0060] In another embodiment, in step 105 above, the multi-scale fusion of the cognitive feature vector with the geometric features of the incomplete point cloud sample through the cognitive fusion module may include the following steps 105b1 to 105b13: Step 105b1: At the front end of the geometric encoder, the geometric features and the cognitive feature vector are adaptively weighted and fused by a gating fusion block to obtain the modulated input features.

[0061] For example, in the shallow stage of the geometric encoder, the point cloud geometric features contain rich local details but lack overall structural awareness. The gated fusion block selects geometric features related to the global structure based on the cognitive feature vector, thereby achieving feature alignment and enhancement in the initial stage.

[0062] For example, in the embodiments of this application, the abstraction of low-level geometry is crucial in the initial stage of point cloud encoding. In this embodiment, a gated fusion block is placed at the front end of the geometric encoder to adaptively introduce cognitive guidance in geometrically ambiguous regions while preserving the integrity of the original geometric details. Figure 4 The diagram shown is a structural schematic of the gating fusion module. and These represent the initial point cloud features and the cognitive prior feature vectors, respectively. A lightweight MLP is used to jointly process the two modalities, generating a learnable gated mask. Modulated input Defined as: ; in, As a scaling factor, this gating mechanism enables the model to dynamically calibrate the degree of influence of cognitive priors based on the geometric confidence of the input data; B This refers to the batch size, which is the number of point cloud samples input into the network for processing in a single operation. C The number of feature channels is the dimension of each point or the global feature vector.

[0063] Step 105b2: In the global semantic bottleneck layer, the cognitive feature vector is mapped to a first affine parameter through the first cognitive feature linear modulation module, and the modulated input feature is modulated based on the first affine parameter to obtain the semantically calibrated global feature.

[0064] For example, the global semantic bottleneck layer is a key node for the network to extract the global topology. The first cognitive feature linear modulation module uses the cognitive feature vector to generate affine transformation parameters and performs affine transformation on the aggregated global features so that the global semantic features conform to the semantic-level cognitive calibration of the human brain for three-dimensional shapes.

[0065] Step 105b3: At the reconstruction decoder end, the cognitive feature vector is mapped to the second affine parameter through the second cognitive feature linear modulation module, and the semantically calibrated global features are modulated based on the second affine parameter to obtain the reconstructed features after detail restoration.

[0066] For example, in the high-level stage of reconstructing the decoder, the network is committed to recovering local geometric details. The second cognitive feature linear modulation module introduces cognitive priors again to modulate the features of the decoder, ensuring that the generated details are consistent with the local geometric logic and also with the overall structural cognition of the human brain.

[0067] For example, in this embodiment of the application, to ensure that the generated 3D structure is semantically consistent with human perception, this embodiment proposes a Cognitive Feature Linear Modulation (CFLM) module for mid-level and late-stage feature refinement. Two CFLM modules are deployed at the global semantic bottleneck (Semantic Fusion) and the reconstruction head (Recon Fusion), respectively. For a given geometric feature... CFLM maps cognitive features (i.e., the aforementioned cognitive prior feature vectors) to vectors containing scaling vectors. With bias vector Affine parameters are used to transform the feature space: ; At semantic bottlenecks, this modulation ensures that abstract geometric structures align with cognitive categories, at which point the above... This represents the global features after semantic calibration; during the reconstruction phase, the second CFLM utilizes subtle cognitive information to recover missing structural details that are often ambiguous in pure geometric reasoning. At this point, the aforementioned... This indicates the reconstructed features after the details have been restored.

[0068] Specifically, in step 105b1 above, the adaptive weighted fusion of the geometric features and the cognitive feature vector through a gated fusion block to obtain the modulated input features may include the following steps 105b11 and 105b12: Step 105b11: The geometric features and the cognitive feature vectors are jointly processed by a multilayer perceptron to generate a learnable gating mask.

[0069] For example, geometric features and cognitive feature vectors are concatenated along the feature dimension and then input into a multilayer perceptron. The multilayer perceptron outputs a mask between 0 and 1, which can adaptively determine the importance of each channel in the geometric features.

[0070] Step 105b12: Based on the gated mask and the preset scaling factor, the geometric features and the cognitive feature vector are weighted and summed to obtain the modulated input features.

[0071] For example, a preset scaling factor is used to balance the initial contribution weights of the geometric branch and the cognitive branch, while the gating mask serves as a dynamic coefficient to control the degree to which the cognitive feature vector complements the geometric features. The weighted summed features retain both local geometry and global cognition.

[0072] In another embodiment, step 105 above, which involves feature modulation based on the first affine parameter or feature modulation based on the second affine parameter, may include the following steps 105c1 to 105c3: Step 105c1: Map the cognitive feature vector to an affine parameter containing a scaling vector and a bias vector through the target cognitive feature linear modulation module; the target cognitive feature linear modulation module is either the first cognitive feature linear modulation module or the second cognitive feature linear modulation module.

[0073] For example, the target cognitive feature linear modulation module contains a linear mapping layer that decompresses the compressed cognitive feature vector into a scaling vector and a bias vector consistent with the current number of feature channels, for subsequent affine transformations.

[0074] Step 105c2: Scale the current layer's modulated features using the scaling vector, and then superimpose the scaled features with the bias vector and the current layer's modulated features.

[0075] For example, the above operation recalibrates the amplitude of each channel of the feature by scaling vector, performs translation compensation by bias vector, and performs residual connection with the original feature to be modulated, which introduces cognitive modulation and avoids the loss of original feature information.

[0076] Step 105c3: When performing feature modulation based on the first affine parameter, the feature to be modulated in the current layer is the modulated input feature; when performing feature modulation based on the second affine parameter, the feature to be modulated in the current layer is the semantically calibrated global feature.

[0077] Specifically, by distinguishing the features to be modulated at different stages, a multi-scale deep penetration of cognitive priors is achieved, from local input modulation to global semantic modulation and then to reconstruction detail modulation.

[0078] Step 106: Freeze the parameters of the teacher network, input the incomplete point cloud sample into the initial student network, perform hierarchical alignment based on the intermediate representations of the teacher network and the initial student network at multiple levels, calculate the distillation loss, and train the initial student network based on the distillation loss to obtain the point cloud completion model.

[0079] For example, after the teacher network training converges, its internal features already possess excellent cognitive guidance capabilities. At this point, the teacher network is frozen, and the student network is driven solely by point cloud data. By constraining the intermediate features of the student network at multiple levels to approximate the teacher network, the student network can still simulate the structural reasoning pattern of the teacher network even after the cognitive fusion module is removed.

[0080] Specifically, in step 106 above, the step of performing hierarchical alignment based on the intermediate representations of the teacher network and the initial student network at multiple levels and calculating the distillation loss may include the following steps 106a1 to 106a3: Step 106a1: For the spatial structured features of the input level and the fine reconstruction level, the spatial attention map is obtained by normalizing the activation amplitude through temperature scaling. The divergence loss of the spatial attention maps of the teacher network and the initial student network is calculated as the attention transfer loss.

[0081] For example, the shallow input level and the deep reconstruction level contain rich local spatial structure information. The activation values ​​are smoothed by temperature scaling and attention maps are generated. The difference in attention distribution between the two networks is then measured using divergence loss, prompting the student network to pay attention to the same local key structures as the teacher network.

[0082] Step 106a2: For the compact global features at the global semantic level, calculate the mean square error between the global features of the teacher network and the initial student network, as the direct feature alignment loss.

[0083] For example, the global features of the bottleneck layer directly determine the overall topology of the generated shape, and the mean square error can force the global features of the student network to approximate the teacher network in the numerical space, ensuring the accurate transmission of the macroscopic structure.

[0084] Step 106a3: Combine the attention transfer loss, the direct feature alignment loss, and the point cloud reconstruction task loss to determine the distillation loss.

[0085] For example, the point cloud reconstruction task loss ensures basic shape recovery capability, while the attention transfer loss and direct feature alignment loss realize the transfer of cognitive knowledge from both local and global dimensions. The joint optimization of the three ensures that the student network obtained by distillation has both reconstruction accuracy and generalization performance.

[0086] For example, although teacher networks can effectively guide point cloud completion through cognitive fusion, the requirement to simultaneously acquire fMRI data during the inference stage is not feasible in practice. Therefore, this application proposes a hierarchical cognitive distillation framework, which specifically includes: The teacher network T is a complete cognitive conditional model, and the student network S is a model with the same architecture but without the cognitive fusion module. The core idea is to force students to internalize cognitive modulation patterns by imitating the teacher's intermediate representations. Hierarchical alignment is established at three key locations: the Pre-Fusion input level, the Global semantic level, and the Refine-Decoder level.

[0087] The distillation objective is designed as follows: For the first level Pre-Fusion (input stage) ) and the third level Refine-Decoder (fine reconstruction level) Based on the spatial structuring characteristics of fMRI, assuming that the main contribution of fMRI is to guide network attention to information-rich regions, an attention transfer mechanism is adopted. This involves obtaining a spatial attention map by normalizing activation amplitude through temperature scaling and softmax, and minimizing the difference in teacher and student attention using KL divergence, denoted as... For the second level, Global (global semantic level) The compact global features of the algorithm are used to employ direct feature alignment of MSE to constrain the student encoder to produce latent vectors that are aligned with the teacher's semantic calibration representation.

[0088] The training process follows a two-phase course: In the first stage, the teacher network is trained to convergence under the supervision of complete cognitive signals to obtain the optimal cognitive mapping ability; in the second stage, the teacher parameters are frozen, and the student network is optimized with a composite objective. ; in, For standard Chamfer Distance (CD) reconstruction loss, For feature alignment loss, , 2. For each distillation loss weight, (1), (2), and (3) above are the hierarchical indices. This end-to-end distillation ensures that the student network not only reconstructs the accurate geometry but also replicates the teacher's underlying cognitive reasoning process.

[0089] For example, to verify the plug-and-play capability of this scheme framework, this scheme integrates it into two representative backbone networks, AdaPoinTr

[12] and SnowFlakeNet[9], respectively, and the resulting cognitive enhancement models are denoted as Cog-AdaPoinTr and Cog-SnowFlakeNet, respectively.

[0090] Training details: The framework is implemented based on PyTorch, and training is performed on an NVIDIA RTX A6000 GPU. Both teacher training and distillation processes utilize the AdamW optimizer, with an initial learning rate of 1×10⁻⁶. -4 Weight decay 5×10 -4 The learning rate decreases by 0.9 every 21 epochs. Both models are trained for 600 epochs with a batch size of 48. To ensure stability, gradient clipping with a maximum norm of 1.0 is used.

[0091] Dataset: The main dataset is the ShapeNet-13 dataset paired with fMRI. The fMRI data comes from Mind-3D

[24] , which systematically pairs human brain activity with three-dimensional object stimuli. The dataset contains 1,404 paired samples, 13 object categories, and 108 independent instances per category. During data acquisition, subjects watched a 10-second video clip of a static three-dimensional object rotating 360° around the vertical axis to convey complete structural information, while BOLD signals were recorded simultaneously using a 3T fMRI scanner. Finally, a 10×256×256 planar mapping map of the cerebral cortex was used for the experiment. The point cloud data strictly followed the data format of ProjectedShapeNet-55

[12] . The training set and the test set were strictly divided at 80% / 20% (1,123 training / 281 test). To evaluate generalization, 42 unseen categories other than the 13 categories in ShapeNet-55 were introduced for zero-shot evaluation, and the PCN dataset

[11] was used for cross-domain evaluation (pure inference mode, without any fine-tuning).

[0092] Evaluation Metrics: Following standard protocols, Chamfer Distance (CD) and F1-Score were used as the primary evaluation metrics. CD measures the average nearest neighbor distance between the predicted point cloud and the ground truth point cloud, reflecting geometric fidelity; CD-L1 (insensitive to outliers) based on the L1 norm and CD-L2 (based on the L2 norm) were reported separately. Efficiency was evaluated using inference time (ms), frames per second (FPS), and floating-point operations (FLOPs, G).

[0093] The three-dimensional visual point cloud completion method based on human brain nuclear magnetic resonance cognitive distillation provided in this application internalizes the human brain's three-dimensional visual cognitive priors into the point cloud completion model through hierarchical distillation. This enables the model to have robust structural reasoning capabilities without relying on additional prior inputs during the reasoning stage, significantly improving the completion generalization in severely occluded and cross-domain scenarios, while maintaining real-time reasoning efficiency.

[0094] The 3D visual point cloud completion method based on human brain MRI cognitive distillation provided in this application first obtains an incomplete 3D point cloud to be completed; then, the incomplete 3D point cloud is input into a point cloud completion model to obtain a completed 3D point cloud. The point cloud completion model includes a geometric encoder for extracting geometric features from the incomplete 3D point cloud to obtain point cloud features; a global semantic bottleneck layer for extracting global semantics from the point cloud features to obtain global features; and a reconstruction decoder for finely reconstructing the global features to output the completed point cloud. By internalizing the human brain's 3D visual cognitive priors through hierarchical distillation into the point cloud completion model, the model can achieve robust structural reasoning capabilities during the inference stage without relying on additional prior inputs, significantly improving completion generalization in severely occluded and cross-domain scenarios while maintaining real-time inference efficiency.

[0095] It should be noted that the 3D visual point cloud completion method based on human brain nuclear magnetic resonance cognitive distillation provided in this application embodiment can be executed by a 3D visual point cloud completion device based on human brain nuclear magnetic resonance cognitive distillation, or by a control module within that device for executing the 3D visual point cloud completion method based on human brain nuclear magnetic resonance cognitive distillation. This application embodiment uses the execution of the 3D visual point cloud completion method based on human brain nuclear magnetic resonance cognitive distillation by the 3D visual point cloud completion device as an example to illustrate the 3D visual point cloud completion device based on human brain nuclear magnetic resonance cognitive distillation provided in this application embodiment.

[0096] It should be noted that, in the embodiments of this application, the three-dimensional visual point cloud completion methods based on human brain nuclear magnetic resonance cognitive distillation shown in the accompanying drawings are all illustrated by way of example with reference to one of the accompanying drawings in the embodiments of this application. In specific implementation, the three-dimensional visual point cloud completion methods based on human brain nuclear magnetic resonance cognitive distillation shown in the accompanying drawings can also be implemented in conjunction with any other accompanying drawings shown in the above embodiments, which will not be elaborated here.

[0097] The following describes the three-dimensional visual point cloud completion device based on human brain nuclear magnetic resonance cognitive distillation provided in this application. The three-dimensional visual point cloud completion device based on human brain nuclear magnetic resonance cognitive distillation described below can be referred to in correspondence with the three-dimensional visual point cloud completion method based on human brain nuclear magnetic resonance cognitive distillation described above.

[0098] Figure 5 This is a schematic diagram of the structure of the three-dimensional visual point cloud completion device based on human brain nuclear magnetic resonance cognitive distillation provided in the embodiments of this application, as shown below. Figure 5 As shown, it specifically includes: The acquisition module 501 is used to acquire the incomplete 3D point cloud to be completed; the point cloud completion module 502 is used to input the incomplete 3D point cloud into the point cloud completion model to obtain the completed 3D point cloud; wherein, the geometric encoder in the point cloud completion model is used to extract geometric features from the incomplete 3D point cloud to obtain point cloud features; the global semantic bottleneck layer in the point cloud completion model is used to extract global semantics from the point cloud features to obtain global features; the reconstruction decoder in the point cloud completion model is used to perform fine reconstruction of the global features and output the completed point cloud.

[0099] Optionally, the device further includes: a model building module and a model training module; the model building module is used to build a teacher network and an initial student network; the teacher network includes a cognitive fusion module, and the initial student network is a network with the same architecture after removing the cognitive fusion module; the acquisition module 501 is further used to acquire training data; the training data includes incomplete point cloud samples and corresponding functional magnetic resonance imaging (fMRI) signals; the model training module is specifically used to input the training data into the teacher network, extract cognitive feature vectors from the fMRI signals through a spatiotemporal cognitive encoder, and perform multi-scale fusion of the cognitive feature vectors with the geometric features of the incomplete point cloud samples through the cognitive fusion module, training the teacher network until convergence; the model training module is also specifically used to freeze the parameters of the teacher network, input the incomplete point cloud samples into the initial student network, perform hierarchical alignment based on the intermediate representations of the teacher network and the initial student network at multiple levels, calculate distillation loss, and train the initial student network based on the distillation loss to obtain the point cloud completion model.

[0100] Optionally, the device further includes: a feature extraction module; the feature extraction module is used to preprocess the functional magnetic resonance imaging signal to obtain a spatiotemporal tensor form of a cortical plane mapping map; the feature extraction module is also used to extract spatial features from each frame of the cortical plane mapping map using a spatial extractor to obtain a spatial embedding sequence; the feature extraction module is also used to perform temporal dependency modeling on the spatial embedding sequence using a long short-term memory network to obtain a context-aware feature sequence; the feature extraction module is also used to perform attention-weighted aggregation of the context-aware feature sequence using a learnable query vector through a global aggregation module to output the cognitive feature vector.

[0101] Optionally, the apparatus further includes: a feature fusion module; the feature fusion module is configured to adaptively weight and fuse the geometric features and the cognitive feature vector at the front end of the geometric encoder using a gated fusion block to obtain modulated input features; the feature fusion module is further configured to map the cognitive feature vector to a first affine parameter at the global semantic bottleneck layer using a first cognitive feature linear modulation module, and perform feature modulation on the modulated input features based on the first affine parameter to obtain semantically calibrated global features; the feature fusion module is further configured to map the cognitive feature vector to a second affine parameter at the reconstruction decoder using a second cognitive feature linear modulation module, and perform feature modulation on the semantically calibrated global features based on the second affine parameter to obtain reconstructed features after detail recovery.

[0102] Optionally, the device further includes: a feature modulation module; the feature modulation module is configured to jointly process the geometric features and the cognitive feature vector through a multilayer perceptron to generate a learnable gating mask; the feature modulation module is further configured to perform a weighted summation of the geometric features and the cognitive feature vector based on the gating mask and a preset scaling factor to obtain the modulated input features.

[0103] Optionally, the feature fusion module is specifically used to map the cognitive feature vector into an affine parameter containing a scaling vector and a bias vector through a target cognitive feature linear modulation module; the target cognitive feature linear modulation module is either the first cognitive feature linear modulation module or the second cognitive feature linear modulation module; the feature fusion module is further used to scale the feature to be modulated in the current layer using the scaling vector, and to superimpose the scaled feature with the bias vector and the feature to be modulated in the current layer; wherein, when performing feature modulation based on the first affine parameter, the feature to be modulated in the current layer is the modulated input feature; when performing feature modulation based on the second affine parameter, the feature to be modulated in the current layer is the semantically calibrated global feature.

[0104] Optionally, the module training module is specifically used to obtain a spatial attention map by temperature scaling and normalizing the activation amplitude for the spatial structured features at the input level and the fine reconstruction level, and to calculate the divergence loss of the spatial attention maps of the teacher network and the initial student network as the attention transfer loss; the module training module is also specifically used to calculate the mean square error between the global features of the teacher network and the initial student network for the compact global features at the global semantic level as the direct feature alignment loss; the module training module is also specifically used to combine the attention transfer loss, the direct feature alignment loss, and the point cloud reconstruction task loss to determine the distillation loss.

[0105] The 3D visual point cloud completion device based on human brain MRI cognitive distillation provided in this application first acquires an incomplete 3D point cloud to be completed; then, the incomplete 3D point cloud is input into a point cloud completion model to obtain a completed 3D point cloud. The point cloud completion model includes a geometric encoder for extracting geometric features from the incomplete 3D point cloud to obtain point cloud features; a global semantic bottleneck layer for extracting global semantics from the point cloud features to obtain global features; and a reconstruction decoder for finely reconstructing the global features to output the completed point cloud. Thus, by internalizing the human brain's 3D visual cognitive priors through hierarchical distillation into the point cloud completion model, the model can possess robust structural reasoning capabilities during the inference stage without relying on additional prior inputs, significantly improving completion generalization in severely occluded and cross-domain scenarios while maintaining real-time reasoning efficiency.

[0106] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a three-dimensional visual point cloud completion method based on human brain nuclear magnetic resonance cognitive distillation. The method includes: first, acquiring the incomplete three-dimensional point cloud to be completed; then, inputting the incomplete three-dimensional point cloud into a point cloud completion model to obtain a completed three-dimensional point cloud; wherein, the geometric encoder in the point cloud completion model is used to extract geometric features from the incomplete three-dimensional point cloud to obtain point cloud features; the global semantic bottleneck layer in the point cloud completion model is used to extract global semantics from the point cloud features to obtain global features; and the reconstruction decoder in the point cloud completion model is used to perform fine reconstruction of the global features and output the completed point cloud. Thus, by internalizing the prior knowledge of human three-dimensional visual cognition into the point cloud completion model through hierarchical distillation, the model can have robust structural reasoning ability without relying on additional prior input during the reasoning stage, which significantly improves the completion generalization in severe occlusion and cross-domain scenarios, while maintaining real-time reasoning efficiency.

[0107] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0108] On the other hand, this application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer is able to execute the three-dimensional visual point cloud completion method based on human brain nuclear magnetic resonance cognitive distillation provided by the above methods. The method includes: first, acquiring a three-dimensional incomplete point cloud to be completed; then, inputting the three-dimensional incomplete point cloud into a point cloud completion model to obtain a completed three-dimensional point cloud; wherein, the geometric encoder in the point cloud completion model is used to extract geometric features from the three-dimensional incomplete point cloud to obtain point cloud features; the global semantic bottleneck layer in the point cloud completion model is used to extract global semantics from the point cloud features to obtain global features; and the reconstruction decoder in the point cloud completion model is used to perform fine reconstruction of the global features and output the completed point cloud. Thus, by internalizing the prior knowledge of human three-dimensional visual cognition into the point cloud completion model through hierarchical distillation, the model can have robust structural reasoning ability without relying on additional prior input during the reasoning stage, which significantly improves the completion generalization in severe occlusion and cross-domain scenarios, while maintaining real-time reasoning efficiency.

[0109] Furthermore, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the aforementioned three-dimensional visual point cloud completion methods based on human brain nuclear magnetic resonance cognitive distillation. The method includes: first, acquiring an incomplete three-dimensional point cloud to be completed; then, inputting the incomplete three-dimensional point cloud into a point cloud completion model to obtain a completed three-dimensional point cloud; wherein, a geometric encoder in the point cloud completion model is used to extract geometric features from the incomplete three-dimensional point cloud to obtain point cloud features; a global semantic bottleneck layer in the point cloud completion model is used to extract global semantics from the point cloud features to obtain global features; and a reconstruction decoder in the point cloud completion model is used to perform fine reconstruction of the global features and output the completed point cloud. Thus, by internalizing the human brain's three-dimensional visual cognitive prior through hierarchical distillation into the point cloud completion model, the model can possess robust structural reasoning capabilities during the inference stage without relying on additional prior input, significantly improving completion generalization in severely occluded and cross-domain scenarios while maintaining real-time inference efficiency.

[0110] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0111] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for completing three-dimensional visual point clouds based on human brain nuclear magnetic resonance cognitive distillation, characterized in that, include: Obtain the incomplete 3D point cloud to be completed; The incomplete 3D point cloud is input into the point cloud completion model to obtain the completed 3D point cloud. The geometric encoder in the point cloud completion model is used to extract geometric features from the incomplete 3D point cloud to obtain point cloud features; the global semantic bottleneck layer in the point cloud completion model is used to extract global semantics from the point cloud features to obtain global features; and the reconstruction decoder in the point cloud completion model is used to perform fine reconstruction of the global features and output the completed point cloud.

2. The method according to claim 1, characterized in that, The training steps of the point cloud completion model include: Construct a teacher network and an initial student network; the teacher network includes a cognitive fusion module, and the initial student network is a homogeneous network with the cognitive fusion module removed; Acquire training data; the training data includes incomplete point cloud samples and corresponding functional magnetic resonance imaging signals; The training data is input into the teacher network, and the cognitive feature vector in the functional magnetic resonance imaging signal is extracted by the spatiotemporal cognitive encoder. The cognitive feature vector is then fused with the geometric features of the incomplete point cloud sample at multiple scales by the cognitive fusion module, and the teacher network is trained until convergence. The parameters of the teacher network are frozen, the incomplete point cloud samples are input into the initial student network, hierarchical alignment is performed based on the intermediate representations of the teacher network and the initial student network at multiple levels, the distillation loss is calculated, and the initial student network is trained based on the distillation loss to obtain the point cloud completion model.

3. The method according to claim 2, characterized in that, The step of extracting cognitive feature vectors from the functional magnetic resonance imaging signal using a spatiotemporal cognitive encoder includes: The functional magnetic resonance imaging signal is preprocessed to obtain a planar mapping of the cerebral cortex in the form of a spatiotemporal tensor. Spatial features are extracted from each frame of the planar mapping of the cerebral cortex using a spatial extractor to obtain a spatial embedding sequence. Temporal dependency modeling of the spatially embedded sequence is performed using a long short-term memory network to obtain a context-aware feature sequence. The global aggregation module uses learnable query vectors to perform attention-weighted aggregation on the context-aware feature sequence, and outputs the cognitive feature vector.

4. The method according to claim 2, characterized in that, The step of fusing the cognitive feature vector with the geometric features of the incomplete point cloud sample at multiple scales through the cognitive fusion module includes: At the front end of the geometric encoder, the geometric features and the cognitive feature vector are adaptively weighted and fused by a gated fusion block to obtain the modulated input features. In the global semantic bottleneck layer, the cognitive feature vector is mapped to a first affine parameter through the first cognitive feature linear modulation module, and the modulated input feature is modulated based on the first affine parameter to obtain the semantically calibrated global feature. At the reconstruction decoder, the cognitive feature vector is mapped to a second affine parameter through the second cognitive feature linear modulation module, and the semantically calibrated global features are modulated based on the second affine parameter to obtain the reconstructed features after detail restoration.

5. The method according to claim 4, characterized in that, The step of adaptively weighting and fusing the geometric features and the cognitive feature vector through a gated fusion block to obtain modulated input features includes: The geometric features and the cognitive feature vectors are jointly processed by a multilayer perceptron to generate a learnable gating mask; Based on the gated mask and a preset scaling factor, the geometric features and the cognitive feature vector are weighted and summed to obtain the modulated input features.

6. The method according to claim 4, characterized in that, The feature modulation based on the first affine parameter, or the feature modulation based on the second affine parameter, includes: The target cognitive feature linear modulation module maps the cognitive feature vector to an affine parameter containing a scaling vector and a bias vector; the target cognitive feature linear modulation module is either the first cognitive feature linear modulation module or the second cognitive feature linear modulation module. The scaling vector is used to scale the features to be modulated in the current layer, and the scaled features are then superimposed with the bias vector and the features to be modulated in the current layer. Specifically, when performing feature modulation based on the first affine parameter, the feature to be modulated in the current layer is the modulated input feature; when performing feature modulation based on the second affine parameter, the feature to be modulated in the current layer is the semantically calibrated global feature.

7. The method according to claim 2, characterized in that, The step of performing hierarchical alignment based on intermediate representations at multiple levels of the teacher network and the initial student network, and calculating distillation loss, includes: For the spatial structured features of the input level and the fine reconstruction level, the spatial attention map is obtained by normalizing the activation amplitude through temperature scaling. The divergence loss of the spatial attention maps of the teacher network and the initial student network is calculated as the attention transfer loss. For compact global features at the global semantic level, the mean square error between the global features of the teacher network and the initial student network is calculated as the direct feature alignment loss. The distillation loss is determined by combining the attention transfer loss, the direct feature alignment loss, and the point cloud reconstruction task loss.

8. The method according to any one of claims 1 to 7, characterized in that, The backbone network of the point cloud completion model is a Transformer-based point cloud completion network.

9. The method according to claim 8, characterized in that, The backbone network is either AdaPoinTr or SnowFlakeNet.

10. A three-dimensional visual point cloud completion device based on human brain nuclear magnetic resonance cognitive distillation, characterized in that, The device includes: The acquisition module is used to acquire the incomplete 3D point cloud to be completed; The point cloud completion module is used to input the incomplete 3D point cloud into the point cloud completion model to obtain the completed 3D point cloud. The geometric encoder in the point cloud completion model is used to extract geometric features from the incomplete 3D point cloud to obtain point cloud features; the global semantic bottleneck layer in the point cloud completion model is used to extract global semantics from the point cloud features to obtain global features; and the reconstruction decoder in the point cloud completion model is used to perform fine reconstruction of the global features and output the completed point cloud.

Images