Multi-view clustering method and system for non-line-of-sight imaging data, electronic device and storage medium

Abstract

The application relates to the technical field of image data processing, and discloses a multi-view clustering method and system for non-line-of-sight imaging data, electronic equipment and a storage medium. The method extracts a plurality of modal category reconstruction depth image data sets of a target recognition object, performs feature extraction of a plurality of view angles on each category of the reconstruction depth image data sets, describes the internal structural characteristics of each category of the reconstruction depth image data sets from different view angles, performs data dimension reduction on a plurality of view angle feature vector matrices, jointly optimizes the depth non-negative matrix decomposition and a multi-optimization target, ensures that each layer of the decomposition result can meet the multi-optimization target while maintaining non-negativity, fuses the optimized decomposition feature matrices, performs clustering analysis on the fusion feature matrices corresponding to each reconstruction depth image data set, and thus accurate clustering analysis of each reconstruction depth image data set is realized, and the convergence speed in the clustering process is improved.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to a multi-view clustering method, system, electronic device, and storage medium for non-line-of-sight imaging data. Background Technology

[0002] Non-line-of-sight imaging is an imaging technology that reconstructs the three-dimensional structure of a hidden target by indirectly scattering light signals. It aims to achieve imaging of objects that are obscured outside the field of view by analyzing diffuse reflection information collected by sensors.

[0003] In object recognition tasks following non-line-of-sight imaging, end-to-end deep learning methods employ Softmax cross-entropy loss as the classifier. Existing methods typically input the raw data directly into the model, resulting in a large amount of noise, irrelevant, or redundant information mixed in the input. This significantly increases the computational burden and reduces the accuracy of subsequent clustering tasks. Furthermore, when processing high-dimensional data, it is easy to retain too many redundant components, leading to insufficiently compact decomposition results, which in turn results in slow convergence speed and long overall processing time during clustering. Summary of the Invention

[0004] In view of this, the present invention provides a multi-view clustering method, system, electronic device, and storage medium for non-line-of-sight imaging data, solving the technical problem that existing methods significantly increase computational burden and reduce the accuracy of subsequent clustering tasks. Furthermore, when processing high-dimensional data, they tend to retain too many redundant components, resulting in insufficiently compact decomposition results, which in turn leads to slow convergence speed and long overall processing time during clustering.

[0005] The first aspect of this invention provides a multi-view clustering method for non-line-of-sight imaging data, comprising:

[0006] Extract reconstructed depth image datasets of objects with multiple modal categories for target recognition;

[0007] For each reconstructed depth image dataset, feature extraction is performed from multiple perspectives to obtain a feature vector matrix corresponding to each reconstructed depth image dataset.

[0008] For each of the reconstructed depth image datasets, the multiple view feature vector matrices are subjected to dimensionality reduction to obtain the simplified view feature submatrices corresponding to each view feature vector matrix.

[0009] Based on the deep nonnegative matrix factorization framework, the simplified view feature submatrices are decomposed into hierarchical feature matrices, and the decomposed feature matrices are optimized iteratively with multiple optimization objectives to obtain the optimized decomposed feature matrices.

[0010] The optimized decomposed feature matrices corresponding to each of the simplified view feature sub-matrices are fused to obtain the fused feature matrices corresponding to each of the reconstructed depth image datasets. Cluster analysis is then performed on the fused feature matrices corresponding to each of the reconstructed depth image datasets to obtain the clustering results corresponding to each of the reconstructed depth image datasets.

[0011] In one embodiment, the modal categories of the reconstructed depth image dataset include light conic transform reconstructed depth image dataset, filtered back projection reconstructed depth image dataset, and non-line-of-sight transform reconstructed depth image dataset;

[0012] The reconstructed depth image dataset for extracting multiple modal categories of the target object includes:

[0013] Acquire non-line-of-sight imaging data of the target object;

[0014] The non-line-of-sight imaging data is reconstructed using the LCT reconstruction algorithm to generate the light conic transform reconstructed depth image dataset.

[0015] The non-line-of-sight imaging data is reconstructed using the FBP reconstruction algorithm to generate the filtered back-projection reconstructed depth image dataset.

[0016] The non-line-of-sight imaging data is reconstructed using the NLOST reconstruction algorithm to generate the non-line-of-sight transformed reconstructed depth image dataset.

[0017] In one embodiment, the view feature vector matrix includes the Gray view feature vector matrix, the LBP view feature vector matrix, and the HOG view feature vector matrix;

[0018] The step of extracting features from multiple perspectives for each of the reconstructed depth image datasets to obtain a feature vector matrix from multiple perspectives corresponding to each of the reconstructed depth image datasets includes:

[0019] Each of the reconstructed depth image datasets is converted into a grayscale image, and a grayscale co-occurrence matrix is ​​determined based on the grayscale image. The grayscale co-occurrence matrix is ​​normalized, and multiple-dimensional statistical features are extracted from the normalized grayscale co-occurrence matrix. The multiple-dimensional statistical features are then flattened into the Gray viewpoint feature vector matrix.

[0020] LBP encoding is performed on each of the reconstructed depth image datasets to obtain LBP feature maps. The LBP feature maps are mapped to a 58-dimensional histogram in a uniform pattern. The 58-dimensional histogram is divided into 1×1 blocks, and the LBP distribution of each histogram block is statistically analyzed to form the LBP viewpoint feature vector matrix.

[0021] Gradient orientation histogram is calculated for each of the reconstructed depth image datasets. The calculated gradient orientation histogram is divided into multiple cell histograms. The multiple cell histograms are normalized and then concatenated to form the HOG viewpoint feature vector matrix.

[0022] In one embodiment, the step of performing dimensionality reduction on the plurality of view feature vector matrices of each of the reconstructed depth image datasets to obtain simplified view feature sub-matrices corresponding to each view feature vector matrix includes:

[0023] For each view feature vector matrix of each reconstructed depth image dataset, the multi-order statistics of the view feature vectors in each column of the view feature vector matrix are statistically analyzed, and view feature vectors that meet the preset multi-order statistical thresholds are selected from the view feature vector matrix according to the multi-order statistics and the preset multi-order statistical thresholds to form a first candidate view feature submatrix.

[0024] Unsupervised pre-clustering is performed on all view feature vectors in the view feature vector matrix to obtain pseudo labels. The normalized mutual information score between each dimension of the view feature feature and the pseudo label in the first candidate view feature sub-matrix is ​​calculated. From the first candidate view feature sub-matrix, all view feature vectors with normalized mutual information scores not lower than a preset mutual information threshold are selected to form the second candidate view feature sub-matrix.

[0025] The correlation entropy is calculated for any two-dimensional view feature vectors in the second candidate view feature sub-matrix. Based on the greedy forward-backward floating strategy, and combined with the correlation entropy and the preset correlation entropy threshold, redundant features in the second candidate view feature sub-matrix are iteratively removed to obtain the third candidate view feature sub-matrix.

[0026] For each dimension of the view feature vector in the third candidate view feature sub-matrix, the normalized mutual information score, multi-order statistical sensitivity score, and independence score of the view feature vector are calculated respectively. The normalized mutual information score, multi-order statistical sensitivity score, and independence score of the view feature vector are then weighted and fused to obtain the comprehensive discrimination score of the view feature vector. All view feature vectors whose comprehensive discrimination score is greater than or equal to a preset comprehensive discrimination score threshold are selected to form an initial simplified view feature sub-matrix.

[0027] The initial simplified view feature submatrix is ​​subjected to cross-view consistency verification, and the initial simplified view feature submatrix is ​​adjusted for equalization based on the verification results to obtain the simplified view feature submatrix.

[0028] In one embodiment, the step of performing cross-view consistency verification on the initial simplified view feature sub-matrix and adjusting the initial simplified view feature sub-matrix for equalization based on the verification result includes:

[0029] Singular value decomposition is performed on the initial simplified perspective feature submatrix, and the condition number is determined based on the ratio of the maximum singular value to the minimum singular value obtained from the singular value decomposition.

[0030] Determine whether the number of conditions is greater than a preset threshold number of conditions;

[0031] If the condition number is determined to be greater than the preset condition number threshold, then the variance inflation factor of each view feature vector is determined according to the initial simplified view feature submatrix; view feature vectors whose variance inflation factor is greater than the preset variance inflation factor threshold are removed to obtain the first corrected simplified view feature submatrix.

[0032] Determine whether the condition number of the first modified simplified view feature submatrix is ​​greater than the preset condition number threshold;

[0033] If it is determined that the condition number of the first modified simplified view feature submatrix is ​​greater than the preset condition number threshold, then the view feature vector with the largest variance inflation factor is removed from the first modified simplified view feature submatrix until the condition number is not greater than the preset condition number threshold, and the second modified simplified view feature submatrix is ​​obtained.

[0034] The number of view feature vectors in the second modified simplified view feature submatrix corresponding to each of the view feature vector matrices is counted. The view balance index is calculated based on the absolute value of the difference between the number of view feature vectors in any two second modified simplified view feature submatrices.

[0035] If the viewpoint balance index is greater than the preset balance index threshold, then the viewpoint feature vectors of each of the second modified simplified viewpoint feature sub-matrices are added or deleted until the viewpoint balance index is not greater than the preset balance index threshold, thus obtaining the simplified viewpoint feature sub-matrices; wherein, the simplified viewpoint feature sub-matrices are not empty sets.

[0036] In one embodiment, the deep nonnegative matrix factorization framework performs hierarchical feature matrix decomposition on each of the simplified view feature sub-matrices, and iterates the decomposed feature matrices with multiple optimization objectives to obtain the optimized decomposed feature matrix, including:

[0037] For each of the simplified view feature sub-matrices, an R-layer nonnegative decomposition is performed on the simplified view feature sub-matrices based on a deep nonnegative matrix factorization framework to obtain an R-layer basis matrix and a final representation matrix; wherein, the R-layer basis matrix is ​​used to characterize the basis coordinates of the simplified view feature sub-matrices, and the final representation matrix is ​​used to characterize the coefficient matrix of the basis coordinates of the simplified view feature sub-matrices.

[0038] The optimization objectives are to minimize the non-negative decomposition error of the R-layer, minimize the hypergraph regularization proximity of the simplified view feature sub-matrix, minimize the sample deviation between any two simplified view feature sub-matrixes, and minimize the redundancy of the R-layer basis matrix. A joint objective function is constructed with the R-layer basis matrix and the final representation matrix as optimization variables.

[0039] The joint objective function is optimized by gradient descent and alternating direction multiplier method, and the optimal final representation matrix is ​​determined as the optimized decomposition feature matrix based on the optimal solution.

[0040] In one embodiment, the process of fusing the optimized decomposed feature matrices corresponding to each of the simplified viewpoint feature sub-matrices to obtain fused feature matrices corresponding to each of the reconstructed depth image datasets, and performing cluster analysis on the fused feature matrices corresponding to each of the reconstructed depth image datasets to obtain clustering results for each of the reconstructed depth image datasets, includes:

[0041] For each of the reconstructed depth image datasets, the optimized decomposed feature matrices corresponding to each of the simplified view feature sub-matrices of the reconstructed depth image datasets are summed and averaged to obtain the fusion feature matrix of the reconstructed depth image dataset.

[0042] kNN graph clustering is performed on the feature vectors of each sample in the fusion feature matrix of each reconstructed depth image dataset to obtain the clustering result of each reconstructed depth image dataset.

[0043] Secondly, the present invention also provides a multi-view clustering system for non-line-of-sight imaging data, comprising:

[0044] The dataset extraction module is used to extract reconstructed depth image datasets of multiple modal categories of the target object for recognition.

[0045] The viewpoint feature extraction module is used to extract features from multiple viewpoints for each of the reconstructed depth image datasets, and obtain multiple viewpoint feature vector matrices corresponding to each of the reconstructed depth image datasets.

[0046] The data dimensionality reduction module is used to perform data dimensionality reduction on multiple view feature vector matrices of each of the reconstructed depth image datasets to obtain simplified view feature submatrices corresponding to each view feature vector matrix.

[0047] The feature decomposition module is used to perform hierarchical feature matrix decomposition on each of the simplified perspective feature submatrices based on the deep non-negative matrix decomposition framework, and to optimize and iterate the decomposed feature matrix with multiple optimization objectives to obtain the optimized decomposed feature matrix.

[0048] The clustering module is used to fuse the optimized decomposed feature matrices corresponding to each of the simplified view feature sub-matrices to obtain the fused feature matrices corresponding to each of the reconstructed depth image datasets, and to perform clustering analysis on the fused feature matrices corresponding to each of the reconstructed depth image datasets to obtain the clustering results corresponding to each of the reconstructed depth image datasets.

[0049] Thirdly, the present invention also provides an electronic device, the electronic device including a memory and a processor, the memory storing a computer program, the computer program being executed by the processor causing the processor to perform the steps of the multi-view clustering method for non-line-of-sight imaging data as described in the first aspect.

[0050] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the steps of the multi-view clustering method for non-line-of-sight imaging data as described in the first aspect.

[0051] As can be seen from the above technical solutions, this invention extracts reconstructed depth image datasets of multiple modal categories of target recognition objects, thereby considering reconstructed depth image data under multiple reconstruction methods. It then performs feature extraction from multiple perspectives on each type of reconstructed depth image dataset, characterizing the intrinsic structural features of each dataset from different viewpoints. Furthermore, it performs dimensionality reduction on the feature vector matrices from multiple perspectives to reduce data redundancy and retain low-dimensional, clean, and highly discriminative simplified perspective feature sub-matrices. Finally, it performs joint optimization through depth non-negative matrix factorization and multiple optimization objectives, ensuring that the decomposition results at each level maintain non-negativity while satisfying multiple optimization objectives. The optimized decomposition feature matrices are then fused, and cluster analysis is performed on the fused feature matrices corresponding to each reconstructed depth image dataset, thereby achieving accurate cluster analysis for each reconstructed depth image dataset. The dimensionality reduction method improves the convergence speed during the clustering process. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is an application environment diagram of a multi-view clustering method for non-line-of-sight imaging data provided in an embodiment of the present invention;

[0054] Figure 2 A flowchart illustrating a multi-view clustering method for non-line-of-sight imaging data provided in an embodiment of the present invention;

[0055] Figure 3 This is a schematic diagram of the structure of a multi-view clustering system for non-line-of-sight imaging data provided in an embodiment of the present invention;

[0056] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0057] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] The multi-view clustering method for non-line-of-sight imaging data provided in this application embodiment can be applied to, for example... Figure 1The application environment shown is illustrated. Terminal 101 communicates with server 102 via a network. A data storage system can store the data that server 102 needs to process. The data storage system can be integrated onto server 102, or it can be located in the cloud or on another network server. Terminal 101 or server 102 executes a multi-view clustering method for non-line-of-sight imaging data. This method includes: extracting reconstructed depth image datasets of multiple modal categories of the target object; performing feature extraction from multiple views on each reconstructed depth image dataset to obtain multiple view feature vector matrices corresponding to each reconstructed depth image dataset; performing dimensionality reduction on the multiple view feature vector matrices of each reconstructed depth image dataset to obtain simplified view feature sub-matrices corresponding to each view feature vector matrix; performing hierarchical feature matrix decomposition on each simplified view feature sub-matrices based on a depth non-negative matrix factorization framework, and iteratively optimizing the decomposed feature matrices with multiple optimization objectives to obtain optimized decomposed feature matrices; fusing the optimized decomposed feature matrices corresponding to each simplified view feature sub-matrices to obtain fused feature matrices corresponding to each reconstructed depth image dataset, and performing cluster analysis on the fused feature matrices corresponding to each reconstructed depth image dataset to obtain clustering results corresponding to each reconstructed depth image dataset.

[0059] Terminal 101 can be, but is not limited to, various personal computers, laptops, smartphones, and tablets.

[0060] Server 102 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0061] like Figure 2 As shown, this application provides a multi-view clustering method for non-line-of-sight imaging data, which is applied to... Figure 1 Taking terminal 101 or server 102 as an example, the explanation includes the following steps 201 to 205. Wherein:

[0062] 201. Extract reconstructed depth image datasets of multiple modal categories of target recognition objects.

[0063] Among them, the target recognition object is the hidden object to be identified in the non-line-of-sight scene, such as the human silhouette behind the wall, the stationary obstacle at the corner, or the moving target (such as a car) behind the occlusion.

[0064] The reconstructed depth image dataset is extracted from videos acquired by non-line-of-sight imaging systems. The modal categories of the reconstructed depth image dataset in this application include LCT (Light Cone Transform) reconstructed depth image dataset, FBP (Filtered Backprojection) reconstructed depth image dataset, and NLOST (Non-Line-of-Sight Transform) reconstructed depth image dataset.

[0065] Among them, the LCT reconstructed depth image dataset models the time-of-flight signal using light conic transform to accurately restore the geometric depth distribution of hidden targets; the FBP reconstructed depth image dataset uses a filtered back-projection algorithm to suppress scattering noise and enhance the discernibility of edge structures; and the NLOST reconstructed depth image dataset is based on physics-driven iterative optimization, simulating light propagation and continuously adjusting until the most accurate result is achieved, approximating the true three-dimensional shape of the target. The three datasets exhibit complementary characteristics in terms of resolution, signal-to-noise ratio, and spatial fidelity: LCT preserves high-frequency depth transitions, FBP enhances contour continuity, and NLOST improves the consistency of internal structures.

[0066] 202. Perform feature extraction from multiple perspectives on each reconstructed depth image dataset to obtain the feature vector matrix of multiple perspectives corresponding to each reconstructed depth image dataset.

[0067] For each reconstructed depth image dataset, features are extracted from three perspectives: Gray, LBP, and HOG, forming three perspective feature vector matrices. The Gray perspective captures the spatial dependence of texture through the Gray-Level Co-occurrence Matrix (GLCM) and calculates four statistical measures: contrast, correlation, energy, and entropy, thereby reflecting the overall texture, depth uniformity, contrast, and overall texture and depth distribution of the object in the entire image.

[0068] LBP perspective captures the local contrast of light and shadow and subtle textures in each small area of ​​the image, and is resistant to noise and depth changes.

[0069] The HOG perspective focuses on the edges, outlines, and lines of an object, highlighting the shape and structure of the object in a non-viewpoint image.

[0070] By complementing each other from the above three perspectives, all the useful visual information in the non-line-of-sight depth map is extracted, laying a solid foundation for subsequent filtering and clustering.

[0071] 203. Perform dimensionality reduction on the multiple view feature vector matrices of each reconstructed depth image dataset to obtain the simplified view feature submatrices corresponding to each view feature vector matrix.

[0072] Since the original complex high-dimensional data affects the computational efficiency and model generalization ability, this application performs multiple rounds of dimensionality reduction on each view feature vector matrix to reduce dimensionality redundancy, suppress noise interference, and enhance class discriminability, thereby obtaining a clean, effective and concise view feature submatrix.

[0073] 204. Based on the deep nonnegative matrix factorization framework, the feature submatrices of each simplified perspective are decomposed into hierarchical feature matrices, and the decomposed feature matrices are optimized iteratively with multiple optimization objectives to obtain the optimized decomposed feature matrices.

[0074] Among them, the deep nonnegative matrix factorization framework is a multi-layer stacked extension of the traditional single-layer nonnegative matrix factorization (NMF). Under the constraint of full matrix nonnegativity (all matrix elements ≥ 0), it mines the hierarchical latent semantic features of data through layer-by-layer linear decomposition, taking into account both physical interpretability and deep feature abstraction capabilities. It is an unsupervised feature extraction and dimensionality reduction framework for nonnegative data (such as image intensity and depth values).

[0075] Generally, a multi-level cascaded decomposition is used to split the input data into multiple base matrices and coefficient matrices (used to represent the weights of feature combinations) layer by layer. The shallow layer extracts local details, and the deep layer extracts global abstract features.

[0076] This application performs hierarchical feature matrix decomposition on the simplified perspective feature submatrix using a deep nonnegative matrix factorization framework, and then optimizes and iterates the decomposed feature matrix by establishing multiple optimization objectives to obtain the optimized decomposed feature matrix.

[0077] 205. The optimized decomposed feature matrices corresponding to each simplified view feature submatrix are fused to obtain the fused feature matrices corresponding to each reconstructed depth image dataset. Cluster analysis is then performed on the fused feature matrices corresponding to each reconstructed depth image dataset to obtain the clustering results corresponding to each reconstructed depth image dataset.

[0078] Specifically, for each reconstructed depth image dataset, the optimized decomposed feature matrices corresponding to each simplified viewpoint feature submatrix of the reconstructed depth image dataset are fused to obtain the fused feature matrix of the reconstructed depth image dataset. This fused feature matrix integrates discriminative information from Gray, LBP, and HOG viewpoints at different semantic levels, effectively representing the spatial structure and scattering texture characteristics of hidden objects in non-line-of-sight scenes, thereby supporting subsequent clustering analysis to accurately identify the category of hidden objects. By using the fused feature matrix as an input sample and performing clustering analysis on it, the distribution of hidden object categories corresponding to the reconstructed depth image dataset can be accurately obtained.

[0079] It should be noted that, in this embodiment, reconstructed depth image datasets of multiple modal categories of target recognition objects are extracted, thereby considering reconstructed depth image data under various reconstruction methods. Multiple perspectives are used to extract features from each type of reconstructed depth image dataset, characterizing the intrinsic structural features of each dataset from different viewpoints. Then, the feature vector matrices from multiple perspectives are reduced in dimensionality to decrease data redundancy and retain low-dimensional, clean, and highly discriminative simplified perspective feature submatrices. Furthermore, joint optimization is performed through depth non-negative matrix factorization and multiple optimization objectives to ensure that the decomposition results at each level maintain non-negativity while satisfying multiple optimization objectives. The optimized decomposition feature matrices are then fused, and cluster analysis is performed on the fused feature matrices corresponding to each reconstructed depth image dataset, thereby achieving accurate cluster analysis for each reconstructed depth image dataset. The data dimensionality reduction method improves the convergence speed during the clustering process.

[0080] In some embodiments, the modal categories of the reconstructed depth image dataset include a light conic transform reconstructed depth image dataset, a filtered back projection reconstructed depth image dataset, and a non-line-of-sight transform reconstructed depth image dataset. In this case, extracting the reconstructed depth image dataset for multiple modal categories of the target object includes: acquiring non-line-of-sight imaging data of the target object; performing depth image reconstruction on the non-line-of-sight imaging data based on the LCT reconstruction algorithm to generate a light conic transform reconstructed depth image dataset; performing depth image reconstruction on the non-line-of-sight imaging data based on the FBP reconstruction algorithm to generate a filtered back projection reconstructed depth image dataset; and performing depth image reconstruction on the non-line-of-sight imaging data based on the NLOST reconstruction algorithm to generate a non-line-of-sight transform reconstructed depth image dataset.

[0081] The non-line-of-sight (NLS) imaging data for target recognition can be obtained by acquiring video files (MP4 video files containing photon intensity and depth information of the reconstructed object) of the target object under a NLS imaging system. Frame-by-frame extraction of video frames containing the target object is performed. The high-dimensional temporal data acquired by the NLS imaging system is typically a tensor of shape [T, X, Y], where T is the temporally resolved bin, and X and Y are the spatial scanning grid. Preprocessing is performed on each video file, including estimated dark marker rate denoising, normalization, and calibration, to improve data quality and the stability of subsequent feature extraction. The preprocessed video frames are then input into three reconstruction modules: LCT, FBP, and NLOST, to generate depth image datasets for the corresponding modalities.

[0082] Preferably, the LCT reconstruction algorithm defines the reconstructed image as... With a resolution of 256×256, for each spatial point The flight time t of light from the laser, hitting the object, and bouncing back to the sensor is calculated from the scanning point of the non-line-of-sight imaging system. The photon count measured at time t (transient measurement histogram) Extracting intensity values Finally, for all scan points The intensity signals are summed, and then the maximum value is projected along the depth axis z to compress the three-dimensional volume into a two-dimensional depth image, which is the final output light cone transform reconstructed depth image data, where the time of flight t is:

[0083]

[0084] in, Let c be the position of the laser and c be the speed of light.

[0085] Preferably, the FBP reconstruction algorithm is used for the temporal signal of each spatial point. (Obtained by frame decomposition of the video file) A Lamb-Lack filter is applied to remove clutter and interference, and the filtered signal is projected into the spatial domain. The filtered signal is then back-projected onto the spatial location, the data is normalized to the 0-1 range, and then downsampled using bilinear interpolation or average pooling. The 2D projected image is uniformly adjusted to a resolution of 256×256 to standardize the input size for subsequent processing, ultimately generating 256×256 resolution filtered back-projection reconstructed depth image data. ,

[0086] In the formula, This is the signal after the original time-series signal has been filtered by a Ram-Lack filter.

[0087] The NLOST reconstruction algorithm builds an imaging operator (mathematical model) that associates spatial images with sensor-measured data and iteratively optimizes them using gradient descent: continuously adjusting the image to best match the sensor's measured data; projecting the optimized 3D depth information along the Z-axis (depth direction) into a 2D image; and obtaining non-line-of-sight transformed reconstructed depth image data with a resolution of 256×256.

[0088] Specifically, we define an imaging operator A to transform the spatial image Mapped to measurement data H, where A is based on the geometry of light propagation. Then, the image is iteratively optimized using gradient descent. ,in, The reconstructed image estimate is the value obtained after optimization, where I is the 3D spatial image variable to be optimized. R(I) is the regularization weight hyperparameter, controlling the balance between reconstruction fidelity and smoothness. R(I) is the total variation (TV) regularization term, suppressing noise artifacts and preserving edge structures. H is the time-series signal measured by the sensor, iteratively updated until the convergence criterion is met, i.e., the residual is less than 10. -4Or the relative rate of change is less than 10 -5 The final output 3D reconstructed image I(x,y,z) is projected along the depth direction z to generate NLOST reconstructed depth image data with a resolution of 256×256.

[0089] In some embodiments, the view feature vector matrix includes the Gray view feature vector matrix, the LBP view feature vector matrix, and the HOG view feature vector matrix. In this case, feature extraction from multiple perspectives is performed on each reconstructed depth image dataset to obtain multiple perspective feature vector matrices corresponding to each reconstructed depth image dataset. This includes: converting each reconstructed depth image dataset into a grayscale image, determining the gray-level co-occurrence matrix based on the grayscale image, normalizing the gray-level co-occurrence matrix, extracting multiple-dimensional statistical features from the normalized gray-level co-occurrence matrix, and flattening the multiple-dimensional statistical features into a Gray perspective feature vector matrix; performing LBP encoding on each reconstructed depth image dataset to obtain an LBP feature map, mapping the LBP feature map to a 58-dimensional histogram in a uniform mode, dividing the 58-dimensional histogram into 1×1 blocks, and statistically analyzing the LBP distribution of each histogram block to form an LBP perspective feature vector matrix; calculating the gradient orientation histogram for each reconstructed depth image dataset, dividing the calculated gradient orientation histogram into multiple cell histograms, normalizing the multiple cell histograms, and concatenating the normalized multiple cell histograms into a HOG perspective feature vector matrix.

[0090] Generally, image preprocessing is also required for each reconstructed depth image dataset, including fixing its size, aligning it, and compressing it to a final normalized resolution of 64×64 for two-dimensional depth images. .

[0091] The Gray view feature vector matrix focuses on the intensity texture information of the depth map. It captures statistical features such as contrast, uniformity, energy, and correlation through the Gray-Level Co-occurrence Matrix (GLCM), making it suitable for understanding the depth distribution and texture differences in non-line-of-sight imaging data. Specifically, each reconstructed depth image dataset is converted into a grayscale image, and several offsets are selected (usually single-step offsets in four directions, such as 0°, 45°, 90°, and 135°, with a fixed offset step size of 1 pixel). For each offset... Define the elements of the co-occurrence matrix:

[0092]

[0093] In the formula, For the quantized depth image, that is, the continuous or high-order depth values ​​are mapped to discrete gray levels, the co-occurrence matrix elements are statistically analyzed, a gray-level co-occurrence matrix is ​​constructed, and the contrast, uniformity, energy and correlation statistics in the gray-level co-occurrence matrix are extracted and flattened into a series of feature numbers to form a Gray view feature vector matrix.

[0094] LBP (Layered Backpropagation) view feature vector matrix captures local binary texture patterns, suitable for nonlinear depth scattering textures in non-line-of-sight imaging data, emphasizing local depth contrast and robustness to depth range variations. It calculates the center depth value by circling eight neighboring pixels (e.g., a circle with a radius of 1) around each pixel in the image. (in, Center point ( (depth value) and neighborhood value Under the definition of radius R=1 and sampling points P=8 (if the sampling points are not on integer coordinates, bilinear interpolation is used for sampling supplementation), surrounding pixels are deeper than the center, denoted as 1, and shallower than the center, denoted as 0. This yields an 8-bit binary number, which is then converted to decimal to obtain the LBP encoding. The LBP encoding is then used to form an LBP feature map, denoted as:

[0095]

[0096] The LBP feature map is mapped to a 58-dimensional histogram in a uniform pattern. The 58-dimensional histogram is divided into 1×1 blocks. The number of different LBP values ​​in each block is calculated and counted. The statistical results are combined into a string of feature numbers to obtain the LBP perspective feature vector matrix.

[0097] The HOG viewpoint feature vector matrix captures the histogram of oriented gradients, highlighting the depth and edge structure of the image data. Specifically, it calculates the horizontal and vertical gradients in the image, where the horizontal gradient is:

[0098]

[0099] In the formula, For horizontal gradient, For the horizontal Sobel operator kernel, The horizontal gradient map is obtained after convolution; similarly, the vertical gradient map is also shown. Perform similar calculations.

[0100] Based on the horizontal and vertical gradients in the image, a gradient direction histogram is determined. The gradient magnitude and direction of change at each point are then calculated using this histogram.

[0101] The gradient magnitude is:

[0102]

[0103] The direction of change is:

[0104] ,

[0105] Divide the gradient orientation histogram into 64×64 input cells and 8×8 blocks (64 cells in total), and calculate the histogram for each cell. Where c represents the pixel region of a single cell. For the weighted gradient magnitude histogram values ​​in cell c, the distribution of gradient directions in each small cell is statistically analyzed, and multiple cell histograms are drawn. These multiple cell histograms are then normalized to standardize the features in the larger blocks, eliminating the influence of illumination and depth. The normalized multiple cell histograms are then concatenated into a HOG viewpoint feature vector matrix.

[0106] In some embodiments, dimensionality reduction is performed on multiple view feature vector matrices of each reconstructed depth image dataset to obtain simplified view feature sub-matrices corresponding to each view feature vector matrix. This includes: for each view feature vector matrix of each reconstructed depth image dataset, calculating the multi-order statistics of the view feature vectors in each column of the view feature vector matrix, and selecting view feature vectors that meet the preset multi-order statistical threshold from the view feature vector matrix based on the multi-order statistics and a preset multi-order statistical threshold to form a first candidate view feature sub-matrix; performing unsupervised pre-clustering on all view feature vectors in the view feature vector matrix to obtain pseudo-labels, and calculating the normalized mutual information score between each dimension of the view feature and the pseudo-label in the first candidate view feature sub-matrix; selecting all view feature vectors with normalized mutual information scores not lower than a preset mutual information threshold from the first candidate view feature sub-matrix to form a second candidate view feature sub-matrix; and performing unsupervised pre-clustering on all view feature vectors in the view feature vector matrix to obtain pseudo-labels, and calculating the normalized mutual information score between each dimension of the view feature and the pseudo-label in the first candidate view feature sub-matrix; and selecting all view feature vectors with normalized mutual information scores not lower than a preset mutual information threshold from the second candidate view feature sub-matrix to form a second candidate view feature sub-matrix; and performing unsupervised pre-clustering on all view feature vectors in the second candidate view feature sub-matrix to obtain pseudo-labels. The correlation entropy of any two-dimensional view feature vectors in the candidate view feature sub-matrix is ​​calculated. Then, based on a greedy forward-backward floating strategy, combined with the correlation entropy and a preset correlation entropy threshold, redundant features in the second candidate view feature sub-matrix are iteratively eliminated to obtain the third candidate view feature sub-matrix. For each dimension of the view feature vector in the third candidate view feature sub-matrix, the normalized mutual information score, multi-order statistical sensitivity score, and independence score are calculated. These scores are then weighted and fused to obtain the comprehensive discrimination score of the view feature vector. All view feature vectors with a comprehensive discrimination score greater than or equal to a preset comprehensive discrimination score threshold are selected to form the initial simplified view feature sub-matrix. Cross-view consistency verification is performed on the initial simplified view feature sub-matrix, and the initial simplified view feature sub-matrix is ​​adjusted for equalization based on the verification results to obtain the final simplified view feature sub-matrix.

[0107] To address the issues of high feature dimensionality, severe redundancy, weak discriminative power, and noise sensitivity in non-line-of-sight (NLOS) imaging data under Gray, LBP, and HOG perspectives, this application employs multiple rounds of screening to independently and in parallel execute the feature matrix for each perspective. Specifically, this includes:

[0108] For each viewpoint feature vector matrix in each reconstructed depth image dataset, perform multi-order statistics on the viewpoint feature vectors in each column of the viewpoint feature vector matrix, including the first-order mean. Second-order variance Third-order skewness With fourth-order kurtosis , Let be the value of the j-th feature at the i-th sample; N is the total number of samples.

[0109] Among them, the first-order mean is used to characterize the central tendency of the depth distribution, the second-order variance describes the dispersion of local scattering intensity, the third-order skewness reflects the left-right asymmetry of the NLOS path delay distribution, and the fourth-order kurtosis is sensitive to the energy spike characteristics of sparse echo pulses.

[0110] Subsequently, a multi-level statistical threshold was set, namely the second-order variance threshold. (like Third-order skewness threshold (e.g., 0.3) and fourth-order kurtosis threshold (e.g., 1.5), and initially retain the requirement to meet the following conditions. , or The view feature vectors under any of the following conditions constitute the first candidate view feature submatrix, thus satisfying at least one of the discriminative priors of high variance, strong skewness, or peaked distribution, laying a robust statistical foundation for the subsequent introduction of category information.

[0111] To overcome the problem of pure statistics ignoring category information, this application further performs unsupervised pre-clustering (such as k-means clustering) on ​​all view feature vectors in the view feature vector matrix to obtain pseudo-labels. Specifically, the cluster assignment labels obtained by performing k-means clustering (with the number of categories set to the preset clustering target number) on all samples of the current view are in the form of an integer vector of length N. This label is obtained through unsupervised estimation and serves as approximate category information for subsequent mutual information scoring.

[0112] The normalized mutual information score between each dimension of the view feature feature and the pseudo-label in the first candidate view feature sub-matrix is ​​obtained as follows:

[0113]

[0114] In the formula, H(f_i) represents the mutual information between the i-th dimension feature in the first candidate perspective feature submatrix and the pseudo-label y_pseudo. H(f_i) represents the information entropy of the feature and is used for normalization to eliminate dimensional differences.

[0115] Subsequently, a mutual information threshold ΘMI is set, specifically obtained using an adaptive threshold determination method based on the Otsu algorithm. Specifically, the normalized mutual information scores of all features under the current viewpoint are treated as a set of pixel values ​​in a one-dimensional grayscale image (i.e., the number of features is treated as the total number of pixels). Their statistical frequency distribution forms a one-dimensional histogram. The classic Otsu algorithm is then applied to this one-dimensional histogram to automatically search for the difference between class variances. The largest grayscale segmentation point T otsu To further enhance the retention rate of highly discriminative features and avoid the Otsu algorithm being too aggressive when the histogram approximates a single peak, this application sets the mutual information threshold ΘMI as follows:

[0116]

[0117] In the formula, Q75 and Q25 are the 75th and 25th percentiles of MI_score, respectively. This formula ensures that a pure Otsu threshold T is used when the histogram exhibits obvious bimodality. otsu When the distribution is close to flat or unimodal, it automatically degenerates to a conservative threshold based on the interquartile range, thus maintaining robustness on all non-line-of-sight datasets.

[0118] After obtaining the mutual information threshold, retain all view feature vectors that satisfy MI_score(i) ≥ΘMI to form the second candidate view feature submatrix.

[0119] Furthermore, this application employs a self-defined "correlation entropy" as a nonlinear, nonparametric redundancy metric to overcome the shortcomings of traditional Pearson correlation coefficients or mutual information, which are sensitive to noise and cannot effectively capture nonlinear collinearity. Specifically, the correlation entropy is calculated for any two-dimensional view feature vectors in the second candidate view feature submatrix. Defined as:

[0120]

[0121] In the formula, for and The probability density of the rank difference sequence of the sorted samples A Gaussian kernel function is used for estimation, and the kernel width σ is adaptively determined using the Silverman rule or the maximum likelihood criterion. After calculating the correlation entropy matrix of all feature pairs, the following greedy forward-backward floating strategy is adopted:

[0122] (1) Sort the features in the second candidate perspective feature submatrix C from high to low according to their mutual information score MI_score to form an ordered sequence L;

[0123] (2) Initialize the selected feature set S to be empty;

[0124] (3) Take the feature f_cur with the highest current score from the head of the sequence L that has never been removed: a. If S is empty, add f_cur directly to S; b. If S is not empty, calculate the correlation entropy CE(f_cur,f_sel) between f_cur and each selected feature f_sel in S; c. If there exists any f_sel such that CE(f_cur,f_sel)>ΘCE and MI_score(f_cur)<MI_score(f_sel), then permanently remove fcur and continue to the next feature; d. Otherwise, add f_cur to S;

[0125] (4) After each new feature is added, a backward check is triggered: the correlation entropy between all current features in S and the latest f_cur is recalculated. If it is found that a low-scoring feature is completely redundant with a higher-scoring feature (satisfying the condition that the same CE>Θ_CE), the low-scoring feature is removed from S.

[0126] (5) Repeat steps (3) to (4) until sequence L is traversed.

[0127] The redundancy threshold ΘCE is adaptively determined in the interval [0.75, 0.95] using the Otsu algorithm or the interquartile range criterion.

[0128] Through the greedy forward-backward floating redundancy elimination mechanism based on correlation entropy, this module can effectively eliminate linear and nonlinear redundancy, collinear features, and pseudo-correlation features caused by NLOS noise, while ensuring that the highest discriminative features are retained first. This makes the final output third candidate perspective feature submatrix have both extremely high independence and minimal information overlap.

[0129] To completely eliminate the risk of over- or under-removal caused by fixed thresholds or single criteria in the aforementioned modules, this application further designs a fully data-driven comprehensive discriminative dynamic threshold adaptive process. This process filters each dimension of the perspective feature vector in the third candidate perspective feature sub-matrix. Specifically, for each dimension of the perspective feature vector i in the third candidate perspective feature sub-matrix, the normalized mutual information score, multi-order statistical sensitivity score, and independence score of the perspective feature vector are calculated. The multi-order statistical sensitivity score is obtained by taking the maximum value after Z-score standardization of the second-order variance, third-order skewness, and fourth-order kurtosis. The independence score is obtained by calculating the maximum value of the correlation entropy between the feature and the retained feature set and taking the complement, ensuring that the closer it is to 1, the more independent it is.

[0130] Then, the normalized mutual information score, multi-order statistical sensitivity score, and independence score of the viewpoint feature vector are weighted and fused to obtain the comprehensive discrimination score of the viewpoint feature vector:

[0131] Score_final(i)=α·M(i)+β·S(i)+γ·R(i)

[0132] In the formula, the weight coefficients α, β, and γ satisfy α + β + γ = 1, and preferably α ≥ 0.5 (emphasizing discriminability priority). For example, the weights can be directly fixed as α = 0.6, β = 0.25, and γ = 0.15. M(i), S(i), and R(i) correspond to the normalized mutual information score, the multi-order statistic sensitivity score, and the independence score, respectively.

[0133] Subsequently, a comprehensive discrimination score threshold is set. Specifically, the Score_final(i) of all features is treated as a set of pixel values ​​in a one-dimensional "grayscale image," and the following improved multi-threshold Otsu algorithm is executed: A multi-threshold Otsu algorithm based on maximizing inter-class variance is adopted (extended to 2-4 thresholds), automatically searching for the threshold combination {T1, T2, …, T{k-1}} that maximizes the global inter-class variance; if the histogram shows obvious multi-peaks (kurtosis > 3 or ≥ 3 local maxima are detected), the highest threshold T_{max} = T_{k-1} is directly used as the final segmentation point; if the histogram is close to flat or unimodal (kurtosis ≤ 1.5), it automatically degenerates to a conservative threshold, i.e.:

[0134]

[0135] The final dynamic threshold determination rule, this application adopts the following fusion strategy to determine the unique comprehensive discrimination score threshold:

[0136] Θ_final = max(T_{max}, T_{conservative}, knee value of Score_final)

[0137] The knee point value is obtained through the Kneedle algorithm or the detection of the maximum curvature point of the second derivative.

[0138] By taking the largest value from the above three factors, we can ensure that: when the discriminative distribution is clear, the optimal feature is strictly preserved; when the distribution is ambiguous or the noise is too great, the threshold is automatically relaxed to avoid erroneously removing key weak features.

[0139] After obtaining Θ_final, only the view feature vectors that satisfy Score_final(i)≥Θ_final are retained to form the initial simplified view feature submatrix.

[0140] Under the aforementioned screening rules, the features of each perspective can be compressed to a very low dimension while maintaining high discriminativeness. In order to ensure that the output features are both numerically stable and physically complete, and that the contributions of the three perspectives are balanced, this invention further performs cross-perspective consistency verification on the initial simplified perspective feature sub-matrix, and adjusts the initial simplified perspective feature sub-matrix for equalization based on the verification results, so as to ensure that the output features are both numerically stable and physically complete, and that the contributions of the three perspectives are balanced.

[0141] Specifically, the initial simplified perspective feature submatrix undergoes cross-perspective consistency verification, and is then adjusted for equalization based on the verification results. This includes: performing singular value decomposition on the initial simplified perspective feature submatrix; determining the condition number based on the ratio of the maximum to the minimum singular value obtained from the singular value decomposition; determining whether the condition number is greater than a preset condition number threshold; if the condition number is greater than the preset condition number threshold, determining the variance inflation factor of each perspective feature vector based on the initial simplified perspective feature submatrix; removing perspective feature vectors with variance inflation factors greater than the preset variance inflation factor threshold to obtain the first corrected simplified perspective feature submatrix; determining whether the condition number of the first corrected simplified perspective feature submatrix is ​​greater than the preset condition number threshold; if the condition number of the first corrected simplified perspective feature submatrix is ​​greater than the preset condition number threshold, then... If the number of condition vectors exceeds a preset condition number threshold, then from the first modified simplified view feature submatrix, the view feature vector with the largest variance inflation factor is removed until the condition number is no greater than the preset condition number threshold, resulting in the second modified simplified view feature submatrix. The number of view feature vectors in the second modified simplified view feature submatrix corresponding to each view feature vector matrix is ​​counted. Based on the absolute value of the difference between the number of view feature vectors in any two second modified simplified view feature submatrixes, the view balance index is calculated. If the view balance index is greater than a preset balance index threshold, then the view feature vectors in each second modified simplified view feature submatrix are added or deleted until the view balance index is no greater than the preset balance index threshold, resulting in the simplified view feature submatrix. The simplified view feature submatrix is ​​not an empty set.

[0142] The condition number reflects the ill-conditioning of the initial simplified view feature matrix. The smaller the value, the higher the numerical stability of the matrix. The initial simplified view feature matrix is ​​decomposed by singular value decomposition. The condition number is determined by the ratio of the maximum singular value to the minimum singular value obtained by singular value decomposition. If the condition number is greater than the preset condition number threshold (e.g., 10), the matrix is ​​judged to have serious multicollinearity, and the variance inflation factor (VIF) iterative elimination mechanism needs to be activated. If the condition number is not greater than the preset condition number threshold, the initial simplified view feature matrix is ​​directly retained as the input for subsequent view equalization adjustment.

[0143] Among them, the variance inflation factor (VIF) reflects the variance amplification factor of a linear combination of a feature with other features. The larger the VIF value, the stronger the redundancy of the feature. This invention sets the VIF threshold to 5. When the VIF of a feature is greater than 5, it is judged as highly redundant and is removed first. Each time, only the feature with the largest VIF is removed. Then, the VIF of the remaining features is recalculated to avoid chain-reaction deletion and ensure that the matrix condition number decreases monotonically after each iteration and eventually converges.

[0144] The variance inflation factor is calculated using the coefficient of determination obtained from a multiple linear regression with one feature in the initial simplified feature matrix as the dependent variable and the remaining features as independent variables. The coefficient of determination R of this regression model is used to calculate... .

[0145] The first modified simplified view feature submatrix is ​​obtained by removing view feature vectors whose variance inflation factor is greater than the preset variance inflation factor threshold. Then, its condition number is further determined. If it is still greater than the threshold, the VIF iteration is repeated until convergence. Finally, the second modified simplified view feature submatrix is ​​obtained, whose condition number is strictly controlled within 10 and all feature VIFs are ≤5 to ensure numerical stability and low redundancy.

[0146] Then, in order to make the three perspective dimensions basically consistent and the contribution balanced, this application also counts the number of perspective feature vectors in the second modified simplified perspective feature submatrix corresponding to each perspective feature vector matrix, and calculates the perspective balance index by the absolute value of the difference between the number of perspective feature vectors in any two second modified simplified perspective feature submatrixes to reflect the degree of difference between the number of perspective feature vectors in two perspectives. If the perspective balance index is greater than the preset balance index threshold (such as 5), then the perspective feature vectors of the two second modified simplified perspective feature submatrixes are added or deleted.

[0147] Generally, feature interpolation can be used to expand a simplified view feature submatrix with a small number of view feature vectors. This is done by adding features to the simplified view feature submatrix from the self-selection pool according to the aforementioned comprehensive discrimination score from high to low, until the view balance index between the number of view feature vectors and the number of view feature vectors does not exceed a preset balance index threshold. The self-selection pool is composed of features from the original view feature vector matrix that were not selected for the simplified view feature submatrix, and each candidate feature is reordered based on the comprehensive discrimination score.

[0148] In addition, the simplified view feature submatrix with a large number of view feature vectors can be pruned, prioritizing the removal of redundant features with the lowest comprehensive discrimination score and VIF closest to the threshold, ensuring that the numerical stability and physical category integrity constraints are still met after pruning. All addition and deletion operations synchronously update the simplified view feature submatrix of all views, making the dimensionality of the simplified view feature submatrix of all views basically consistent and the contribution balanced. Finally, three compact, non-redundant, highly discriminative, and mutually synergistic feature subspaces are formally encapsulated and named as F_GLCM, F_LBP, and F_HOG, which can retain all the effective information of non-line-of-sight imaging concealed targets to the greatest extent, providing theoretically optimal input for subsequent multi-view clustering or deep non-negative matrix factorization.

[0149] It should be noted that the simplified view feature submatrix of each viewpoint needs to retain at least one dimension of features to avoid the accidental deletion of key physical information by pure data-driven approaches. That is, each simplified view feature submatrix is ​​not an empty set.

[0150] In some embodiments, based on a deep nonnegative matrix factorization framework, each simplified view feature submatrix is ​​subjected to hierarchical feature matrix decomposition, and the decomposed feature matrices are iteratively optimized with multiple optimization objectives to obtain the optimized decomposed feature matrix. This includes: for each simplified view feature submatrix, performing R-level nonnegative decomposition based on the deep nonnegative matrix factorization framework to obtain an R-level basis matrix and a final representation matrix; wherein the R-level basis matrix is ​​used to characterize the basis coordinates of the simplified view feature submatrix, and the final representation matrix is ​​used to characterize the coefficient matrix of the basis coordinates of the simplified view feature submatrix; using minimizing the R-level nonnegative decomposition error, minimizing the hypergraph regularized proximity of the simplified view feature submatrix, minimizing the sample deviation between any two simplified view feature submatrices, and minimizing the redundancy of the R-level basis matrix as multiple optimization objectives, and using the R-level basis matrix and the final representation matrix as optimization variables, a joint objective function is constructed; the joint objective function is optimized using gradient descent and alternating direction multiplier methods, and the optimal final representation matrix is ​​determined as the optimized decomposed feature matrix based on the optimal solution.

[0151] Specifically, for each simplified view feature submatrix, an R-layer nonnegative decomposition is performed on the simplified view feature submatrix based on a deep nonnegative matrix factorization framework, decomposing it into an R-layer basis matrix. and the final representation matrix Here, c and n represent the preset number of clusters and the total number of samples in the dataset, respectively. (Constraints) To maintain nonnegativity and ensure physical meaning, the target generates a hierarchical feature representation. Ultimately, it is merged into a multi-view depth representation. , dimension n × c, where, Let be the weight coefficients for viewpoint p. Specifically, for each viewpoint p, the data matrix is ​​decomposed layer by layer to obtain:

[0152]

[0153] Among them, each layer It represents intermediate representations and abstracts semantics layer by layer (shallow layers capture local depth textures, deep layers capture global object structures).

[0154] Then, a multi-optimization objective is established, with minimizing the nonnegative decomposition error of the R-layer, minimizing the hypergraph regularization proximity of the simplified view feature matrix, minimizing the sample deviation between any two simplified view feature matrices, and minimizing the redundancy of the R-layer basis matrix as the optimization objectives. The R-layer basis matrix and the final representation matrix are used as optimization variables to construct a joint objective function. The joint objective function is:

[0155]

[0156] In the formula, To optimize the target value overall, The non-negative decomposition error value of the R-layer is... For hypergraph regularization of neighbor values, The sample deviation value between the two simplified view feature submatrices. The redundancy of the R-layer basis matrix.

[0157] Among them, use - Norm measures the R-level nonnegative decomposition error to be robust to outliers in NLOS data; that is, the R-level nonnegative decomposition error value is:

[0158]

[0159] In the formula, For action - The sum of norms, prioritizing the suppression of noisy rows rather than individual elements, where m is the number of columns in the matrix.

[0160] To capture higher-order relationships between samples, a hypergraph is constructed by simplifying the viewpoint feature submatrix, denoted as G = (Y, E, W), where Y is the set of sample vertices (|Y|=n), E is the set of hyperedges, and W is the hyperedge weight matrix. A hot kernel function is used. σ is the bandwidth parameter of the heat kernel function, adaptively determined based on the average distance of the dataset, and e is the hyperedge. Define the indicator matrix. If the vertex , Let H be the j-th hyperedge in the hypergraph G=(Y,E,W). ij =1, otherwise 0; Vertex degree w is the diagonal element in the hyperedge weight matrix, representing the weight of the j-th hyperedge, and the hyperedge degree. The Laplacian matrix of the normalized hypergraph is calculated as follows:

[0161]

[0162] In the formula, , L is the normalized hypergraph Laplacian matrix, and I is the identity matrix. Let H be the vertex degree diagonal matrix, H be the hypergraph indicator matrix, W be the hyperedge weight diagonal matrix, and D be the hypergraph weight matrix. e It is a diagonal matrix with a side length.

[0163] Regularizing the hypergraph representation to preserve higher-order geometric structure yields:

[0164]

[0165] In the formula, tr(·) is the trace operation, which encourages similar samples to remain close in the representation space. Higher-order correlations improve the capture of complex NLOS scattering. The weight hyperparameters for hypergraph regularization control the strength of hypergraph constraints.

[0166] By introducing pairwise consistency learning, complementary information from multiple views is mined to obtain the sample deviation between two simplified view feature submatrices. for:

[0167]

[0168] In the formula, To control the hyperparameters of the weights, Let p be the final representation matrix of the Rth layer. Let be the final representation matrix of the Rth layer from viewpoint q.

[0169] Introduction of non-line-of-sight imaging datasets - Norm sparsification of the basis matrices at each layer avoids feature redundancy and enhances interpretability, resulting in the following redundancy of the R-layer basis matrix:

[0170] In the formula, Here, R is a hyperparameter, and R is the total number of layers in the DNMF decomposition. This forces the basis matrix to tend towards a sparse pattern, highlighting class-specific features.

[0171] Gradient descent combined with alternating direction multipliers introduces an augmented Lagrangian function, transforming constrained optimization into a sequence of unconstrained subproblems; each iteration updates... and And verify its convergence on the NLOS reconstruction dataset. Specifically, first fix the final representation matrix. And for each view Each layer of the base matrix Solve the subproblems, using - Norm reconstruction error enhances robustness; diagonal weight matrix is ​​introduced. Process noisy samples and combine them with exponentially decaying... -norm sparse terms Shallow basis vector sparsity is achieved by deriving the multiplicative update rule using the Lagrange multiplier method:

[0172]

[0173] in, and These are the auxiliary matrices for the forward and backward product, respectively.

[0174] Subsequently, fix all basis matrices. And for the final layer coefficient matrix Solve the subproblems and integrate them. - Norm reconstruction, cross-vision Figure 1 Consistent learning items Hypergraph regularization term For view (The Laplacian matrix of the hypergraph). Similarly, diagonal weights are introduced. And derive the multiplicative update rule:

[0175]

[0176] In the formula, It is a product of all basis matrices.

[0177] The entire iteration has a relative change in the overall objective function of less than The solution converges in time and eventually outputs the optimal solution. and In subsequent clustering, this application only applies the final representation matrix. Therefore, the optimal final representation matrix is ​​determined. As the optimized decomposition feature matrix.

[0178] In some embodiments, the optimized decomposed feature matrices corresponding to each simplified viewpoint feature submatrix are fused to obtain the fused feature matrices corresponding to each reconstructed depth image dataset, and cluster analysis is performed on the fused feature matrices corresponding to each reconstructed depth image dataset to obtain the clustering results corresponding to each reconstructed depth image dataset. This includes: for each reconstructed depth image dataset, summing and averaging the optimized decomposed feature matrices corresponding to each simplified viewpoint feature submatrix of the reconstructed depth image dataset to obtain the fused feature matrix of the reconstructed depth image dataset; and performing kNN graph clustering on the feature vectors of each sample in the fused feature matrix of each reconstructed depth image dataset to obtain the clustering results of each reconstructed depth image dataset.

[0179] Since each reconstructed depth image dataset has three simplified view feature sub-matrices from different perspectives, the fused feature matrix of the reconstructed depth image dataset, also known as the consensus coefficient matrix, is obtained by summing and averaging the optimized decomposed feature matrices corresponding to these three simplified view feature sub-matrices.

[0180]

[0181] Then, the consensus coefficient matrix is ​​used as the sample input for kNN graph clustering. kNN graph clustering is used to cluster each element in the consensus coefficient matrix, and finally, a high-precision non-line-of-sight target spatial distribution clustering result is generated.

[0182] The process of kNN graph clustering is as follows:

[0183] 1) Construct a kNN nearest neighbor graph: For each sample in the consensus coefficient matrix, calculate its feature distance to all other samples; for each sample, connect it with only the sample with the smallest feature distance by a line (edge) to form a kNN nearest neighbor graph, where the points in the graph are samples and the lines are similarity connections between samples.

[0184] 2) Calculate the similarity weight of the graph: assign weights to each edge according to the size of the feature distance, with the smaller the distance, the greater the weight, to obtain the weighted kNN nearest neighbor graph.

[0185] 3) Generate graph structure features: Based on the weighted kNN nearest neighbor graph, calculate the graph Laplacian matrix, which describes the topological relationship between samples and is converted into standardized structural features.

[0186] 4) Perform final clustering (k-means): Perform k-means clustering on the graph structure features and finally output the cluster labels of each sample to accurately locate the spatial distribution clusters of NLOS hidden targets.

[0187] The proposed method first maps non-line-of-sight data into three explicit perspectives: Gray, LBP, and HOG. It then introduces an adaptive feature selection method using a "joint threshold of higher-order statistics," achieving a dual reduction of noise and redundancy. This significantly compresses dimensionality while preserving cross-modal discriminative information, substantially compressing the original high-dimensional non-line-of-sight data. Simultaneously, it effectively removes salt-and-pepper and modality aliasing noise, significantly improving clustering accuracy. At the decomposition end, a weighted sparsity mechanism is introduced to prune redundant basis vectors layer by layer. Learnable gating vectors are used to dynamically evaluate the importance of elements in each layer's basis matrix, in conjunction with... / The mixed norm hard threshold makes each base layer automatically tend towards a class-specific sparse mode during training, making the hidden representation more compact and energy more concentrated, and significantly shortening the time consumed by clustering.

[0188] Based on the same inventive concept, this application also provides a multi-view clustering system for non-line-of-sight imaging data for implementing the multi-view clustering method for non-line-of-sight imaging data involved above.

[0189] The solution provided by this system is similar to the solution described in the above method. Therefore, the specific limitations of one or more multi-view clustering system embodiments for non-line-of-sight imaging data provided below can be found in the limitations of the multi-view clustering method for non-line-of-sight imaging data described above, and will not be repeated here.

[0190] like Figure 3 As shown in the embodiments of this application, a multi-view clustering system for non-line-of-sight imaging data is also provided, including:

[0191] The dataset extraction module 100 is used to extract reconstructed depth image datasets of multiple modal categories of the target recognition object;

[0192] The view feature extraction module 200 is used to extract features from multiple views for each reconstructed depth image dataset, and obtain multiple view feature vector matrices corresponding to each reconstructed depth image dataset.

[0193] The data dimensionality reduction module 300 is used to reduce the dimensionality of multiple view feature vector matrices in each reconstructed depth image dataset, so as to obtain the simplified view feature submatrices corresponding to each view feature vector matrix.

[0194] The feature decomposition module 400 is used to perform hierarchical feature matrix decomposition on each simplified perspective feature submatrix based on the deep non-negative matrix decomposition framework, and to optimize and iterate the decomposed feature matrix with multiple optimization objectives to obtain the optimized decomposed feature matrix.

[0195] Clustering module 500 is used to fuse the optimized decomposed feature matrices corresponding to each simplified view feature submatrix to obtain the fused feature matrices corresponding to each reconstructed depth image dataset, and to perform clustering analysis on the fused feature matrices corresponding to each reconstructed depth image dataset to obtain the clustering results corresponding to each reconstructed depth image dataset.

[0196] In some embodiments, the modal categories of the reconstructed depth image dataset include light conic transform reconstructed depth image dataset, filtered back projection reconstructed depth image dataset, and non-line-of-sight transform reconstructed depth image dataset.

[0197] Dataset extraction module 100 is used for:

[0198] Acquire non-line-of-sight imaging data of the target object;

[0199] Based on the LCT reconstruction algorithm, depth image reconstruction is performed on non-line-of-sight imaging data to generate a light cone transformation reconstructed depth image dataset.

[0200] Based on the FBP reconstruction algorithm, depth image reconstruction is performed on non-line-of-sight imaging data to generate a filtered back-projection reconstructed depth image dataset.

[0201] The NLOST reconstruction algorithm is used to reconstruct depth images from non-line-of-sight imaging data, generating a non-line-of-sight transformation reconstructed depth image dataset.

[0202] In some embodiments, the view feature vector matrix includes the Gray view feature vector matrix, the LBP view feature vector matrix, and the HOG view feature vector matrix.

[0203] Viewpoint feature extraction module 200, used for:

[0204] Each reconstructed depth image dataset is converted into a grayscale image, and a grayscale co-occurrence matrix is ​​determined based on the grayscale image. The grayscale co-occurrence matrix is ​​normalized, and multiple-dimensional statistical features are extracted from the normalized grayscale co-occurrence matrix. The multiple-dimensional statistical features are then flattened into a Gray viewpoint feature vector matrix.

[0205] LBP encoding is performed on each reconstructed depth image dataset to obtain LBP feature maps. The LBP feature maps are then mapped to a 58-dimensional histogram in a uniform pattern. The 58-dimensional histogram is divided into 1×1 blocks, and the LBP distribution of each histogram block is statistically analyzed to form an LBP viewpoint feature vector matrix.

[0206] For each reconstructed depth image dataset, a gradient orientation histogram is calculated. The calculated gradient orientation histogram is divided into multiple cell histograms, and the multiple cell histograms are normalized. The normalized multiple cell histograms are then concatenated into a HOG viewpoint feature vector matrix.

[0207] In some embodiments, the data dimensionality reduction module 300 is used for:

[0208] For each view feature vector matrix of each reconstructed depth image dataset, the multi-order statistics of the view feature vectors in each column of the view feature vector matrix are statistically analyzed. Based on the multi-order statistics and the preset multi-order statistical threshold, view feature vectors that meet the preset multi-order statistical threshold are selected from the view feature vector matrix to form the first candidate view feature submatrix.

[0209] Unsupervised pre-clustering is performed on all view feature vectors in the view feature vector matrix to obtain pseudo labels. The normalized mutual information score between each dimension of the view feature and the pseudo label in the first candidate view feature sub-matrix is ​​calculated. From the first candidate view feature sub-matrix, all view feature vectors with normalized mutual information scores not lower than the preset mutual information threshold are selected to form the second candidate view feature sub-matrix.

[0210] The correlation entropy is calculated for any two-dimensional view feature vectors in the second candidate view feature submatrix. Then, based on the greedy forward-backward floating strategy, and combined with the correlation entropy and the preset correlation entropy threshold, redundant features in the second candidate view feature submatrix are iteratively removed to obtain the third candidate view feature submatrix.

[0211] For each dimension of the view feature vector in the third candidate view feature sub-matrix, calculate the normalized mutual information score, multi-order statistical sensitivity score, and independence score of the view feature vector. Then, perform weighted fusion on the normalized mutual information score, multi-order statistical sensitivity score, and independence score of the view feature vector to obtain the comprehensive discrimination score of the view feature vector. Select all view feature vectors whose comprehensive discrimination score is greater than or equal to the preset comprehensive discrimination score threshold to form the initial simplified view feature sub-matrix.

[0212] Cross-view consistency verification is performed on the initial simplified view feature submatrix, and the initial simplified view feature submatrix is ​​adjusted for equalization based on the verification results to obtain the simplified view feature submatrix.

[0213] In some embodiments, the data dimensionality reduction module 300 is used for:

[0214] Singular value decomposition is performed on the initial simplified perspective feature submatrix, and the condition number is determined based on the ratio of the maximum singular value to the minimum singular value obtained from the singular value decomposition.

[0215] Determine whether the number of conditions is greater than a preset threshold.

[0216] If the number of judgment conditions is greater than the preset number of judgment conditions threshold, the variance inflation factor of each view feature vector is determined according to the initial simplified view feature submatrix; view feature vectors with variance inflation factors greater than the preset variance inflation factor threshold are removed to obtain the first corrected simplified view feature submatrix.

[0217] Determine whether the condition number of the first modified simplified view feature submatrix is ​​greater than the preset condition number threshold;

[0218] If the condition number of the first modified simplified perspective feature submatrix is ​​greater than the preset condition number threshold, then the perspective feature vector with the largest variance inflation factor is removed from the first modified simplified perspective feature submatrix until the condition number is not greater than the preset condition number threshold, and the second modified simplified perspective feature submatrix is ​​obtained.

[0219] The number of view feature vectors in the second modified simplified view feature submatrix corresponding to each view feature vector matrix is ​​counted. The view balance index is calculated based on the absolute value of the difference between the number of view feature vectors in any two second modified simplified view feature submatrixes.

[0220] If the viewpoint balance index is greater than the preset balance index threshold, then the viewpoint feature vectors of each of the second modified simplified viewpoint feature submatrices are added or deleted until the viewpoint balance index is not greater than the preset balance index threshold, thus obtaining a simplified viewpoint feature submatrix; wherein, the simplified viewpoint feature submatrix is ​​not an empty set.

[0221] In some embodiments, the feature decomposition module 400 is used for:

[0222] For each simplified view feature submatrix, an R-level nonnegative decomposition is performed on the simplified view feature submatrix based on the deep nonnegative matrix decomposition framework to obtain the R-level basis matrix and the final representation matrix; wherein, the R-level basis matrix is ​​used to characterize the basis coordinates of the simplified view feature submatrix, and the final representation matrix is ​​used to characterize the coefficient matrix of the basis coordinates of the simplified view feature submatrix.

[0223] The goal is to minimize the non-negative decomposition error of the R-layer, minimize the hypergraph regularization proximity of the simplified view feature matrix, minimize the sample deviation between any two simplified view feature matrices, and minimize the redundancy of the R-layer basis matrix. A joint objective function is constructed with the R-layer basis matrix and the final representation matrix as optimization variables.

[0224] The joint objective function is optimized by gradient descent and alternating direction multiplier method. The optimal final representation matrix is ​​determined based on the optimal solution as the optimized decomposition feature matrix.

[0225] In some embodiments, the clustering module 500 is used for:

[0226] For each reconstructed depth image dataset, the optimized decomposed feature matrices corresponding to each simplified view feature submatrix of the reconstructed depth image dataset are summed and averaged to obtain the fused feature matrix of the reconstructed depth image dataset.

[0227] kNN graph clustering is performed on the feature vectors of each sample in the fusion feature matrix of each reconstructed depth image dataset to obtain the clustering results of each reconstructed depth image dataset.

[0228] like Figure 4 As shown, this application provides an electronic device. The electronic device 10 includes a memory 20 and a processor 30. The memory 20 stores a computer program. When the computer program is executed by the processor 30, the processor 30 performs the steps of the multi-view clustering method for non-line-of-sight imaging data as described in the above embodiment.

[0229] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed, implements the steps of the multi-view clustering method for non-line-of-sight imaging data as described in the above embodiments.

[0230] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, electronic devices, and computer storage media described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0231] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0232] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0233] In the embodiments provided by this invention, it should be understood that the disclosed systems, electronic devices, computer storage media, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0234] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0235] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0236] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for executing all or part of the steps of the methods described in the various embodiments of the present invention through a computer device (which may be a personal computer, a server, or a network device, etc.). The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0237] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-view clustering method for non-line-of-sight imaging data, characterized in that, include: Extract reconstructed depth image datasets of objects with multiple modal categories for target recognition; For each reconstructed depth image dataset, feature extraction is performed from multiple perspectives to obtain a feature vector matrix corresponding to each reconstructed depth image dataset. For each of the reconstructed depth image datasets, the multiple view feature vector matrices are subjected to dimensionality reduction to obtain the simplified view feature submatrices corresponding to each view feature vector matrix. Based on the deep nonnegative matrix factorization framework, the simplified view feature submatrices are decomposed into hierarchical feature matrices, and the decomposed feature matrices are optimized iteratively with multiple optimization objectives to obtain the optimized decomposed feature matrices. The optimized decomposed feature matrices corresponding to each of the simplified view feature sub-matrices are fused to obtain the fused feature matrices corresponding to each of the reconstructed depth image datasets. Cluster analysis is then performed on the fused feature matrices corresponding to each of the reconstructed depth image datasets to obtain the clustering results corresponding to each of the reconstructed depth image datasets.

2. The multi-view clustering method for non-line-of-sight imaging data according to claim 1, characterized in that, The modal categories of the reconstructed depth image dataset include light conic transform reconstructed depth image dataset, filtered back projection reconstructed depth image dataset, and non-line-of-sight transform reconstructed depth image dataset. The reconstructed depth image dataset for extracting multiple modal categories of the target object includes: Acquire non-line-of-sight imaging data of the target object; The non-line-of-sight imaging data is reconstructed using the LCT reconstruction algorithm to generate the light conic transform reconstructed depth image dataset. The non-line-of-sight imaging data is reconstructed using the FBP reconstruction algorithm to generate the filtered back-projection reconstructed depth image dataset. The non-line-of-sight imaging data is reconstructed using the NLOST reconstruction algorithm to generate the non-line-of-sight transformed reconstructed depth image dataset.

3. The multi-view clustering method for non-line-of-sight imaging data according to claim 1, characterized in that, The view feature vector matrix includes the Gray view feature vector matrix, the LBP view feature vector matrix, and the HOG view feature vector matrix; The step of extracting features from multiple perspectives for each of the reconstructed depth image datasets to obtain a feature vector matrix from multiple perspectives corresponding to each of the reconstructed depth image datasets includes: Each of the reconstructed depth image datasets is converted into a grayscale image, and a grayscale co-occurrence matrix is ​​determined based on the grayscale image. The grayscale co-occurrence matrix is ​​normalized, and multiple-dimensional statistical features are extracted from the normalized grayscale co-occurrence matrix. The multiple-dimensional statistical features are then flattened into the Gray viewpoint feature vector matrix. LBP encoding is performed on each of the reconstructed depth image datasets to obtain LBP feature maps. The LBP feature maps are mapped to a 58-dimensional histogram in a uniform pattern. The 58-dimensional histogram is divided into 1×1 blocks, and the LBP distribution of each histogram block is statistically analyzed to form the LBP viewpoint feature vector matrix. Gradient orientation histogram is calculated for each of the reconstructed depth image datasets. The calculated gradient orientation histogram is divided into multiple cell histograms. The multiple cell histograms are normalized and then concatenated to form the HOG viewpoint feature vector matrix.

4. The multi-view clustering method for non-line-of-sight imaging data according to claim 1, characterized in that, The step of performing dimensionality reduction on the multiple viewpoint feature vector matrices of each of the reconstructed depth image datasets to obtain simplified viewpoint feature sub-matrices corresponding to each viewpoint feature vector matrix includes: For each view feature vector matrix of each reconstructed depth image dataset, the multi-order statistics of the view feature vectors in each column of the view feature vector matrix are statistically analyzed, and view feature vectors that meet the preset multi-order statistical thresholds are selected from the view feature vector matrix according to the multi-order statistics and the preset multi-order statistical thresholds to form a first candidate view feature submatrix. Unsupervised pre-clustering is performed on all view feature vectors in the view feature vector matrix to obtain pseudo labels. The normalized mutual information score between each dimension of the view feature feature and the pseudo label in the first candidate view feature sub-matrix is ​​calculated. From the first candidate view feature sub-matrix, all view feature vectors with normalized mutual information scores not lower than a preset mutual information threshold are selected to form the second candidate view feature sub-matrix. The correlation entropy is calculated for any two-dimensional view feature vectors in the second candidate view feature sub-matrix. Based on the greedy forward-backward floating strategy, and combined with the correlation entropy and the preset correlation entropy threshold, redundant features in the second candidate view feature sub-matrix are iteratively removed to obtain the third candidate view feature sub-matrix. For each dimension of the view feature vector in the third candidate view feature sub-matrix, the normalized mutual information score, multi-order statistical sensitivity score, and independence score of the view feature vector are calculated respectively. The normalized mutual information score, multi-order statistical sensitivity score, and independence score of the view feature vector are then weighted and fused to obtain the comprehensive discrimination score of the view feature vector. All view feature vectors whose comprehensive discrimination score is greater than or equal to a preset comprehensive discrimination score threshold are selected to form an initial simplified view feature sub-matrix. The initial simplified view feature submatrix is ​​subjected to cross-view consistency verification, and the initial simplified view feature submatrix is ​​adjusted for equalization based on the verification results to obtain the simplified view feature submatrix.

5. The multi-view clustering method for non-line-of-sight imaging data according to claim 4, characterized in that, The step of performing cross-view consistency verification on the initial simplified view feature sub-matrix and adjusting the initial simplified view feature sub-matrix for equalization based on the verification results includes: Singular value decomposition is performed on the initial simplified perspective feature submatrix, and the condition number is determined based on the ratio of the maximum singular value to the minimum singular value obtained from the singular value decomposition. Determine whether the number of conditions is greater than a preset threshold number of conditions; If the condition number is determined to be greater than the preset condition number threshold, then the variance inflation factor of each view feature vector is determined according to the initial simplified view feature submatrix; view feature vectors whose variance inflation factor is greater than the preset variance inflation factor threshold are removed to obtain the first corrected simplified view feature submatrix. Determine whether the condition number of the first modified simplified view feature submatrix is ​​greater than the preset condition number threshold; If it is determined that the condition number of the first modified simplified view feature submatrix is ​​greater than the preset condition number threshold, then the view feature vector with the largest variance inflation factor is removed from the first modified simplified view feature submatrix until the condition number is not greater than the preset condition number threshold, and the second modified simplified view feature submatrix is ​​obtained. The number of view feature vectors in the second modified simplified view feature submatrix corresponding to each of the view feature vector matrices is counted. The view balance index is calculated based on the absolute value of the difference between the number of view feature vectors in any two second modified simplified view feature submatrices. If the viewpoint balance index is greater than the preset balance index threshold, then the viewpoint feature vectors of each of the second modified simplified viewpoint feature sub-matrices are added or deleted until the viewpoint balance index is not greater than the preset balance index threshold, thus obtaining the simplified viewpoint feature sub-matrices; wherein, the simplified viewpoint feature sub-matrices are not empty sets.

6. The multi-view clustering method for non-line-of-sight imaging data according to claim 1, characterized in that, The deep nonnegative matrix factorization framework decomposes each of the simplified view feature sub-matrices into hierarchical feature matrices, and iterates the decomposed feature matrices with multiple optimization objectives to obtain the optimized decomposed feature matrices, including: For each of the simplified view feature sub-matrices, an R-layer nonnegative decomposition is performed on the simplified view feature sub-matrices based on a deep nonnegative matrix factorization framework to obtain an R-layer basis matrix and a final representation matrix; wherein, the R-layer basis matrix is ​​used to characterize the basis coordinates of the simplified view feature sub-matrices, and the final representation matrix is ​​used to characterize the coefficient matrix of the basis coordinates of the simplified view feature sub-matrices. The optimization objectives are to minimize the non-negative decomposition error of the R-layer, minimize the hypergraph regularization proximity of the simplified view feature sub-matrix, minimize the sample deviation between any two simplified view feature sub-matrixes, and minimize the redundancy of the R-layer basis matrix. A joint objective function is constructed with the R-layer basis matrix and the final representation matrix as optimization variables. The joint objective function is optimized by gradient descent and alternating direction multiplier method, and the optimal final representation matrix is ​​determined as the optimized decomposition feature matrix based on the optimal solution.

7. The multi-view clustering method for non-line-of-sight imaging data according to claim 1 or 6, characterized in that, The optimized decomposed feature matrices corresponding to each of the simplified viewpoint feature sub-matrices are fused to obtain fused feature matrices corresponding to each of the reconstructed depth image datasets. Cluster analysis is then performed on the fused feature matrices corresponding to each of the reconstructed depth image datasets to obtain clustering results for each of the reconstructed depth image datasets, including: For each of the reconstructed depth image datasets, the optimized decomposed feature matrices corresponding to each of the simplified view feature sub-matrices of the reconstructed depth image datasets are summed and averaged to obtain the fusion feature matrix of the reconstructed depth image dataset. kNN graph clustering is performed on the feature vectors of each sample in the fusion feature matrix of each reconstructed depth image dataset to obtain the clustering result of each reconstructed depth image dataset.

8. A multi-view clustering system for non-line-of-sight imaging data, characterized in that, include: The dataset extraction module is used to extract reconstructed depth image datasets of multiple modal categories of the target object for recognition; The viewpoint feature extraction module is used to extract features from multiple viewpoints for each of the reconstructed depth image datasets, and obtain multiple viewpoint feature vector matrices corresponding to each of the reconstructed depth image datasets. The data dimensionality reduction module is used to perform data dimensionality reduction on multiple view feature vector matrices of each of the reconstructed depth image datasets to obtain simplified view feature submatrices corresponding to each view feature vector matrix. The feature decomposition module is used to perform hierarchical feature matrix decomposition on each of the simplified perspective feature submatrices based on the deep non-negative matrix decomposition framework, and to optimize and iterate the decomposed feature matrix with multiple optimization objectives to obtain the optimized decomposed feature matrix. The clustering module is used to fuse the optimized decomposed feature matrices corresponding to each of the simplified view feature sub-matrices to obtain the fused feature matrices corresponding to each of the reconstructed depth image datasets, and to perform clustering analysis on the fused feature matrices corresponding to each of the reconstructed depth image datasets to obtain the clustering results corresponding to each of the reconstructed depth image datasets.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the multi-view clustering method for non-line-of-sight imaging data as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the steps of the multi-view clustering method for non-line-of-sight imaging data as described in any one of claims 1-7.

Images