Target point cloud enhancement and curved surface fitting method for sensing integration

Abstract

The invention discloses a target point cloud enhancement and curved surface fitting method for sensing integration, which solves the problems of high requirement on computing power and no fidelity in the prior art, and realizes high-precision and high-integrity three-dimensional real-time reconstruction of a complex urban environment under the condition of not depending on training data. The method comprises the following steps: acquiring original frequency domain data, extracting corresponding time domain channel response vectors, and stacking the time domain channel response vectors to obtain an echo matrix; performing distance dimension detection and spatial spectrum estimation according to the echo matrix to obtain an original sparse point cloud set and the signal amplitude of each point; performing weighted condensation processing based on signal amplitude on the original sparse point cloud set to obtain a denoised point cloud skeleton; performing local surface fitting and smooth projection on the point cloud skeleton based on a moving least square method to obtain a smooth point cloud; and performing iterative edge splitting interpolation under spatial constraint on the smooth point cloud until the point cloud density meets a preset condition, and outputting the enhanced dense point cloud.

Description

Technical Field

[0001] This invention relates to the fields of radar signal processing and three-dimensional imaging technology, and in particular to a method for target point cloud enhancement and surface fitting oriented towards integrated sensing. Background Technology

[0002] As mobile communication technology evolves towards 5G-Advanced and 6G, Integrated Sensing and Communication (ISAC) has become a key research direction in academia and industry. ISAC technology aims to leverage widely deployed cellular network infrastructure, such as base stations and user terminals, to achieve high-precision perception and reconstruction of the surrounding environment while transmitting high-speed data. This technology reuses spectrum resources and hardware, providing low-cost, wide-coverage environmental perception capabilities for applications such as smart cities, digital twins, and autonomous driving, making it a crucial enabling technology for future wireless networks.

[0003] In specific cellular network cooperative sensing scenarios, base stations (BS) are typically used as signal receivers or transmitters, and user terminals (UEs) are used as cooperative nodes. Uplink and downlink signals are used to image and reconstruct 3D images of static targets in the urban environment, such as buildings and street facilities. Unlike traditional monostatic radar, this distributed architecture can acquire multi-view information of targets and has the potential for higher spatial resolution. However, in the complex electromagnetic environment of real cities, achieving high-precision environmental imaging faces severe physical challenges, such as the problem of uneven point cloud distribution.

[0004] Limited by the beam scanning mechanisms of communication equipment, such as the step size limitations of mechanical rotation or electronic phased array scanning, and the sparsity of the observation aperture, the raw point clouds generated by radar signal processing often exhibit non-uniform distribution characteristics. For example, the point cloud is extremely dense or even redundant along the beam scanning trajectory, while there are obvious gaps between the scan lines, exhibiting a structured sparseness of "line clusters" or "strips". This non-uniform density distribution results in a rough and discontinuous surface of the reconstructed 3D model, making it difficult to meet the requirements of high-precision digital twins. At the same time, radar point clouds are not only sparse but also have many noise points, and each point carries unique physical properties, such as signal-to-noise ratio or signal amplitude, which existing geometric processing methods often ignore.

[0005] In the field of point cloud density enhancement and super-resolution reconstruction, data-driven methods based on deep learning are currently the mainstream technical approach. A representative existing technique proposes a point cloud upsampling method based on deep neural networks, also known as point cloud densification. Unlike traditional geometric interpolation, this method learns the local geometric features of the point cloud in a data-driven manner to predict the location of new points.

[0006] Furthermore, in the traditional field of geometric processing, interpolation methods based on surface fitting and subdivision are also a common technical approach. This paper proposes an interpolation scheme suitable for ultrasound image reconstruction. This scheme first extracts key feature points with high curvature and high density based on normal and curvature analysis. It then uses least squares or B-splines to estimate the local shape and perform surface fitting. Finally, it generates new points on the reconstructed surface using Loop subdivision or Catmull-Clark subdivision algorithms to address tomographic issues caused by scanning. The core logic of this type of method relies on geometric features, specifically curvature, to guide shape completion.

[0007] While the aforementioned deep learning-based solutions and traditional geometric fitting perform reasonably well in lidar or medical ultrasound imaging, they suffer from fundamental physical defects and engineering application bottlenecks when applied to distributed radar scenarios with integrated sensing and communication (ISAC). Existing technologies are all based on pure geometric coordinates for reasoning, completely ignoring the crucial physical information carried in radar echoes—namely, signal amplitude. Unlike lidar, radar point clouds are often accompanied by a large number of spurious noise points generated by multipath effects and sidelobes. Deep learning models only focus on the plausibility of the geometric distribution and cannot distinguish between high-confidence skeleton points and low-confidence noise points through amplitude, as this invention does. This causes the network to easily mistake noise for sparse structural features and amplify it, resulting in noise amplification. When faced with large holes between radar scan lines, it is easy to generate false surfaces that conform to visual logic but do not exist physically, severely affecting the realism of the imaging.

[0008] All the aforementioned deep learning-based solutions rely on a large amount of rigorously paired ground-value data for training. In real-world ISAC urban scenarios, the environment is complex and ever-changing, making it difficult to obtain perfect ground-value point clouds for training. Once the radar deployment location, frequency band, or surrounding environment changes, the pre-trained deep learning model often fails, requiring re-collection of data for training. Point cloud interpolation methods based on a small number of feature points tend to preserve and enhance points with "drastic curvature changes" as key features. However, in radar imaging, high-curvature areas often correspond to irregular outliers and noise. Directly using such methods can lead to noise being misjudged as structural features for surface fitting and interpolation, severely compromising the reliability of the imaging.

[0009] Whether it's a multilayer perceptron or a complex generative adversarial network, the inference process involves a large number of matrix multiplications and convolution operations, placing extremely high demands on computing resources and storage space. Sensor-enabled user terminals (UEs) or edge micro base stations are typically power-constrained and cannot support the real-time operation of such heavy algorithms.

[0010] In the application of sensor-integrated computing (ISAC) 3D imaging for complex urban environments, achieving high-fidelity and high-completeness environmental reconstruction faces severe challenges. Existing technologies suffer from the following systemic deficiencies when addressing the sparsity and noise issues of radar point clouds: The lack of physical information leads to noise amplification: Existing deep learning or interpolation methods based on surface fitting and subdivision typically treat point clouds as pure geometric data, completely ignoring the crucial amplitude or signal-to-noise ratio information carried in radar signals. Radar point clouds differ from lidar; they contain a large number of spurious noise points generated by multipath effects. Existing algorithms cannot distinguish between high-confidence "skeleton points" and low-confidence "noise points," and blindly performing geometric upsampling often results in the preservation or even amplification of noise, compromising the reliability of the imaging.

[0011] Geometric discontinuities caused by structured sparsity: Due to the limitations of radar beam scanning mechanisms, the original point cloud exhibits a significant "cluster-like" or "strip-like" distribution, with physical voids between the scan lines. Traditional linear interpolation methods are prone to producing sharp-edged artifacts at these voids; while neural network methods based on probabilistic prediction tend to generate discrete point clusters that do not conform to the physical manifold, resulting in a rough and discontinuous reconstructed surface.

[0012] The computing power bottleneck for edge deployment: Deep neural network models have a large number of parameters, high computational complexity, and rely on data from specific scenarios for training, resulting in poor generalization ability. This makes it difficult to meet the stringent requirements of integrated edge terminals (UEs) or micro base stations for real-time performance, low power consumption, and scenario adaptability.

[0013] In summary, under the constraint of limited computing power at the edge, we need to design a lightweight algorithm that can deeply integrate the physical information of radar signals, especially the amplitude / confidence information and the geometric prior of the scene, so as to achieve high-fidelity, high-continuity and physically reliable 3D environment reconstruction on sparse, noisy radar point clouds with structured holes. Summary of the Invention

[0014] This invention provides a target point cloud enhancement and surface fitting method for synesthesia integration, which solves the problems of high computing power requirements and low fidelity in the prior art. It achieves high-precision and high-completeness three-dimensional real-time reconstruction of complex urban environments without relying on training data.

[0015] This invention provides a method for enhancing target point clouds and fitting surfaces for synesthetic integration, the method comprising: Acquire the raw frequency domain data output from the receiver of the integrated induction system. Extract the original frequency domain data Time-domain channel response vector and the time-domain channel response vector Stacking is performed to obtain the echo matrix. ; According to the echo matrix Distance dimension detection and spatial spectrum estimation are performed to obtain the original sparse point cloud set and the signal amplitude of each point. ; The original sparse point cloud set is subjected to signal amplitude-based processing. The weighted agglomeration process is used to obtain the denoised point cloud skeleton. The point cloud skeleton is locally surface-fitted and smoothly projected using the moving least squares method to obtain a smooth point cloud. ; For the smooth point cloud Perform spatially constrained iterative edge splitting interpolation until the point cloud density meets the preset conditions, and output the enhanced dense point cloud.

[0016] In one possible implementation, the raw frequency domain data output by the receiver of the integrated sensing system is acquired. Extract the original frequency domain data Time-domain channel response vector and the time-domain channel response Stacking is performed to obtain the echo matrix. ,include: For the original frequency domain data Perform weighted window function Processing is performed to suppress spectral leakage, resulting in windowed frequency domain data; Perform an inverse Fourier transform on the windowed frequency domain data to obtain the time-domain channel impulse response of each receiving channel; Based on the M receiving channels of the aforementioned integrated sensing system, the time-domain channel impulse response vector corresponding to each channel is... Stacking is performed to obtain the echo matrix. ; wherein, the echo matrix Each column corresponds to a peak distance unit, and each row corresponds to the observation data of a spatial channel.

[0017] In one possible implementation, the statement based on the echo matrix... Distance dimension detection and spatial spectrum estimation are performed to obtain the original sparse point cloud set and the signal amplitude of each point. ,include: For the echo matrix Time-domain channel response vector of each receiving channel Peak detection is performed within a preset range to determine the peak distance unit where the peak point is located. and the corresponding signal amplitude value ; According to the peak distance unit The corresponding signal propagation delay is used to calculate the target distance estimation result. ; Extract the peak distance unit The corresponding spatial observation vector is used, and the arrival angle of the target is estimated using a spatial spectrum estimation algorithm. The azimuth angle is then obtained based on the arrival angle. and pitch angle ; Distance estimation results based on the target Azimuth and pitch angle The target is mapped to a global Cartesian coordinate system to generate peak distance units. The corresponding three-dimensional coordinates; Based on each peak distance unit The corresponding three-dimensional coordinates yield the original sparse point cloud set and the signal amplitude of each point. .

[0018] In one possible implementation, the process of performing signal amplitude-based processing on the original sparse point cloud set... The weighted agglomeration process yields the denoised point cloud skeleton, including: The three-dimensional space containing the original sparse point cloud set is divided into multiple cubic units of equal size; wherein the side length of the cubic unit is set according to the expected point cloud resolution. Traverse all non-empty cube cells and obtain the three-dimensional coordinates and corresponding signal amplitude of each point within each cube cell. Thus, the first three-dimensional coordinate set and the first signal amplitude set corresponding to the non-empty cube element are obtained; Calculate the amplitude-weighted centroid corresponding to the current non-empty cube element based on the first three-dimensional coordinate set and the first signal amplitude set. ; The magnitude-weighted centroid corresponding to each non-empty cube unit. As a point cloud skeleton.

[0019] In one possible implementation, the magnitude-weighted centroid Represented as: ; in, Represents the first element within a cube cell. The signal amplitude corresponding to each point; This represents the first point in the point cloud skeleton. One point; This represents the number of points in a non-empty cube element; This represents the number of points within a cubic unit. In one possible implementation, the step of performing local surface fitting and smooth projection on the point cloud skeleton based on the moving least squares method to obtain a point cloud on a smooth manifold includes: For each point in the point cloud skeleton Perform the following steps: Based on the current point Search within the neighborhood of Euclidean space Find the nearest neighbor points to construct a local neighborhood point set. ; Calculate the local neighborhood point set Weighted covariance matrix And for the weighted covariance matrix Perform eigenvalue decomposition to determine the normal vector of the local reference plane. To establish a local reference plane ; In the local reference plane Below, the weighted least squares method is used to fit a quadratic polynomial surface. ; Point Along the local reference plane The normal direction is orthogonally projected onto the quadratic polynomial surface. The smoothed projection point coordinates are obtained above. ; By traversing all points in the point cloud skeleton and performing a projection operation, a point cloud located on the smooth manifold is obtained.

[0020] In one possible implementation, the smoothing of the point cloud... Perform spatially constrained iterative edge splitting interpolation until the point cloud density meets preset conditions, and output the enhanced dense point cloud, including: Calculate the smoothed point cloud Average nearest neighbor distance of all points Determine the average nearest neighbor distance Is it less than or equal to the preset target point spacing threshold? ; If so, then the smoothed point cloud is determined. If the density meets the condition, output the smooth point cloud. As the enhanced dense point cloud; If not, an iterative interpolation process is performed to increase the point cloud density, updating the smoothed point cloud after each iteration. This continues until the preset conditions are met.

[0021] In one possible implementation, the iterative interpolation process includes: Calculate the smoothed point cloud For all point pairs, the Euclidean distance between the point pairs is calculated. Greater than the preset target point spacing threshold The point pairs are determined to be sparse point pairs. ; For each pair of sparse points Perform edge splitting operations to generate corresponding candidate interpolation points. ; Generate the candidate interpolation points Corresponding preset safety radius With the preset safety radius With radius, and with candidate interpolation points Determine the spherical detection area centered on the target area; Determine whether other points in the smooth point cloud already exist within the spherical detection region. If not, then add the candidate interpolation points. The point is identified as a valid new point and added to the smoothed point cloud. If so, then discard the candidate interpolation point. .

[0022] In one possible implementation, the statement regarding each pair of sparse points... Perform edge splitting operations to generate corresponding candidate interpolation points. ,include: Calculate the sparse point pairs Geometric midpoint of the line connecting the two endpoints ; In the smooth point cloud With the geometric midpoint As the query point, obtain the geometric midpoint. of The nearest neighbor; based on The nearest neighbor points are used to fit the first local surface using the moving least squares method, and the geometric midpoint is then... Projecting the points onto the first local curved surface yields projection points; The projection point is used as a candidate interpolation point. .

[0023] In one possible implementation, the safety radius It is based on the preset target point spacing threshold. Dynamic determination is performed.

[0024] One or more technical solutions provided in this invention have at least the following technical effects or advantages: This invention first acquires the raw frequency domain data output from the receiver of the integrated sensing system, extracts the time-domain channel response vector from this raw frequency domain data, and stacks these vectors to form an echo matrix. The advantage of this step is that it efficiently converts the raw frequency domain signal into a time-domain channel response, and by stacking and integrating multi-dimensional information, it provides a structured data foundation for subsequent processing, ensuring the accuracy of subsequent detection and estimation. Next, distance dimension detection and spatial spectrum estimation are performed based on the echo matrix to obtain the original sparse point cloud set and the signal amplitude of each point. This step, by combining distance and spatial information, can accurately extract the spatial location and signal strength of the target point, generating a sparse point cloud with amplitude information, providing key input for subsequent denoising and enhancement, and improving the representativeness of the point cloud. Then, the original sparse point cloud set undergoes weighted agglomeration processing based on signal amplitude to obtain a denoised point cloud skeleton. This step uses signal amplitude as a weight for agglomeration, effectively suppressing noise and outliers while highlighting high signal-to-noise ratio regions, forming a denoised point cloud skeleton, and enhancing the reliability and feature preservation capability of the point cloud. Next, the point cloud skeleton is fitted with a local surface and smoothly projected using the moving least squares method to obtain a smooth point cloud. This step adapts to the local geometry using the moving least squares method, achieving accurate surface fitting and smooth projection, generating a continuous and smooth point cloud surface, laying the foundation for detail enhancement and reducing surface irregularities. Finally, the smooth point cloud is subjected to spatially constrained iterative edge-splitting interpolation until the point cloud density meets the preset conditions, outputting an enhanced dense point cloud. This step gradually increases the point cloud density under spatial constraints through iterative interpolation, filling blank areas while maintaining geometric consistency, ultimately generating a high-resolution, detail-rich dense point cloud, significantly improving the visual effect and application value of the point cloud. Attached Figure Description

[0025] Figure 1 A flowchart illustrating the steps of a target point cloud enhancement and surface fitting method for synesthetic integration provided in an embodiment of the present invention. Figure 2 This is a true point map of the wall surface to be imaged provided in an embodiment of the present invention; Figure 3 This is the original single-UE point cloud result image provided in the embodiment of the present invention; Figure 4 This is a diagram showing the weighted centroid aggregation result provided in an embodiment of the present invention; Figure 5 The image shows the result of iterative interpolation processing provided in an embodiment of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0027] This invention provides a method for enhancing target point clouds and fitting surfaces for synesthetic integration, the method comprising the following steps S101 to S105.

[0028] S101, Obtain the raw frequency domain data output from the receiver of the integrated sensing system. Extract the original frequency domain data Time-domain channel response vector and the time-domain channel response vector Stacking is performed to obtain the echo matrix. ; Specifically, in step S101, the raw frequency domain data output by the receiver of the integrated sensing system is acquired. Extract the original frequency domain data Time-domain channel response vector and the time-domain channel response Stacking is performed to obtain the echo matrix. This includes the following steps S1011 to S1013.

[0029] S1011, for the original frequency domain data Perform weighted window function Processing is performed to suppress spectral leakage, resulting in windowed frequency domain data; S1012, Perform inverse Fourier transform on the windowed frequency domain data to obtain the time domain channel impulse response of each receiving channel; S1013, based on the M receiving channels of the integrated sensing system, the time-domain channel impulse response vector corresponding to each channel is... Stacking is performed to obtain the echo matrix. Among them, the echo matrix Each column corresponds to a peak distance unit, and each row corresponds to the observation data of a spatial channel.

[0030] For example, the raw frequency domain channel data is first obtained through the receiver of the integrated sensing system, such as the raw frequency domain channel data transmitted by the UE and received by the base station in the uplink. This data is represented in complex form as follows: (1.1) in, For the first The frequency points of each subcarrier, This represents the total number of subcarriers. This frequency domain data contains geometric information such as the target's range, angle, and reflection intensity.

[0031] To convert frequency domain observations into time domain features that include distance information and to reduce sidelobe leakage caused by spectral truncation, this step performs the following processing: (1) Windowing: Introduce a window function (such as a Chebyshev window) in the frequency dimension. right Weighting is applied to suppress spurious peaks and diffusion effects in the time-domain impulse response.

[0032] (2) Time-domain transformation: Perform an inverse Fourier transform (IFFT) on the windowed frequency domain data to obtain the time-domain impulse response vector, denoted as: (1.2) in, This is the index for time-domain sampling points, corresponding to different signal propagation delays.

[0033] For the receiving antenna array of a sensing-integrated system, assuming there are M receiving channels or M snapshots, calculate the time-domain impulse response vector of each channel obtained in the above steps. Stack the signals to construct the received echo matrix. The echo matrix Each column represents data from a peak distance unit, and each row represents observation data from a spatial channel or snapshot. Echo Matrix This serves as the input data for subsequent strong interference suppression steps.

[0034] S102, based on the echo matrix Distance dimension detection and spatial spectrum estimation are performed to obtain the original sparse point cloud set and the signal amplitude of each point. ; Specifically, in step S102, based on the echo matrix Distance dimension detection and spatial spectrum estimation are performed to obtain the original sparse point cloud set and the signal amplitude of each point. The steps include S1021 to S1025.

[0035] S1021, for the echo matrix Time-domain channel response vector of each receiving channel Peak detection is performed within a preset range to determine the peak distance unit where the peak point is located. and the corresponding signal amplitude value ; S102,2, based on peak distance unit The corresponding signal propagation delay is used to calculate the target distance estimation result. ; S1023, Extract peak distance unit The corresponding spatial observation vector is used, and the target's angle of arrival is estimated using a spatial spectrum estimation algorithm. The azimuth angle is then obtained from the angle of arrival. and pitch angle ; S1024, Distance estimation results based on the target Azimuth and pitch angle The target is mapped to a global Cartesian coordinate system to generate peak distance units. The corresponding three-dimensional coordinates; S1025, based on each peak distance unit The corresponding three-dimensional coordinates yield the original sparse point cloud set and the signal amplitude of each point. .

[0036] For example, based on the interference-suppressed time-domain echo matrix obtained in step S1 Perform the following operations to generate the original sparse point cloud set: (1) Distance dimension detection and amplitude extraction.

[0037] For echo matrix For each received channel data, amplitude detection is performed within a preset region of interest, such as Constant False Alarm Rate (CFAR) or peak detection. For the detected... Each peak point is recorded, along with its corresponding peak distance unit. and the signal amplitude at that location The final set of peak distance cells is denoted as... The corresponding target distance estimation result is: (1.3) in, At the speed of light, The time-domain sampling interval is denoted as .

[0038] (2) Angle estimation.

[0039] Extracting peak distance units The corresponding spatial observation vector is used to estimate the target's angle of arrival using spatial spectrum estimation algorithms, such as digital beamforming (DBF) or iterative adaptive IAA, to obtain the azimuth angle. and pitch angle .

[0040] (3) Three-dimensional coordinate calculation.

[0041] Distance estimation results based on the target Azimuth and pitch angle The target is transformed into a global Cartesian coordinate system with the base station as the origin. The calculation formula is as follows: (1.4) in, The coordinates of the base station are known.

[0042] Finally, the output includes The original sparse point cloud set of each point and the signal amplitude of each point. This serves as the weight input for subsequent steps.

[0043] S103, perform signal amplitude-based processing on the original sparse point cloud set. The weighted agglomeration process is used to obtain the denoised point cloud skeleton. Specifically, in step S103, the original sparse point cloud set is processed based on signal amplitude. The weighted aggregation process is used to obtain the denoised point cloud skeleton, including the following steps S1031 to S1034.

[0044] S1031, the three-dimensional space containing the original sparse point cloud set is divided into multiple cubic units of equal size; wherein, the side length of the cubic unit is set according to the expected point cloud resolution; S1032, traverse all non-empty cube elements to obtain the three-dimensional coordinates and corresponding signal amplitude of each point within each cube element. Thus, the first three-dimensional coordinate set and the first signal amplitude set corresponding to the non-empty cube element are obtained; S1033, Calculate the amplitude-weighted centroid corresponding to the current non-empty cube element based on the first three-dimensional coordinate set and the first signal amplitude set. ; Here, the magnitude-weighted centroid Represented as: (1.5) in, Represents the first element within a cube cell. The signal amplitude corresponding to each point; This represents the first point in the point cloud skeleton. Three-dimensional coordinate vectors of points; This represents the number of points in a non-empty cube element; This indicates the number of points within a cube cell.

[0045] S1034, the magnitude-weighted centroids corresponding to each non-empty cube element. As a point cloud skeleton.

[0046] For example, to address the local redundancy and noise issues in the original sparse point cloud set, step amplitude information is utilized. Perform weighted downsampling.

[0047] First, the three-dimensional space is divided into cubic units of equal size, namely voxels, and the side length of the voxels is set to the expected point cloud resolution.

[0048] Iterate through each non-empty voxel, assuming a voxel contains Q points. The signal amplitude corresponding to each point is Calculate the amplitude-weighted centroid of the voxel. As a reserved point.

[0049] This step utilizes the physical characteristics of radar echoes to assign greater weight to high-amplitude points, thereby causing the aggregated point cloud skeleton to automatically converge to the surface of the real object, effectively filtering out low-amplitude outliers.

[0050] S104, based on the moving least squares method, local surface fitting and smooth projection are performed on the point cloud skeleton to obtain a smooth point cloud. ; Specifically, in step S104, the point cloud skeleton is subjected to local surface fitting and smooth projection based on the moving least squares method to obtain the point cloud on the smooth manifold, including the following steps S1041 to S1045.

[0051] For each point in the point cloud skeleton Perform the following steps: S1041, based on the current point Search within the neighborhood of Euclidean space Find the nearest neighbor points to construct a local neighborhood point set. ; S1042, Calculate the local neighborhood point set Weighted covariance matrix And the weighted covariance matrix Perform eigenvalue decomposition to determine the normal vector of the local reference plane. To establish a local reference plane ; S1043, in the local reference plane Below, the weighted least squares method is used to fit a quadratic polynomial surface. ; S1044, point Along the local reference plane The normal direction is orthogonally projected onto the quadratic polynomial surface. The smoothed projection point coordinates are obtained above. ; S1045: Traverse all points in the point cloud skeleton, complete the projection operation, and obtain the point cloud located on the smooth manifold.

[0052] For example, the original sparse point cloud set is often affected by radar measurement noise and deviates from the true geometric surface of the target. Before interpolation, an implicit surface is first constructed using a moving least squares algorithm to project the discrete point cloud onto a latent smooth manifold. For any point in the original sparse point cloud set... The smoothing process is as follows: (1) Local neighborhood construction: at point Search within the neighborhood of Euclidean space The nearest neighbors form a local point set. .

[0053] (2) Local reference plane fitting: Calculate the local point set Weighted covariance matrix .in By accumulating local neighborhoods Internal points Relative to weighted centroid The outer product of the deviation vectors, and using a weight function that decays with spatial distance. The result obtained by weighted summation is... Through the analysis of Perform eigenvalue decomposition and extract the eigenvector corresponding to the smallest eigenvalue as the local normal vector. This determines a local reference plane. .

[0054] (3) Polynomial surface approximation: on a local reference plane Define a quadratic polynomial function in the local coordinate system. : (1.6) in, and These are parameters in the local coordinate system. arrive These are the coefficients to be determined. During the fitting process, the height field of the neighborhood points is fitted by minimizing the weighted squared error, i.e.: (1.7) in, It is the number of points in the neighborhood. It is the first Local coordinates of a point It is the actual height value of the point. These are weighting coefficients, used to weight points based on their confidence level. By minimizing this error, we can obtain the optimal polynomial coefficients. arrive .

[0055] (4) Projection: After fitting the quadratic polynomial function Then, the original point Projecting onto the fitted quadratic polynomial function Corresponding surface Above. To orthogonally project the original points onto the fitted surface along the normal direction of the surface, the surface is first written in implicit form: (1.8) Surface at any parameter point The normal vector at a given point is given by the gradient: (1.9) Let the coordinates of the original point to be smoothed in the local coordinate system be... Its orthogonal projection points on the fitted surface Defined as the closest point that minimizes the distance from the point to the surface, i.e.: (1.10) Solving for the results Then, the smoothed points in the local coordinate system are mapped back to the global coordinate system and used as the coordinates of the point cloud after orthogonal projection smoothing.

[0056] S105, for smoothing point clouds Perform spatially constrained iterative edge splitting interpolation until the point cloud density meets the preset conditions, and output the enhanced dense point cloud.

[0057] Specifically, in step S105, the point cloud is smoothed. Perform spatially constrained iterative edge splitting interpolation until the point cloud density meets the preset conditions, and output the enhanced dense point cloud, including the following: Calculate smooth point cloud Average nearest neighbor distance of all points Determine the average nearest neighbor distance Is it less than or equal to the preset target point spacing threshold? ; If so, then determine that the point cloud is smooth. The density meets the condition, and the output is a smooth point cloud. As an enhanced dense point cloud; If not, an iterative interpolation process is performed to increase the point cloud density, and the smoothed point cloud is updated after each iteration. Continue until the preset conditions are met.

[0058] Here, the iterative interpolation process includes: (1) Calculate smooth point cloud For all point pairs, the Euclidean distance between the point pairs is calculated. Greater than the preset target point spacing threshold The point pairs are determined to be sparse point pairs. ; (2) For each pair of sparse points Perform edge splitting operations to generate corresponding candidate interpolation points. Here, for each pair of sparse points... Perform edge splitting operations to generate corresponding candidate interpolation points. ,include: (2.1) Sparse point pairs Geometric midpoint of the line connecting the two endpoints ; (2.2) In smooth point cloud Using the geometric midpoint Used as the query point, the geometric midpoint is obtained. of The nearest neighbor; (2.3) Based on The nearest neighbor points are used to fit the first local surface using the moving least squares method, and the geometric midpoint is determined. Project the points onto the first local surface to obtain the projection points; (2.4) Use the projection points as candidate interpolation points .

[0059] (3) Generate candidate interpolation points Corresponding preset safety radius With a preset safety radius With radius, and with candidate interpolation points Determine the spherical detection area centered on [the target area]; here, the safety radius is [defined]. It is based on the preset target point spacing threshold. Dynamic determination is performed.

[0060] (4) Check if other points in the smooth point cloud already exist within the spherical detection area. If not, then add the candidate interpolation points. The point was identified as a valid new point and added to the smoothed point cloud. If so, discard the candidate interpolation point. .

[0061] For example, to address the structured sparsity, such as line clusters, caused by radar scanning, an iterative interpolation method based on local manifold reconstruction is used for completion. To solve the structured sparsity problem (such as the gaps between radar scan lines), this scheme designs an iterative interpolation strategy constrained by a "safety radius." Let the target point spacing be... The current point cloud set is The iterative process is as follows: (1) Local density convergence assessment: Calculate the current point cloud set Average nearest neighbor distance of all points .like If the point cloud density meets the requirements, the algorithm converges and terminates; otherwise, it proceeds to the next round of interpolation.

[0062] (2) Sparse topology identification and candidate point generation: Traverse the point cloud and identify the distance to its nearest neighbor. sparse point pairs For each pair of sparse points, perform an "edge splitting" operation: (2.1) Calculation of the linear midpoint: First, calculate the Euclidean geometric midpoint between the two points. It is important to note that this midpoint is usually located on the connecting line, not on the surface of the object; for example, the midpoint of the chord on the surface of a cylinder is located inside the cylinder.

[0063] (2.2) Nonlinear manifold projection: To correct geometric errors, the moving least squares projection algorithm described above is called again. For the query point, search its surroundings. Nearest neighbors, fit the local surface, and Projecting onto this surface yields candidate interpolation points located on the object's surface manifold. .

[0064] (3) Spatial constraint check based on safety radius: To prevent invalid interpolation (overfitting) in areas with dense existing data, a spatial exclusion mechanism is introduced. A safe radius is defined. ,in As a safety factor. For each candidate point. Construct a sphere with its center and radius as... Detection sphere If the intersection of the detected sphere and the current point cloud set is empty, it indicates that a data hole does exist at that location, and the data is accepted. A new data point is considered valid; if other data points already exist within the detected sphere, it indicates that the density in that region is sufficient, and the data point is discarded. Avoid the formation of local clusters of data.

[0065] (4) Collection update: Set all new points that pass the security check Merge into the original set: Then return to step 1 of the iteration process, re-evaluate the global density and start a new round of encryption until the convergence condition is met, and output the final dense point cloud.

[0066] In a specific embodiment provided by the present invention To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This method is applied to a cooperative sensing system including a base station (BS) and a user terminal (UE). First, a realistic integrated sensing scenario is constructed, containing a static target to be imaged. In this example, the target is a self-built wall, and the wall's location is as follows: Figure 2 And collaborative sensing nodes deployed around the target. A stationary UE transmits signals, and the base station receives the signals; each UE forms an uplink and downlink with only a single base station.

[0067] Chamfer distance is a core metric for evaluating the similarity between a reconstructed point cloud and a ground truth 3D model. It measures the mean squared distance from each point in the point cloud to the nearest point in the ground truth model, and the sum of the mean squared distances from each point in the ground truth model to the nearest point in the point cloud. Its expression is as follows: (1.11) In the formula Represents the true scattering point. This represents the reconstructed scattering points. A smaller Chamfer distance indicates that the reconstructed point cloud is closer to the real building. The forward distance measures the average nearest point distance between the simulated point cloud and the ground truth point cloud, reflecting the accuracy of the reconstructed point cloud—how close the points in the simulated point cloud are to the ground truth points. The backward distance measures the average nearest point distance between the ground truth point cloud and the simulated point cloud, reflecting the completeness of the reconstructed point cloud—whether the simulated point cloud contains enough points to cover the ground truth scene.

[0068] After generating the original sparse point cloud, the point cloud result of a single link is obtained, as follows: Figure 3 : Depend on Figure 3 As can be seen, due to the inherent characteristics of mechanical or electronic scanning, the original point cloud exhibits a significant "striped" distribution: the point cloud is extremely dense along the scan line direction, with a large amount of redundancy; while in the direction perpendicular to the scan line, there are obvious gaps between the lines, resulting in severe structure sparsity problems. At this point, the Chamfer distance is 0.5064m. Although the point cloud roughly outlines the wall contour, its uneven distribution severely affects the quality of subsequent 3D reconstruction. To address the redundancy problem of the original data, amplitude-weighted centroid condensation processing is first performed, with the following results. Figure 4 As shown.

[0069] From a visual perspective, the original dense "line clusters" were successfully simplified into a more uniformly distributed discrete lattice, effectively extracting the geometric "skeleton" of the wall and eliminating noise buildup caused by repeated scanning. At this stage, the Chamfer distance (0.6145m) and the reverse distance (1.0050m) were slightly higher than the original point cloud. This is in line with expected physical phenomena: the downsampling process is essentially a lossy compression; although it sacrifices some microscopic details and results in less complete coverage, and the reverse distance temporarily decreases, it successfully removes high-frequency noise, providing a reliable geometric reference for subsequent high-quality interpolation.

[0070] Based on the condensed point cloud, iterative moving least squares interpolation is performed, and the final result is as follows. Figure 5 As shown.

[0071] In comparison Figure 5 As can be seen, the algorithm identified sparse regions between points and generated new interpolation points using local surface fitting technology. The filled wall point cloud is dense and continuous, effectively covering the original scanning gaps, and does not produce excessive accumulation around existing agglomeration points, maintaining good surface smoothness.

[0072] The processing results are compared in the table below: Table 1 Comparison of point cloud density homogenization results.

[0073] Table 1

[0074] Simulation results show that the proposed two-stage algorithm not only effectively solves the problem of uneven radar point cloud density, but also significantly improves the geometric reconstruction accuracy and completeness of the point cloud through high-fidelity interpolation while removing noise.

[0075] The present invention proposes a method for enhancing target point clouds and fitting surfaces for synesthetic integration, which has the following advantages compared with the prior art: Compared to existing methods based on deep learning or geometric interpolation that only utilize coordinate information, this invention introduces radar echo amplitude into the point cloud processing flow. Through an amplitude-based adaptive voxel aggregation algorithm, leveraging the physical characteristic that the echo intensity of the real target is much higher than that of sidelobe noise, the algorithm automatically assigns higher weights to skeleton points. This allows the algorithm to intelligently identify and remove outliers with low signal-to-noise ratios (SNR) without relying on any training data, significantly improving the SNR and structural accuracy of the point cloud and avoiding the problem of "noise being amplified by the law of rules" in existing technologies. For the "line-cluster" voids caused by radar scanning, this invention employs a moving least squares (MLS) manifold interpolation strategy. This method constructs a locally implicit surface, ensuring that the newly generated interpolation points are strictly constrained to a smooth geometric manifold, effectively filling the scanning blind zone and significantly reducing the Chamfer distance error. Simulations show a reduction of approximately 26%, achieving dense and continuous target surface reconstruction. The matrix operation, weighted statistics, and mathematical fitting algorithms employed in this invention reduce computational complexity by several orders of magnitude compared to deep learning models such as Multilayer Perceptrons (MLP) or Generative Adversarial Networks (GANs). This solution eliminates the need for massive data collection and pre-training, and does not require expensive GPU computing power. It can be directly deployed on user terminals (UEs) or base stations with limited computing resources, meeting the millisecond-level real-time response requirements to dynamic environmental changes.

[0076] The various embodiments described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. All or part of this invention can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, mobile communication terminals, multiprocessor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0077] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the present invention.

Claims

1. A method for enhancing target point clouds and fitting surfaces for synesthetic integration, characterized in that, include: Acquire the raw frequency domain data output from the receiver of the integrated induction system. Extract the original frequency domain data Time-domain channel response vector and the time-domain channel response vector Stacking is performed to obtain the echo matrix. ; According to the echo matrix Distance dimension detection and spatial spectrum estimation are performed to obtain the original sparse point cloud set and the signal amplitude of each point. ; The original sparse point cloud set is subjected to signal amplitude-based processing. The weighted agglomeration process is used to obtain the denoised point cloud skeleton. The point cloud skeleton is locally surface-fitted and smoothly projected using the moving least squares method to obtain a smooth point cloud. ; For the smooth point cloud Perform spatially constrained iterative edge splitting interpolation until the point cloud density meets the preset conditions, and output the enhanced dense point cloud.

2. The target point cloud enhancement and surface fitting method for synesthetic integration according to claim 1, characterized in that, The raw frequency domain data output from the receiver of the integrated sensing system is obtained. Extract the original frequency domain data Time-domain channel response vector and the time-domain channel response Stacking is performed to obtain the echo matrix. ,include: For the original frequency domain data Perform weighted window function Processing is performed to suppress spectral leakage, resulting in windowed frequency domain data; Perform an inverse Fourier transform on the windowed frequency domain data to obtain the time-domain channel impulse response of each receiving channel; Based on the M receiving channels of the aforementioned integrated sensing system, the time-domain channel impulse response vector corresponding to each channel is... Stacking is performed to obtain the echo matrix. ; wherein, the echo matrix Each column corresponds to a peak distance unit, and each row corresponds to the observation data of a spatial channel.

3. The target point cloud enhancement and surface fitting method for synesthetic integration according to claim 1, characterized in that, According to the echo matrix Distance dimension detection and spatial spectrum estimation are performed to obtain the original sparse point cloud set and the signal amplitude of each point. ,include: For the echo matrix Time-domain channel response vector of each receiving channel Peak detection is performed within a preset range to determine the peak distance unit where the peak point is located. and the corresponding signal amplitude value ; According to the peak distance unit The corresponding signal propagation delay is used to calculate the target distance estimation result. ; Extract the peak distance unit The corresponding spatial observation vector is used, and the arrival angle of the target is estimated using a spatial spectrum estimation algorithm. The azimuth angle is then obtained based on the arrival angle. and pitch angle ; Distance estimation results based on the target Azimuth and pitch angle The target is mapped to a global Cartesian coordinate system to generate peak distance units. The corresponding three-dimensional coordinates; Based on each peak distance unit The corresponding three-dimensional coordinates yield the original sparse point cloud set and the signal amplitude of each point. .

4. The target point cloud enhancement and surface fitting method for synesthetic integration according to claim 1, characterized in that, The original sparse point cloud set is subjected to signal amplitude-based processing. The weighted agglomeration process yields the denoised point cloud skeleton, including: The three-dimensional space containing the original sparse point cloud set is divided into multiple cubic units of equal size; wherein the side length of the cubic unit is set according to the expected point cloud resolution. Traverse all non-empty cube cells and obtain the three-dimensional coordinates and corresponding signal amplitude of each point within each cube cell. Thus, the first three-dimensional coordinate set and the first signal amplitude set corresponding to the non-empty cube element are obtained; Calculate the amplitude-weighted centroid corresponding to the current non-empty cube element based on the first three-dimensional coordinate set and the first signal amplitude set. ; The magnitude-weighted centroid corresponding to each non-empty cube unit. As a point cloud skeleton.

5. The target point cloud enhancement and surface fitting method for synesthetic integration according to claim 4, characterized in that, The magnitude-weighted centroid Represented as: ； in, Represents the first element within a cube cell. The signal amplitude corresponding to each point; This represents the first point in the point cloud skeleton. One point; This represents the number of points in a non-empty cube element; This indicates the number of points within a cube cell.

6. The target point cloud enhancement and surface fitting method for synesthetic integration according to claim 1, characterized in that, The step of performing local surface fitting and smooth projection on the point cloud skeleton based on the moving least squares method to obtain a point cloud on a smooth manifold includes: For each point in the point cloud skeleton Perform the following steps: Based on the current point Search within the neighborhood of Euclidean space Find the nearest neighbor points to construct a local neighborhood point set. ; Calculate the local neighborhood point set Weighted covariance matrix And for the weighted covariance matrix Perform eigenvalue decomposition to determine the normal vector of the local reference plane. To establish a local reference plane ; In the local reference plane Below, the weighted least squares method is used to fit a quadratic polynomial surface. ; Point Along the local reference plane The normal direction is orthogonally projected onto the quadratic polynomial surface. The smoothed projection point coordinates are obtained above. ; By traversing all points in the point cloud skeleton and performing a projection operation, a point cloud located on the smooth manifold is obtained.

7. The target point cloud enhancement and surface fitting method for synesthetic integration according to claim 1, characterized in that, The smooth point cloud Perform spatially constrained iterative edge splitting interpolation until the point cloud density meets preset conditions, and output the enhanced dense point cloud, including: Calculate the smoothed point cloud Average nearest neighbor distance of all points Determine the average nearest neighbor distance Is it less than or equal to the preset target point spacing threshold? ; If so, then the smoothed point cloud is determined. If the density meets the condition, output the smooth point cloud. As the enhanced dense point cloud; If not, an iterative interpolation process is performed to increase the point cloud density, updating the smoothed point cloud after each iteration. This continues until the preset conditions are met.

8. The target point cloud enhancement and surface fitting method for synesthetic integration according to claim 7, characterized in that, The iterative interpolation process includes: Calculate the smoothed point cloud For all point pairs, the Euclidean distance between the point pairs is calculated. Greater than the preset target point spacing threshold The point pairs are determined to be sparse point pairs. ; For each pair of sparse points Perform edge splitting operations to generate corresponding candidate interpolation points. ; Generate the candidate interpolation points Corresponding preset safety radius With the preset safety radius With radius, and with candidate interpolation points Determine the spherical detection area centered on the target area; Determine whether other points in the smooth point cloud already exist within the spherical detection region. If not, then add the candidate interpolation points. The point is identified as a valid new point and added to the smoothed point cloud. If so, then discard the candidate interpolation point. .

9. The target point cloud enhancement and surface fitting method for synesthetic integration according to claim 8, characterized in that, For each pair of sparse points Perform edge splitting operations to generate corresponding candidate interpolation points. ,include: Calculate the sparse point pairs Geometric midpoint of the line connecting the two endpoints ; In the smooth point cloud With the geometric midpoint As the query point, obtain the geometric midpoint. of The nearest neighbor; based on The nearest neighbor points are used to fit the first local surface using the moving least squares method, and the geometric midpoint is then... Projecting the points onto the first local curved surface yields projection points; The projection point is used as a candidate interpolation point. .

10. The target point cloud enhancement and surface fitting method for synesthetic integration according to claim 8, characterized in that, The safety radius It is based on the preset target point spacing threshold. Dynamic determination is performed.

Images