Spatial scattering modeling method, parameter estimation method and system, device, medium

Abstract

This application provides a spatial scattering modeling method, parameter estimation method, system, device, and medium, belonging to the field of 3D reconstruction technology. The method includes: inputting the 3D position parameters of sampling points into a spatial branch network, outputting diffuse scattering components, surface roughness, signed distance function values, and spatial implicit feature vectors; inputting the spatial implicit feature vectors, directional features, and incident angle into a directional branch network, outputting specular scattering components; determining the equivalent scattering point intensity based on the diffuse and specular scattering components; performing volume rendering integration based on the equivalent scattering point intensity and visibility weights to generate predicted observation data; updating the network parameters based on the prediction error until a preset number of training iterations is reached, obtaining a trained spatial scattering model. The spatial scattering modeling method, parameter estimation method, system, device, and medium provided in this application can achieve 3D scattering parameter inversion of a target scene.

Description

Technical Field

[0001] This application belongs to the field of three-dimensional reconstruction technology, and more specifically, it relates to a spatial scattering modeling method, parameter estimation method and system, equipment and medium. Background Technology

[0002] In mobile applications such as handheld portable scanning devices or vehicle-mounted terminals, the mainstream 3D reconstruction in the industry still uses a combination of "LiDAR + camera": the former provides high-precision geometry, and the latter provides color and texture. The fusion of the two can quickly build a geometric model. However, this technical approach mainly focuses on geometric surfaces and visible light textures, making it difficult to characterize the electromagnetic scattering properties of objects / mediums. It often lacks robust descriptions of material differences or surface roughness differences, thus reducing the accuracy of 3D modeling. Summary of the Invention

[0003] The purpose of this application is to provide a spatial scattering modeling method, parameter estimation method and system, device and medium that can realize the inversion of three-dimensional scattering parameters of the target scene, thereby improving the accuracy of three-dimensional modeling.

[0004] A first aspect of this application provides a method for estimating spatial scattering parameters, including: The historical data corresponding to the three-dimensional position parameters and incident direction vector of the sampling points in three-dimensional space are obtained; the historical data is calculated based on the multi-frame radar observation data collected from the target scene under multiple carrier poses, and the millimeter-wave radar pose parameters corresponding to each frame of radar observation data. The historical data of the three-dimensional position parameters are input into a spatial branch network to obtain the diffuse scattering component, surface roughness, symbolic distance function value, and spatial implicit feature vector; the normal vector at the three-dimensional spatial sampling point is obtained by differentiating the symbolic distance function value with respect to the three-dimensional position parameters. The reflection direction vector is determined based on the historical data of the incident direction vector and the normal vector, and the incident angle is determined; the reflection direction vector is encoded with roughness weight based on the surface roughness to obtain the directional feature characterizing the bandwidth of the specular scattering direction distribution; the spatial implicit feature vector, the directional feature and the incident angle are input into the directional branch network to output the specular scattering component; The equivalent scattering point intensity is determined based on the diffuse scattering component and the specular scattering component. A discrete sampling point sequence ordered by distance is obtained based on distance-Doppler-channel raster mapping; the opacity is obtained by transforming the signed distance function value at the sampling point using the Logistic function, and the visibility weight is determined based on the opacity. Volume rendering integration is performed based on the equivalent scattering point intensity and the visibility weight to generate predicted observation data. The training loss between the predicted observation data and the historical data of the radar observation data is calculated. Based on the training loss, the network parameters of the spatial branch network and the directional branch network are updated by backpropagation. The step of inputting the historical data of the three-dimensional position parameters into the spatial branch network is returned to the execution until the preset number of training times is reached, and the trained millimeter-wave radar spatial scattering model is obtained.

[0005] A second aspect of this application provides a spatial scattering parameter estimation method, wherein the spatial scattering modeling method obtains a millimeter-wave radar spatial scattering model, comprising: Acquire multiple frames of radar observation data of the target scene under multiple carrier poses, and obtain the millimeter-wave radar pose parameters corresponding to each frame of radar observation data; based on the multiple frames of radar observation data and the multiple frames of millimeter-wave radar pose parameters, determine the three-dimensional position parameters of the three-dimensional spatial sampling points, as well as the incident direction vector corresponding to the three-dimensional spatial sampling points. The three-dimensional position parameters and the incident direction vector are input into the millimeter-wave radar spatial scattering model, and the relative permittivity, surface roughness and diffuse scattering components output by the millimeter-wave radar spatial scattering model are determined as the spatial scattering parameters of the millimeter-wave radar.

[0006] A third aspect of the embodiments of this application provides a spatial scattering parameter estimation system, including an edge computer, a millimeter-wave radar, a motion platform, and a pose acquisition module; The transmitting and receiving antenna elements of the millimeter-wave radar are arranged vertically in the elevation direction to provide elevation angle resolution in a single transmission cycle. The motion platform is used to change the pose of the millimeter-wave radar during the data acquisition process to obtain multi-frame radar observation data corresponding to multiple poses. The pose acquisition module is used to acquire the pose parameters corresponding to each frame of radar observation data. The edge computer is configured to resolve the target's elevation information using vertically arranged array elements, resolve the azimuth information using Doppler parallax information generated by near-horizontal motion, and execute the steps of the aforementioned spatial scattering parameter estimation method.

[0007] A fourth aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the spatial scattering parameter estimation method described above.

[0008] A fifth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described spatial scattering parameter estimation method.

[0009] The beneficial effects of the spatial scattering modeling method, parameter estimation method, system, device, and medium provided in the embodiments of this application are as follows: This application embodiment uses a symbolic distance function to implicitly represent the target scene and uses the implicit surface as the geometric carrier. The normal vector, roughness, relative permittivity and other parameters are used as explicit estimable task parameters. Under the supervision of incoherent distance-Doppler-channel grid (RDK) amplitude, the spatial scattering parameter field is jointly inverted. During the training process, the joint estimation of scene geometry and material properties is realized, and a quantitative and robust description of material and roughness differences is achieved, which is conducive to improving the accuracy of 3D modeling. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A schematic flowchart illustrating a spatial scattering parameter estimation method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the network structure of a millimeter-wave radar spatial scattering model provided in an embodiment of this application; Figure 3 This is a schematic diagram of the incident and reflection directions provided in an embodiment of this application; Figure 4 This is a schematic diagram of the overall structure of a spatial scattering parameter estimation system provided in an embodiment of this application; Figure 5 A schematic diagram of the overall structure of a spatial scattering parameter estimation system provided in one embodiment of this application from another angle; Figure 6 This is a schematic diagram of the radar array element arrangement provided in one embodiment of this application; Figure 7 This is a schematic diagram of a data acquisition process provided in an embodiment of this application; Figure 8 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0012] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0014] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a spatial scattering modeling method provided in an embodiment of this application. The spatial scattering modeling method provided in this embodiment can be executed by an electronic device, and the method may include: S101: Obtain the historical data corresponding to the three-dimensional position parameters of the target scene, radar observation data, and the incident direction vector of the millimeter-wave radar; the historical data of the radar observation data is obtained based on the millimeter-wave radar.

[0015] In this embodiment, the historical data corresponding to the three-dimensional position parameters of the target scene, radar observation data, and the incident direction vector of the millimeter-wave radar can be obtained, and a spatial scattering model can be constructed accordingly.

[0016] Specifically, radar observation data of the target scene collected under multiple carrier poses can be obtained, and millimeter-wave radar pose parameters corresponding to each frame of radar observation data can be obtained. Based on this, the three-dimensional position parameters of the sampling points in three-dimensional space can be determined according to the range-Doppler-channel grid index, and the corresponding incident direction vector can be calculated.

[0017] Millimeter-wave radar transmits and receives linear frequency modulated signals, and after signal processing, obtains range-Doppler-channel (RDK) three-dimensional grid data. Each grid index corresponds to the target echo characteristics in the range, Doppler, and antenna channel dimensions. Combined with the carrier pose and radar extrinsic parameters corresponding to each frame of radar observation data, the observation results in the radar's local coordinate system can be transformed to a unified world coordinate system. Using the antenna array configuration and direction-of-arrival estimation corresponding to the grid index, the incident angle and radial distance of the target relative to the radar can be calculated. Then, through inter-frame pose transformation, multi-view observation constraints are established to associate the radar observation ray with the spatial position, thereby determining the world coordinates of the three-dimensional spatial sampling point. At the same time, the vector pointing from the radar center to the spatial sampling point is the incident direction vector.

[0018] This embodiment achieves the mapping from signal domain grid to spatial geometric parameters through joint calculation of multi-frame pose and radar observation, providing a foundation for subsequent 3D scattering modeling and scene reconstruction.

[0019] S102: Input historical data of three-dimensional position parameters into the spatial branch network to obtain diffuse scattering components, surface roughness, symbolic distance function values, and spatial implicit eigenvectors; obtain the normal vector at the three-dimensional spatial sampling point by differentiating the symbolic distance function values ​​with respect to the three-dimensional position parameters.

[0020] Please refer to Figure 2 In this embodiment, the spatial branching network can be implemented using a multilayer perceptron. A signed distance function (SDF) is fitted within the multilayer perceptron to implicitly represent the geometry of the target scene, denoted as... ,in Given the 3D position parameters of the sampling points in the global coordinate system, for any point in the target scene, the SDF can output the signed distance from that point to the scene surface. The gradient direction of the SDF is exactly the normal direction of the scene surface. Therefore, by calculating the gradient of the signed distance function, the surface normal vector of the target scene can be obtained. , This indicates the calculation of the gradient function.

[0021] Three-dimensional position parameters Input spatial branching network, regress to obtain diffuse scattering component Relative permittivity Surface roughness and spatial implicit eigenvectors Among them, the relative permittivity field The roughness field is a complex number; its real part describes the polarization capability of the medium, and its imaginary part describes the medium's loss. A nonnegative scalar field, used to adjust the angular bandwidth of the specular scattering component; diffuse scattering component field. Used to describe non-mirror scattering energy that is approximately uniform in all directions.

[0022] S103: Determine the reflection direction vector and the incident angle based on historical data and normal vector of the incident direction vector; perform roughness weight direction encoding on the reflection direction vector based on surface roughness to obtain the direction feature characterizing the bandwidth of the specular scattering direction distribution; input the spatial implicit feature vector, direction feature and incident angle into the direction branch network to output the specular scattering component.

[0023] Please refer to Figure 3 In this embodiment, the incident angle ， According to the incident direction vector of millimeter-wave radar The unit direction vector of ray reflection (i.e., the radar receiving direction) is defined as: And combined with the normal vector Calculate the direction vector of mirror reflection Mirror reflection direction vector for about The mirror image.

[0024] The roughness of a scene surface creates a bandwidth range in the reflection direction. The greater the roughness, the wider the dispersion range in the reflection direction, and the weaker the specular effect. By encoding the reflection direction with roughness weights, the two physical pieces of information, the reflection direction and the roughness bandwidth, are fused into a structured directional feature vector. This retains the main trend of the reflection direction while also incorporating the directional dispersion characteristics brought about by roughness.

[0025] Geometric features The input direction branch network consists of direction features and incident angle, and is composed of roughness. Under the control of directional encoding for broadening, the specular scattering component is obtained, and its intensity is expressed as: Its physical meaning is: It only handles the angular shape caused by some geometric features, viewpoint features, and roughness features, with roughness used to control the angular width of the specular scattering lobe. The directional branch network is also implemented based on a multilayer perceptron.

[0026] In this embodiment, the reflection direction is used instead of the line-of-sight direction as the core directional condition. The reflection direction directly corresponds to the echo reception path, which allows the core logic and calculation dimension of scattering modeling to be directly aligned with the physical origin of echo generation. This fundamentally ensures that the modeling process is completely consistent with the actual generation mechanism of radar echoes, eliminating theoretical deviations.

[0027] S104: Determine the equivalent scattering point intensity based on diffuse scattering component and specular scattering component.

[0028] In this embodiment, the scattering components of the target scene surface mainly include diffuse scattering components and specular scattering components. By fusing the diffuse scattering components and specular scattering components, the equivalent scattering point intensity can be obtained. The equivalent scattering point intensity can uniformly characterize the single-point scattering capability of the target scene surface.

[0029] S105: Based on the distance-Doppler-channel raster mapping, a discrete sampling point sequence sorted by distance is obtained; the opacity is obtained by transforming the symbolic distance function value at the sampling point using the Logistic function, and the visibility weight is determined based on the opacity. Based on the equivalent scattering point intensity and the visibility weight, a volume rendering integral is performed to generate the predicted observation data.

[0030] In this embodiment, the symbolic distance function representing the target scene is sampled in one dimension along the incident direction of the millimeter-wave radar to obtain the SDF value of each point on the ray path. The SDF values ​​of the sampled points are input into the Logistic function for nonlinear transformation to obtain an opacity of 0 to 1. The opacity is used to characterize the degree of energy attenuation of electromagnetic waves passing through the point, with 1 representing complete blockage and 0 representing complete transparency. Based on the cumulative opacity characteristics, a visibility weight (0 to 1) is calculated to quantify the effective observation degree of the millimeter-wave radar at the point. The equivalent scattering point intensity of each sampled point is weighted with the visibility weight, and a volume rendering integral is performed along the incident direction. Combined with the antenna beam gain, the predicted range-Doppler-channel grid amplitude data is obtained for comparison with the measured amplitude data during the neural implicit field training process.

[0031] Among them, the RDK raster amplitude obtained through volume rendering integration Specifically:

[0032]

[0033] in For sampling point distance index, The distance under the current distance index. Let be the incident direction vector of the sampling point. For pitch angle index, For visibility weight, For sampling point Doppler index, This represents the beam antenna gain in the current direction in the global coordinate system. This indicates the echo intensity at that sampling point. This represents the number of sampling points on the Doppler ring. Given the current speed of the radar movement, This indicates the direction of the sampling point under the Doppler ring sampling point index at the current distance.

[0034] S106: Calculate the training loss between the predicted observation data and the historical data of the radar observation data. Based on the training loss, backpropagate and update the network parameters of the spatial branch network and the directional branch network. Return to execute the step of inputting the historical data of the three-dimensional position parameters into the spatial branch network until the preset number of training times is reached, and obtain the trained millimeter-wave radar spatial scattering model.

[0035] As can be seen from the above, this embodiment performs implicit surface representation of the target scene based on the symbolic distance function, and uses the implicit surface as the geometric carrier. The normal vector, roughness, relative permittivity and other parameters are used as explicit estimable task parameters. Under the amplitude supervision of incoherent RDK, the spatial scattering parameter field is jointly inverted. During the training process, the joint estimation of scene geometry and material properties is realized, and a quantitative and robust description of material and roughness differences is achieved, thereby improving the accuracy of 3D modeling.

[0036] In one embodiment of this application, roughness weight direction encoding is performed on the reflection direction vector based on surface roughness to obtain directional features characterizing the bandwidth of the specular scattering direction distribution, including: Frequency coding is performed on the reflection direction vector to obtain frequency coding features; By mapping surface roughness to a bandwidth parameter and weighting and attenuating the frequency coding features based on the bandwidth parameter, the directional features characterizing the bandwidth distribution of the specular scattering direction are obtained.

[0037] In this embodiment, by predicting the roughness The bandwidth parameter, mapped to the von Mises-Fischer (vMF) distribution, is used to perform weighted attenuation (low-pass filtering) on ​​the frequency coding characteristics of the reflection direction; that is, the greater the roughness, the lower the retained direction coding frequency, thereby simulating the blurring and broadening effect of the specular reflection beam caused by the rough surface in the angular direction.

[0038] In one embodiment of this application, when inputting historical data of three-dimensional position parameters into a spatial branch network, the relative permittivity is also obtained; The equivalent scattering point intensity is determined based on the diffuse scattering component and the specular scattering component, including: Based on the Fresnel reflection model, the specular scattering gain is determined based on the incident angle and relative permittivity. The specular scattering component is modulated based on the specular scattering gain and added to the diffuse scattering component to obtain the equivalent scattering point intensity.

[0039] In one embodiment of this application, determining the mirror scattering gain based on the incident angle and relative permittivity includes: The mirror scattering gain is determined by the following formula:

[0040] in,

[0041]

[0042]

[0043] Indicates the mirror scattering gain. Represents the vertical polarization reflection coefficient. Represents the horizontal polarization reflection coefficient. Indicates the angle of incidence. Represents the relative permittivity. For intermediate calculation variables.

[0044] In this embodiment, based on Fresnel boundary conditions, with the incident medium being air and the equivalent magnetic permeability... Under the assumptions, calculate the Fresnel amplitude reflection coefficients for vertical and horizontal polarization. Since this method only uses incoherent amplitude data as supervision and does not explicitly estimate the angle between the polarization direction and the incident plane, an unpolarized average approximation is adopted in the Fresnel boundary conditions to obtain the interface specular scattering power gain coefficient.

[0045] Furthermore, the specular scattering component is scaled by the interface specular scattering power gain coefficient and added to the diffuse scattering component to form the equivalent scattering point intensity of the millimeter-wave radar:

[0046] in For radar receiving direction, Angle of incidence is the relative permittivity.

[0047] In one embodiment of this application, determining the visibility weight based on opacity includes: The cumulative transmittance is obtained by accumulating the opacity of each sampling point along the incident direction; Visibility weights are determined by multiplying cumulative transmittance by the opacity of the current sampling point.

[0048] In this embodiment, the Fresnel boundary condition is applicable only if the electromagnetic wave propagates at the air-medium interface; therefore, a visibility weight is introduced during the point scattering intensity rendering process. This constrains the accumulation of point scattering intensity to primarily occur near the object's surface, suppressing non-physical scattering contributions within the object. Specifically:

[0049]

[0050]

[0051]

[0052] in, The symbolic distance function; Indicates the first Discrete opacity of each sampling interval; These are preset control parameters. Indicates by The Logistic function controls the transition width of the surface boundary; Indicates the distance from the ray's origin to the ray's starting point. Cumulative transmittance before each sampling point This represents the initial value of the cumulative transmittance.

[0053] In one embodiment of this application, calculating the training loss between predicted observation data and historical data of radar observation data includes: The power domain mean square error and Doppler centroid error between the predicted observation data and the historical data of the radar observation data are calculated, and a regularization term is introduced to constrain the gradient norm of the sign distance function as the training loss.

[0054] In this embodiment, the training loss uses the power domain mean square error. And Doppler centroid error. This represents the predicted observation data. This refers to historical data representing radar observation data.

[0055] Doppler centroid error: For a training batch, we calculate the error for each radar frame that appears in the batch. The Doppler centroids of observed and predicted data are calculated, and a robust penalty is applied to their differences. The centroid consistency loss is defined as... ; in This indicates the set of frame indexes contained in this batch. express The predicted Doppler centroid under frame, express The Doppler centroid observed in the frame. The centroid formula is expressed as:

[0056] in For the set of Doppler indexes participating in the statistics, For numerically stable terms, Indicates the Doppler value. Same as above ,express Frame with ( Historical data for radar observations under the index. Indicates will The result after aggregating the distance dimension and the channel dimension.

[0057] To ensure the stability of the geometric surface and the normal vector, an Eikonal regularization of the signed distance function is introduced: ,in This indicates the number of sampling points included in the statistics.

[0058] Introducing the Eikonal regularization of the signed distance function can cause the volume density to converge into a narrow layer near the surface, maintain the stability of the normal vector, suppress normal vector noise, and finally obtain the total loss: .

[0059] in, and All of these represent weighting coefficients.

[0060] Based on the same inventive concept, this application also provides a spatial scattering parameter estimation method. The millimeter-wave radar spatial scattering model obtained by applying the above-described spatial scattering modeling method includes: Acquire multiple frames of radar observation data of the target scene under multiple carrier poses, and obtain the millimeter-wave radar pose parameters corresponding to each frame of radar observation data; determine the three-dimensional position parameters of the three-dimensional spatial sampling points and the incident direction vector corresponding to the three-dimensional spatial sampling points based on the multiple frames of radar observation data and the multiple frames of millimeter-wave radar pose parameters. The three-dimensional position parameters and incident direction vector are input into the millimeter-wave radar spatial scattering model, and the relative permittivity, surface roughness and diffuse scattering components output by the millimeter-wave radar spatial scattering model are determined as the spatial scattering parameters of the millimeter-wave radar.

[0061] By inputting the three-dimensional position parameters and the incident direction vector of the millimeter-wave radar into the trained millimeter-wave radar spatial scattering model, the relative permittivity output by the millimeter-wave radar spatial scattering model is extracted. Surface roughness and diffuse scattering component data.

[0062] Furthermore, the aforementioned spatial scattering parameters are mapped to a three-dimensional space aligned with the implicit surface geometry to form a millimeter-wave spatial scattering parameter field that combines geometric structure and physical material property information, which can be used for subsequent point cloud classification, semantic segmentation, or target material analysis tasks.

[0063] Please refer to Figures 4-5 Based on the same inventive concept, this application also provides a spatial scattering parameter estimation system, including an edge computer, a millimeter-wave radar, a motion platform, and a pose acquisition module; The transmitting and receiving antenna elements of millimeter-wave radar are arranged vertically in the elevation direction to provide elevation angular resolution within a single transmission cycle. The motion platform is used to change the pose of the millimeter-wave radar during the data acquisition process to obtain multi-frame radar observation data corresponding to multiple poses. The pose acquisition module is used to acquire the pose parameters corresponding to each frame of radar observation data. The edge computer is configured to resolve the target's elevation information using vertically arranged array elements, resolve the azimuth information using Doppler parallax information generated by near-horizontal motion, and perform the steps of the aforementioned spatial scattering parameter estimation method.

[0064] In this embodiment, the spatial scattering parameter estimation system consists of an edge computing unit, a power and cable management unit, a millimeter-wave radar and camera unit, a lidar unit, and a human-machine display unit (optional) integrated within the same body. Each unit is fixed together by structural components, and sensor driving and extrinsic parameter calibration are completed at the end, forming a closed loop of "acquisition—synchronization—alignment—download / storage". The entire system meets the weight and power consumption constraints for handheld / vehicle-mounted mobile operations, and its external interfaces are unified and easy to maintain and expand.

[0065] The edge computing unit utilizes an embedded industrial control platform based on Linux, but can also be other industrial computers with equivalent computing power and I / O capabilities. This unit deploys drivers and data acquisition programs for millimeter-wave radar, lidar, and optical cameras, and provides sensor extrinsic parameter calibration functions. It writes the acquired raw data streams to a cache and local storage by timestamp. The extrinsic parameter calibration process involves measuring the relative positional relationship between the coordinate systems of each sensor and the radar coordinate system, using the millimeter-wave radar coordinate system as a reference, based on the device's 3D model. Optical and lidar sensors achieve extrinsic parameter calibration through displacement and rotation compensation. To improve reliability for outdoor and long-term operation, the edge computer is housed in an aluminum alloy casing 10, which also serves as a heat sink with pre-installed air ducts or heat sinks. A dual-antenna structure is located on the top of the casing, supporting Wi-Fi hotspot / telemetry links for remote monitoring, parameter distribution, and data feedback. The edge computer is connected to each sensor and power supply unit via internal wiring harnesses, forming a unified power and data path.

[0066] The power and cable management unit provides centralized power supply and orderly cabling for the system. This unit includes a replaceable lithium battery pack and a step-down module. The power box 9 features a cable management cavity and through-hole cabling, with a cable pass-through hole 11 on the back and a pre-drilled cable exit window on the front panel, allowing cables from the radar and camera to pass directly through and connect to the rear edge computer. The power box 9 integrates a power display module 6 on its side and four independent power switches 12 on its back, controlling the main power supply, edge computer 4, LiDAR 3, and millimeter-wave radar 1 sequentially, facilitating separate power-on and safe maintenance. Handholds 7 are located on both sides of the power box 9, and the back of the power box 9 serves as the mounting base for the edge computer 4, achieving a compact, integrated stack.

[0067] The millimeter-wave radar 1 and camera unit 2 are packaged in an independent radar box 8, which houses the FMCW-MIMO millimeter-wave radar and its data acquisition board.

[0068] In near-field implicit spatial scattering parameter estimation tasks based on millimeter-wave radar, multi-view parallax information, especially sufficient observability of elevation, is required to stably retrieve scene geometry and spatial scattering parameters. However, existing millimeter-wave radar-based spatial scattering parameter estimation systems still suffer from an imbalance in the collaborative design of array deployment and motion sampling: existing millimeter-wave imaging radars designed for near-field or vehicle-mounted scenarios often deploy dense antenna arrays in the azimuth direction or rely on long-term platform movement along the azimuth direction to synthesize high resolution; while in the elevation direction, only a few or even a single array of elements are configured, lacking an elevation sampling density that matches the azimuth direction. As a result, the azimuth array and slow-time motion are highly redundant in terms of information, while elevation sampling is significantly insufficient. To compensate for the deficiency in elevation dimension sampling, additional mechanical scanning or large-amplitude attitude maneuvers are often required in the elevation direction, resulting in low acquisition efficiency. Furthermore, it is still difficult to obtain sufficient elevation information under near-horizontal motion conditions such as handheld / vehicle-mounted operation, which easily leads to elevation ambiguity and 3D geometric degradation.

[0069] Please refer to Figure 6To address the aforementioned issues, this embodiment considers that autonomous vehicles, drones, or handheld devices typically move in near-horizontal motion. The invention employs an array configuration with vertically arranged array elements, enabling the vertical elements to provide pitch resolution in a single frame. This resolution is then combined with the Doppler resolution formed by the platform along its horizontal trajectory and mapped to azimuth resolution, thereby achieving joint calculation of the target's three-dimensional information (range-azimuth-pitch). This enriches the dimensions of the received data, reduces reliance on repeated acquisitions of multiple trajectories or multi-altitude trajectories to supplement pitch information, and is more suitable for high-quality three-dimensional reconstruction under "horizontal motion" conditions. The front panel features a window / cutout area matching the radar beam field of view (FOV) to prevent the housing from obstructing transmission and reception and causing multipath effects. A pinhole-type small optical camera 2 (for visible light assistance and calibration) is embedded in the front cover. The camera 2 is coplanarly mounted with the millimeter-wave radar 1 and coaxially / parallelly constrained with the housing to reduce mechanical parallax. The radar box 8 has a wiring hole on the back, which is aligned with the wiring hole on the front panel of the power supply box 10 during assembly, so that the wiring harness of the millimeter-wave radar 1 and camera 2 can be directly connected to the edge computer 4 at the rear of the machine, reducing the risk of wiring harness bending and electromagnetic coupling.

[0070] The lidar unit 3 is mounted on top of the lidar box 8 and connected to the lidar box 8 and the chassis frame via a dedicated bracket, ensuring that the field of view is not obstructed by the upper and lower structures. The power and data cables of this unit are also internally routed through the through hole between the lidar box 8 and the power box 9, and finally connected to the high-speed interface of the edge computer 4; during the factory or assembly stage, the external parameters of the camera / millimeter-wave lidar are solved and the parameters are fixed through calibration boards or natural scenes.

[0071] The HMI display unit 5 is an optional configuration used to display the edge computer page, key operating indicators, and acquisition status in real time. This unit is not a necessary component of the system, but it can significantly improve usability and debugging efficiency in field acquisition. In situations where weight reduction or concealed installation is required, the display unit can be removed, leaving only the wireless link for remote monitoring.

[0072] The various units of this system are interconnected for power and data through internal wiring harnesses and standard interfaces: the power and cable management unit provides distributed regulated power to the edge computer and the three types of sensors; the edge computer establishes data links with the millimeter-wave radar acquisition board, lidar, and camera via gigabit Ethernet / USB buses, and performs extrinsic parameter compensation at the edge; the wireless link is used for configuration and data transmission, and supports high-speed writing to local solid-state storage when necessary. Through the above structure, the millimeter-wave radar, lidar, and optical camera are integrated into a single unit, with orderly internal wiring and collaborative processing at the edge, providing a stable and reproducible experimental platform for subsequent spatial scattering parameter estimation based on a differentiable forward model.

[0073] See Figure 7Based on the aforementioned integrated platform, data acquisition and processing can be completed. After the system is powered on, the edge computer loads the drivers for the millimeter-wave radar, lidar, and optical camera, reads the spatial calibration extrinsic parameter library, and records timestamps using the edge computer's local clock as a unified time reference. The millimeter-wave radar's operating parameters and acquisition control are sent via USB serial port, including start / stop commands and configuration frames; the radar's raw I / Q data is continuously transmitted back to the edge computer's cache via Ethernet UDP and stored synchronously. The lidar driver simultaneously runs the odometry algorithm (LiDAR+IMU), outputting the generated carrier pose trajectory. The input to the odometry algorithm (LiDAR+IMU) is the continuous point cloud frames of the lidar and the raw acceleration and angular velocity data of the built-in IMU; the algorithm first integrates the high-frequency data of the IMU to obtain an initial estimate of the carrier's pose change between two adjacent point cloud frames, and then uses the geometric matching between the current frame point cloud and the previous frame to correct the estimate; finally, it outputs a time-ordered six-DOF carrier pose trajectory (position). and posture It can also output the linear velocity at each moment. acceleration This data is used to provide a reference for subsequent radar data filtering. The above multi-source data is written to the dataset directory according to the timestamp, forming a raw data packet consisting of "LiDAR odometry (accelerometer / velocity / attitude / pose), LiDAR point cloud, camera video stream, millimeter-wave radar raw I / Q data", thus completing one acquisition cycle.

[0074] In summary, compared with existing spatial scattering parameter estimation systems, the spatial scattering parameter estimation system of this embodiment has at least the following advantages: (1) In the task of reconstructing the near-field implicit scattering field, alleviate the contradiction between azimuth sampling redundancy and pitch sampling insufficiency, and improve the observability of elevation / normal vector.

[0075] This embodiment addresses the application characteristics of automotive-grade millimeter-wave radar in near-field implicit spatial scattering parameter estimation methods by employing a vertically arranged array element configuration in the elevation direction. Azimuth resolution is primarily provided by near-horizontal motion and Doppler information, while elevation resolution is directly provided by the vertical array elements. Compared to schemes that only stack array elements in the azimuth direction or extend the synthetic aperture and perform additional mechanical elevation scanning in the elevation direction, this embodiment reduces azimuth sampling redundancy and improves elevation sampling efficiency without significantly increasing hardware complexity and trajectory difficulty. This results in better elevation information and surface normal vector observability in near-field scenarios such as handheld and vehicle-mounted applications, providing more reliable geometric constraints for subsequent millimeter-wave spatial scattering field reconstruction based on implicit representations and mitigating elevation ambiguity and 3D geometric degradation.

[0076] (2) Introduce a structured scattering model based on the normal vector and the reflection direction to weaken the dependence of line-of-sight vector-driven black box modeling on network compensation.

[0077] To address the issue that existing implicit reconstruction methods for millimeter-wave radar often directly use the line-of-sight vector as the sole directional condition, relying on a black-box network regression of line-of-sight related occupancy, reflectivity, and transmittance, this embodiment explicitly introduces the surface normal vector obtained from the signed distance field (SDF) gradient into the rendering model. The reflection direction is calculated based on the law of reflection, and combined with roughness-controlled direction encoding and power-domain Fresnel gain, diffuse scattering and specular components are decomposed and modeled. By embedding physical mechanisms such as "normal vector field + roughness broadening + Fresnel interface gain" into the network in a structured manner, this embodiment reduces the burden of single-line-of-sight vector black-box regression in millimeter-wave radar spatial scattering parameter estimation scenarios. This allows for a clearer physical correspondence between the network output parameters and geometry and material properties, improving the stability and interpretability of reconstruction results in near-field scenes with sparse viewpoints and significant multipath propagation.

[0078] (3) Construct a spatial scattering parameter field that combines geometry and material properties to provide interpretable features for subsequent high-order tasks such as classification / segmentation.

[0079] To address the issue that existing methods often focus on recovering 3D point clouds or units such as occupancy and reflectivity, making it difficult to directly reflect material properties such as dielectric properties and roughness from millimeter-wave observations within a unified framework, this embodiment uses implicit surfaces as geometric carriers and takes normal vectors, roughness, and relative permittivity as explicitly estimable task parameters. Under incoherent RDK amplitude supervision, it jointly inverts the spatial scattering parameter field, providing physically meaningful feature descriptions for subsequent higher-order tasks such as point cloud / occupancy space classification, segmentation, and target material analysis.

[0080] See Figure 8 , Figure 8 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 8 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. The processors 301 are configured to invoke the program instructions to perform the functions of each module / unit in the above-described device embodiments.

[0081] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0082] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0083] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store preset constants such as error thresholds.

[0084] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation method described in the spatial scattering parameter estimation method provided in the embodiments of this application, or they can execute the implementation method of the electronic device described in the embodiments of this application, which will not be repeated here.

[0085] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0086] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., provided on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0087] Those skilled in the art will recognize that the modules / units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0088] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0089] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules, units, or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces or modules / units, or it may be an electrical, mechanical, or other form of connection.

[0090] The modules / units described as separate components may or may not be physically separate. Similarly, the components shown as modules / units may or may not be physical modules / units; they may be located in one place or distributed across multiple network modules / units. Some or all of the modules / units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0091] Furthermore, the functional modules / units in the various embodiments of this application can be integrated into one processing module / unit, or each module / unit can exist physically separately, or two or more modules / units can be integrated into one module / unit. The integrated modules / units described above can be implemented in hardware or in the form of software functional modules / units.

[0092] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A spatial scattering modeling method, characterized in that, include: The historical data corresponding to the three-dimensional position parameters and incident direction vector of the sampling points in three-dimensional space are obtained; the historical data is calculated based on the multi-frame radar observation data collected from the target scene under multiple carrier poses, and the millimeter-wave radar pose parameters corresponding to each frame of radar observation data. The historical data of the three-dimensional position parameters are input into a spatial branch network to obtain the diffuse scattering component, surface roughness, symbolic distance function value, and spatial implicit feature vector; the normal vector at the three-dimensional spatial sampling point is obtained by differentiating the symbolic distance function value with respect to the three-dimensional position parameters. The reflection direction vector is determined based on the historical data of the incident direction vector and the normal vector, and the incident angle is determined; the reflection direction vector is encoded with roughness weight based on the surface roughness to obtain the directional features characterizing the bandwidth of the specular scattering direction distribution; The spatial implicit feature vector, the directional feature, and the incident angle are input into the directional branch network, and the output is the specular scattering component. The equivalent scattering point intensity is determined based on the diffuse scattering component and the specular scattering component. A discrete sampling point sequence ordered by distance is obtained based on distance-Doppler-channel raster mapping; the opacity is obtained by transforming the signed distance function value at the sampling point using the Logistic function, and the visibility weight is determined based on the opacity. Volume rendering integration is performed based on the equivalent scattering point intensity and the visibility weight to generate predicted observation data. The training loss between the predicted observation data and the historical data of the radar observation data is calculated. Based on the training loss, the network parameters of the spatial branch network and the directional branch network are updated by backpropagation. The step of inputting the historical data of the three-dimensional position parameters into the spatial branch network is returned to the execution until the preset number of training times is reached, and the trained millimeter-wave radar spatial scattering model is obtained.

2. The spatial scattering modeling method as described in claim 1, characterized in that, Based on the surface roughness, the reflection direction vector is encoded with roughness weight to obtain directional features characterizing the bandwidth of the specular scattering direction distribution, including: The reflection direction vector is frequency encoded to obtain frequency encoded features; The surface roughness is mapped to a bandwidth parameter, and the frequency coding features are weighted and attenuated based on the bandwidth parameter to obtain the directional features that characterize the bandwidth of the specular scattering direction distribution.

3. The spatial scattering modeling method as described in claim 1, characterized in that, When inputting the historical data of the three-dimensional position parameters into the spatial branch network, the relative permittivity is also obtained; Determining the equivalent scattering point intensity based on the diffuse scattering component and the specular scattering component includes: Based on the Fresnel reflection model, the specular scattering gain is determined based on the incident angle and the relative permittivity. The specular scattering component is modulated based on the specular scattering gain and added to the diffuse scattering component to obtain the equivalent scattering point intensity.

4. The spatial scattering modeling method as described in claim 3, characterized in that, Determining the mirror scattering gain based on the incident angle and the relative permittivity includes: The mirror scattering gain is determined by the following formula: in, Indicates the mirror scattering gain. Represents the vertical polarization reflection coefficient. Represents the horizontal polarization reflection coefficient. Indicates the angle of incidence. Represents the relative permittivity. For intermediate calculation variables.

5. The spatial scattering modeling method as described in claim 1, characterized in that, The process of determining the visibility weight based on the opacity includes: The cumulative transmittance is obtained by accumulating the opacity of each sampling point along the incident direction; The visibility weight is determined based on the product of the cumulative transmittance and the opacity of the current sampling point.

6. The spatial scattering modeling method as described in claim 1, characterized in that, Calculating the training loss between the predicted observation data and the historical data of the radar observation data includes: The power domain mean square error and Doppler centroid error between the predicted observation data and the historical data of the radar observation data are calculated, and a regularization term for constraining the gradient norm of the signed distance function is introduced as the training loss.

7. A method for estimating spatial scattering parameters, comprising a millimeter-wave radar spatial scattering model obtained by applying the spatial scattering modeling method according to any one of claims 1 to 6, characterized in that, include: Acquire multiple frames of radar observation data of the target scene under multiple carrier poses, and obtain the millimeter-wave radar pose parameters corresponding to each frame of radar observation data; Based on the multi-frame radar observation data and multi-frame millimeter-wave radar pose parameters, the three-dimensional position parameters of the three-dimensional spatial sampling points and the incident direction vector corresponding to the three-dimensional spatial sampling points are determined. The three-dimensional position parameters and the incident direction vector are input into the millimeter-wave radar spatial scattering model, and the relative permittivity, surface roughness and diffuse scattering component output by the millimeter-wave radar spatial scattering model are determined as the spatial scattering parameters of the millimeter-wave radar.

8. A spatial scattering parameter estimation system, characterized in that, Includes edge computing, millimeter-wave radar, motion platform, and pose acquisition module; The transmitting and receiving antenna elements of the millimeter-wave radar are arranged vertically in the elevation direction to provide elevation angle resolution in a single transmission cycle. The motion platform is used to change the pose of the millimeter-wave radar during the data acquisition process to obtain multi-frame radar observation data corresponding to multiple poses. The pose acquisition module is used to acquire the pose parameters corresponding to each frame of radar observation data. The edge computer is configured to resolve the target's pitch information using vertically arranged array elements, resolve the azimuth information using Doppler parallax information generated by near-horizontal motion, and perform the steps of the method described in claim 7.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in claim 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in claim 7.

Images