Multi-source point cloud data fusion system based on end-side information collaboration model

Abstract

A multi-source point cloud data fusion system based on an end-side information collaboration model comprises a protocol configuration module, a time synchronization module, a point cloud data downsampling and compression module and an edge side data fusion module, and effective cooperation between an equipment end and an edge side is achieved through end-side information collaboration. At the equipment end, efficient downsampling and compression processing is performed on the point cloud data, the data volume is reduced, and the transmission bandwidth requirement is lowered; on the edge side, by means of boundary clock construction and a peer-to-peer delay mechanism, the accuracy of data fusion is ensured, and real-time fusion processing is carried out on data from different sensors, so that faster and more accurate decision support is provided. Besides, the system has high reliability, high efficiency, high real-time performance and generalization, can adapt to different sensors and industrial environments, and meets strict requirements in industrial control application. The method aims at improving the decision-making speed and accuracy of industrial control application through an efficient data fusion strategy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information processing, and particularly relates to a multi-source point cloud data fusion system based on an end-edge information collaborative model. BACKGROUND

[0002] Under the background of rapid development of industrial intelligent manufacturing, point cloud data, as an important carrier of three-dimensional spatial information, plays an important role in many fields. The existing technology faces many challenges in processing multi-source point cloud data fusion: first, the synchronization accuracy of sensor data is insufficient, resulting in inaccurate fusion results; second, data transmission occupies a large amount of bandwidth, reducing efficiency; third, the fusion speed is low, and a large amount of resources is required to process large-scale data, limiting the system scalability and real-time performance; fourth, lack of adaptability and universality, difficult to dynamically adjust. SUMMARY

[0003] The present application proposes a multi-source point cloud data fusion system based on an end-edge information collaborative model to address the above-mentioned deficiencies in the existing technology. Through end-edge information collaboration, effective cooperation between the device end and the edge side is achieved. At the device end, efficient downsampling and compression processing of point cloud data is performed to reduce data volume and transmission bandwidth requirements. At the edge side, a boundary clock structure and peer-to-peer delay mechanism are used to ensure the accuracy of data fusion and perform real-time fusion processing of data from different sensors to provide faster and more accurate decision support. In addition, the system has high reliability, efficiency, real-time performance, and generalization, and can adapt to different sensors and industrial environments to meet the strict requirements of industrial control applications. The present application is committed to improving the decision speed and accuracy of industrial control applications through efficient data fusion strategies.

[0004] The present application is implemented through the following technical solutions:

[0005] The application relates to a multi-source point cloud data fusion system based on an edge-side information collaborative model, which comprises a protocol configuration module, a time synchronization module, a point cloud data downsampling and compression module and an edge-side data fusion module, wherein: the protocol configuration module establishes a time synchronization architecture through import configuration, establishes a connection and calculates the time delay of data transmission of each sensor, constructs an edge-side information collaborative model comprising an information aggregation unit, a strategy generation unit, a parameter distribution unit and a state monitoring unit, and constructs an overall topology structure through the edge-side information collaborative model according to configuration information to perform global resource scheduling; the time synchronization module establishes a connection based on the collaborative model, receives, analyzes and distributes a clock synchronization signal based on the architecture of time synchronization to perform calibration, and realizes time synchronization from a 5G base station to an edge end to a sensor device; the point cloud data downsampling and compression module performs data compression on collected point cloud data through downsampling and entropy encoding of the weight of points and maintains data quality; and the edge-side data fusion module receives compressed data from the sensor, performs point cloud registration through a Gaussian curvature-based iterative closest point algorithm and generates an accurate three-dimensional point cloud model integrating information of all sensors.

[0006] The edge-side information collaborative model receives initialization parameters and network topology information from the protocol configuration module, receives each node clock offset from the time synchronization module, dynamically generates a resource scheduling strategy and data processing parameters and distributes them to the edge-side data fusion module, and dynamically adjusts the scheduling strategy to ensure synchronization and collaboration of the edge side and the edge side in timing, resources and data processing procedures. Technical effects

[0007] Compared with existing point cloud data fusion methods, the edge-side information collaborative model can effectively solve many problems in the context of efficient fusion of multi-source point cloud data, and the specific technical effects are as follows:

[0008] To solve the problem of insufficient synchronization accuracy, the application adopts a SIB9 time synchronization mechanism based on 5G technology and 3GPP R16 to realize clock synchronization from a 5G base station to a gateway, and then uses a boundary clock to ensure accurate alignment of timestamps of all sensor devices. This method solves the problem of inaccurate data fusion caused by inconsistent data synchronization between sensors in existing methods, and improves the reliability of data fusion.

[0009] To solve the problem of low data transmission efficiency, the application optimizes the data retention strategy by evaluating the weight and quality of points to avoid loss of key information, and further reduces the bandwidth required for data transmission by using a space-time transformation difference technique while maintaining the integrity and accuracy of data. The data transmission efficiency of the system is effectively improved, and the complexity of data processing is reduced.

[0010] In view of the problem of slow data processing speed, the application constructs an efficient data alignment and fusion algorithm to improve the efficiency of point cloud data processing. Compared with existing data fusion methods, the application not only reduces the storage volume of data, but also maintains the core geometric features and attribute information of data, and improves the speed of point cloud data fusion.

[0011] In view of the problem of poor system adaptability, the application provides configuration of master-slave ports and clock synchronization protocol through a protocol configuration module, communication between master-slave ports is carried out in standard data packets, and efficient collaboration between modules is realized by using an end-side information collaborative model, thereby realizing efficient fusion of multi-sensor data, improving the stability and reliability of data processing, enhancing the practicality and generalization ability of the system, enabling it to adapt to different sensors and industrial environments, and meeting the strict requirements in industrial control applications. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 It is a structural schematic diagram of the application.

[0013] Figure 2 It is a supplementary schematic diagram of the end-side information collaborative model.

[0014] Figure 3 It is a system structure diagram of the embodiment. DETAILED DESCRIPTION

[0015] As shown in Figure 3 , it is an application architecture based on a multi-source point cloud data fusion system, which includes a storage layer, a business layer and an application layer arranged in turn from bottom to top, wherein: the storage layer respectively stores fused point cloud data, system logs and device information; the application layer uses a React framework and an antd component library to obtain point cloud data and node state information from an edge side data fusion module through an http request and perform data display; as Figure 1The business layer of the multi-source point cloud data fusion system shown imports configuration to establish a time synchronization architecture through a protocol configuration module, establishes a connection and calculates the time delay of data transmission of each sensor, then builds an end-edge information collaboration model including an information aggregation unit, a strategy generation unit, a parameter distribution unit and a state monitoring unit, and builds the overall topology structure through the end-edge information collaboration model according to the configuration information, and performs global resource scheduling; a connection is established on the basis of the collaboration model through a time synchronization module, the clock synchronization signal is received, analyzed and distributed based on the time synchronization architecture for calibration, and time synchronization is realized from the 5G base station to the edge end to the sensor equipment; the point cloud data is compressed by the point cloud data downsampling and compression module through the weight evaluation of the points for downsampling and entropy encoding, and the data quality is maintained; the compressed data from the sensor is received by the edge side data fusion module, the point cloud registration is performed through the iterative closest point algorithm based on the Gaussian curvature, and the accurate three-dimensional point cloud model integrating all sensor information is generated.

[0016] The storage layer is composed of MongoDB, InfluxDB and PostgreSQL database clusters.

[0017] The protocol configuration module includes a boundary clock construction unit, a delay measurement and parameter recording unit and a collaboration model construction unit, wherein: the boundary clock construction unit adopts the Precision Time Protocol (PTP) to define the master and slave ports, generates a boundary clock structure taking the 5G gateway as the master port and each sensor as the slave port, and then configures the clock synchronization protocol between the master and slave ports to ensure that the clock signal is accurately and correctly distributed to the clock distribution mechanism configuration of all sensors; in this process, the end-edge information collaboration model loads and analyzes the configuration file provided externally to build a DeviceConfig structure list; the delay measurement and parameter recording unit adopts the IEEE 1588 protocol to measure the transmission delay between ports by sending delay measurement packets, and synchronizes all clocks and the most accurate clock in the distributed network; the collaboration model construction unit initializes the end-edge collaboration model based on the boundary clock structure and the delay measurement result.

[0018] The configuration file contains the basic structure of the network, specifically including: the information of all sensor nodes, the network address of the edge server, the master clock source and the data processing parameters.

[0019] In the DeviceConfig structure list, each structure corresponds to a device node and a GlobalParams object, and stores global parameters such as downsampling default parameters, compression algorithm selection, maximum number of iterations of ICP registration algorithm, etc., and the end-edge information collaboration model performs network discovery and delay measurement process based on the DeviceConfig structure list obtained by analysis.

[0020] The synchronization of all clocks with the most accurate clock, specifically comprising:

[0021] 1) The master periodically sends out sync messages and records the precise departure time of the sync message from the master clock Afterwards, the master encapsulates the precise departure time into a Follow_Up message and sends it to the slave.

[0022] 2) The slave records the precise arrival time of the message and sends out a Delay_Req message while recording the precise departure time . The master records the precise arrival time of the Delay_Req message and sends a Delay_Resp message to the slave carrying the precise timestamp information .

[0023] 3) The slave calculates the time offset and the one-way transmission delay between the master and the slave and outputs them to the end-side information coordination model for clock calibration, specifically comprising: the time offset , the one-way transmission delay or .

[0024] The edge server acts as a master clock or a boundary clock and actively sends PTP sync messages to all identified sensor nodes. The edge server uses the exchanged timestamps to calculate the one-way path delay to each sensor while using the exchanged timestamps to preliminarily estimate the clock offset of each sensor, and the measurement result is stored in the NodeLatency structure list.

[0025] The initialization of the end-side coordination model refers to the construction of the core instance of the coordination model by using the DeviceConfig list, the NodeLatency list and the GlobalParams object. The model object maintains several core components in the memory: a dictionary devices that stores the corresponding DeviceConfig information, a dictionary latencies that stores the corresponding NodeLatency information, including dynamically updated delays and offsets, an initialized time synchronization engine and a parameter distribution service. The time synchronization engine is bound to the network port and is used to continuously run the master clock or boundary clock logic of the PTP protocol and maintain a dynamically updated ClockOffsetTable. The parameter distribution service encapsulates the parameters related to the end side in the GlobalParams into structured instruction messages and actively sends them to all related sensor nodes through the network.

[0026] The time synchronization module comprises a communication link unit, a decoding unit, a distribution unit and a monitoring unit, wherein: the communication link unit establishes a communication link with the base station, then listens to the synchronization signal of the SIB9 synchronization information block containing the clock synchronization information from the base station, and extracts the accurate clock synchronization signal from the SIB9 synchronization information block; the decoding unit decodes the received SIB9 information block to extract the timestamp and clock accuracy parameter, then maps the decoded global timestamp to the local timestamp, to ensure the consistency of time while the communication link unit evaluates the accuracy of the synchronization signal to determine the accuracy of synchronization; the distribution unit establishes the PTP clock synchronization protocol and sends the parsed clock signal to each sensor device from node through the configured distribution mechanism to confirm synchronization through feedback; the monitoring unit involves continuously monitoring the time accuracy and synchronization state of the sensor device and recording the key events and state changes in the synchronization process, to facilitate troubleshooting and historical analysis.

[0027] In the feedback confirmation synchronization, the sensor device from node adjusts the local clock according to the received clock signal and the previously calculated communication delay, so that the timestamp is synchronized with the gateway. In addition, the system monitors and compensates the time drift between the sensor device clock and the gateway clock in real time, ensures the stability of synchronization for a long time, and verifies the accuracy of synchronization through the feedback mechanism of the time synchronization protocol, to ensure the alignment of the timestamp.

[0028] The point cloud data downsampling and compression module performs data preprocessing under the premise of ensuring the integrity of the point cloud data, cleans redundant data, calibrates data coordinates and scales, fills in missing data, then performs feature extraction on the point cloud data, calculates the weight of each point, downsamples the data based on the weight, encodes the data according to the probability of occurrence of each symbol in the data, and realizes the reduction of the storage volume and transmission delay of the data while retaining the core geometric features and attribute information of the point cloud data. The module comprises a data preprocessing unit, a point cloud data downsampling unit and a point cloud data compression unit, wherein: the data preprocessing unit uses a statistical filter to identify and remove outliers, calculates transformation parameters representing overall rotation, translation and scaling from the point cloud data, and performs data compression to reduce storage space; the point cloud data downsampling unit calculates the geometric features of each point, then calculates the weight of each point based on the features and intensity values for downsampling; the point cloud data compression unit further encapsulates the standardized data packet after obtaining the compressed point cloud block or object through data space transformation, difference information extraction and entropy-based data compression.

[0029] The point cloud data is collected by devices such as laser radar and camera, and each data point is composed of coordinates and possible other attributes such as color , reflectivity, etc.

[0030] The statistical filter described above applies to each point. Calculate its nearest Average distance of neighbors The distance is compared with the global average distance, and points that exceed the average distance by two to three times are considered outliers and removed.

[0031] The data compression specifically includes:

[0032] Step 1: Calculate the centroid of the point cloud. , It is the arithmetic mean of the point cloud coordinates, where: It is the first point cloud The coordinates of the points It represents the total number of points in the point cloud.

[0033] Step 2: Construct the covariance matrix of the point cloud. , For the covariance matrix Perform eigenvalue decomposition to obtain eigenvalues. and the corresponding feature vector , This allows us to construct a rotation matrix. , .

[0034] Step 3: Calculate the new data coordinates using the following formula. ,in: These are the original coordinates. These are the transformed coordinates.

[0035] Step 4: Calculate the scaling difference between the local point cloud data scale and the standard scale. The point cloud data is scaled to eliminate scale differences between different sensors.

[0036] Step 5: Identify and fill in missing values ​​using nearest neighbor interpolation. For each missing point... , find it nearest neighbor Calculate its mean coordinates , Afterwards, value assigned to This generates preprocessed point cloud data.

[0037] The downsampling specifically includes:

[0038] Step a: Calculate the normal for each point in the point cloud using principal component analysis to find the best-fit plane for the point and its neighborhood. For each point in the point cloud... normal ,in: is the point is the th nearest neighbor of represents principal component analysis, which is used to find the normal of the least square plane of the point and its neighbors.

[0039] Step b, evaluating the variation of the normal in the neighborhood of the point, the curvature of each point in the point cloud is calculated. The curvature can be estimated by the angle between the normal of the point and the normal of its neighbors, .

[0040] Step c, weight assignment and normalization, if the point cloud data contains intensity values, the weight is calculated in combination with the curvature and intensity values of the point, otherwise it is calculated only according to the curvature. The normalized intensity and curvature , , the weight of each point , and is dynamically adjusted according to the parameters issued by the strategy generation unit, if the point cloud data does not contain intensity information, then .

[0041] Step d, adaptive downsampling based on dynamic weight threshold, a hierarchical sampling method is applied: first, receive the dynamic downsampling rate and curvature weight threshold issued by the strategy generation unit, keep all key points with weight higher than to ensure that important geometric features are not lost, then apply probability sampling to the remaining points, the probability of each point being selected is proportional to its weight, the probability is calculated as , by adjusting the sampling ratio to make the total downsampling rate meet the requirement of ; the final generated new point cloud contains key points and non-key points selected by probability, which meets the system resource constraints while ensuring data quality.

[0042] The data space transformation refers to the conversion of point cloud data from the original three-dimensional space coordinate system to a coordinate system more suitable for compression, which is the same as the implementation method of the data coordinate calibration in the preprocessing part.

[0043] The difference information extraction refers to reducing the redundancy of data by calculating the difference between adjacent points in the point cloud, specifically: calculate the difference between each point and its adjacent points in the point cloud to obtain the difference matrix , where: is the point and its previous point The difference between the two.

[0044] The entropy-based data compression refers to assigning a coding length according to the probability of occurrence of each symbol in the data, and assigning a shorter code to a symbol with a high occurrence probability and a longer code to a symbol with a low occurrence probability, specifically: , wherein: is the entropy of the random variable , is the probability of occurrence of the symbol .

[0045] The encapsulation refers to, for each compressed point cloud block or object, the timestamp After adding the time clock information as a key attribute, the compressed data and the added time clock information are encapsulated into a standardized data packet, specifically: a structure containing a header and a data payload is constructed, the header contains metadata and time clock information, and the data payload contains compressed point cloud data, and finally a series of data units conforming to a specific protocol standard are organized to ensure the integrity and readability of the data when transmitted to the edge side.

[0046] The edge-side data fusion module comprises an analysis unit, an alignment unit and a data fusion unit, wherein: the analysis unit analyzes the data received from the sensor, which contains compressed point cloud data and additional information of the point cloud data, and generates structured format data; the alignment unit adjusts the timestamp in the structured format data based on the time correction amount calculated by the system time synchronization mechanism, and adjusts all sensor data to the same time reference; the structured format data is synchronized to the same coordinate system, and all point cloud data is unified to the same coordinate system, ensuring that all point cloud data is consistent in timestamp and spatial position; the data fusion unit uses the point cloud registration based on the ICP algorithm of the Gaussian curvature, estimates the Gaussian curvature of each point in the registration point cloud by using the property that the Gaussian curvature remains unchanged in rigid body transformation, and filters out non-key points, noise points and outliers by setting a threshold, and then uses ICP to register the point cloud containing only key points, ensuring that the point cloud data from different sensors or different time points accurately corresponds in spatial position and eliminates the deviation caused by the difference in sensor position and attitude.

[0047] The alignment specifically comprises:

[0048] Step i, restore to the original sensor coordinate system. The point cloud data is converted to a new coordinate system during preprocessing to save space. First, the data needs to be restored to the original coordinate system. Let be the processed point cloud data, and be the overall rotation matrix and translation vector applied during preprocessing, specifically: .

[0049] Step ii: Convert the point cloud data to a standard coordinate system. (Acquire sensor data) Rotation matrix in standard coordinate system Translation vector Specifically: .

[0050] Considering the different sampling rates of different sensors, point cloud data interpolation is performed for sensors with low sampling rates. Specifically, nearest neighbor search is used for point cloud data. Each point in , find it in The nearest point in Based on the distance and time between them Compared to and proportion To calculate interpolation points Perform point cloud data interpolation to realize sensor exist and Between New point cloud data is generated constantly.

[0051] The data fusion specifically includes:

[0052] Step ① For the existing point cloud data set from multiple sensors , Indicates the first Extract two sets of point cloud data from the point cloud data of each sensor. The overlapping area between the two sets of data coordinate systems is found, and two sets of sub-data are extracted. Registration is then performed based on these two sets of data. Let the source point cloud be... The target point cloud is Overlapping areas ,in: Indicates the distance between two points. This is a preset threshold used to determine the proximity between points, allowing the extraction of two sets of sub-point cloud data describing the same region. .

[0053] Step ② Calculate the multi-scale Gaussian curvature at each point in the equation: at point... of Calculate Gaussian curvature for each of the three neighborhood radii. And take their weighted average as the final curvature: ,in, yes The eigenvalues ​​of the covariance matrix of the local neighborhood. These are the coefficients of the quadratic form in that neighborhood. .

[0054] Step ③ uses the curvature weight threshold issued by the strategy generation unit to screen out key points ;

[0055] Step ④ uses the screened key points to perform feature-enhanced ICP registration, and calculates the feature similarity weight for each key point pair , where is the local feature descriptor of the point, and the target formula is: , are the corresponding point pairs in the registration process.

[0056] Step ⑤ introduces a progressive registration mechanism to evaluate the registration error change rate after each iteration , and terminates the iteration in advance when to avoid overfitting.

[0057] Step ⑥ fuses the two sets of data according to the generated transformation matrix and and replaces the original two sets of data, specifically: , and the new point cloud generated by fusing the two sets of data is: .

[0058] Step ⑦ repeats steps ①-⑥ until only one set of point cloud data remains in the set .

[0059] Preferably, a repeated point removal step is performed on the fused point cloud data to eliminate data redundancy caused by sensor overlapping areas, improve data accuracy and usability. This algorithm identifies and deletes points with similar coordinates to remove redundant data, achieving effect, is a pre-set distance threshold.

[0060] Preferably, a filtering algorithm is used to smooth the data to reduce noise and irregularities in the point cloud data, specifically: , where: is the smoothed point cloud, is the number of points in the neighborhood.

[0061] Preferably, the data fusion unit encapsulates the fused point cloud data into a standard data packet for output or further processing, which contains the fused point cloud information and related metadata such as timestamp and sensor identifier, providing accurate and consistent three-dimensional data for subsequent applications.

[0062] ​​​​The end edge information coordination model has an information aggregation unit for receiving and integrating running state data from each functional module in the system. The unit obtains network topology structure and device configuration information from a protocol configuration module, collects clock offset and synchronization accuracy indicators of each sensor node from a time synchronization module, and monitors the load state and data queue depth of a point cloud data processing module. A unified state information library is established to provide a complete global view for system collaborative decision-making.

[0063] The policy generation unit dynamically generates resource scheduling strategies and data processing parameters based on the global state data provided by the information aggregation unit. The execution process includes the following steps:

[0064] Step 1: Real-time receive and integrate running state parameters from each functional module in the system, construct global state vector , including: clock offset and synchronization error from the time synchronization module, data quality score and original data generation rate from the point cloud data downsampling and compression module, current registration error and target error from the edge side data fusion module, and network topology structure and device configuration parameters from the protocol configuration module. A unified state information library is established to provide a complete global view for system collaborative decision-making.

[0065] Step 2: Based on system real-time demand and resource limitations, establish a multi-dimensional constraint model: , where: is the processing deadline, is the fusion error threshold, is the total system bandwidth. This constraint model fully considers the strict requirements for timing consistency, fusion accuracy and resource utilization in industrial applications.

[0066] Step 3: Use the improved NSGA-II algorithm for multi-objective optimization solution, construct the optimization problem: . Specifically: first, use the Latin hypercube sampling method to initialize the population, ensuring uniform distribution of initial solutions in the decision space; then perform fast non-dominated sorting, divide the population into multiple non-dominated levels, and calculate the crowding distance of each solution, then use binary tournament selection method based on non-dominated level and crowding distance to select parent individuals; then perform simulated binary crossover and polynomial mutation operations, where an adaptive mutation operator is introduced to dynamically adjust the mutation probability , where is the initial mutation probability, Distance representing the average crowding level of the population. The maximum congestion distance is determined; a dynamic constraint processing mechanism is adopted to prioritize satisfying system constraints during non-dominated sorting; after multiple generations of evolution, the Pareto optimal solution set with uniform distribution and good convergence is output.

[0067] Step 4: Select the optimal solution from the Pareto front using the weighted compromise method and calculate key operating parameters: First, calculate the priority score of each data stream. ,in The weight coefficients are dynamically adjusted through an online reinforcement learning mechanism. To adapt to different business scenario requirements; then, an improved proportional fair scheduling algorithm is used to allocate bandwidth. ,in An adaptive exponential weight is used to dynamically adjust the network congestion level; finally, edge processing parameters, including curvature weight threshold, are dynamically generated. Downsampling rate And the compression level is adaptively selected based on network status and edge load through a predefined decision matrix.

[0068] Step 5: Encapsulate the generated resource scheduling strategy and data processing parameters into standardized control instructions and distribute them to each execution node through the parameter distribution unit; at the same time, build a strategy effect evaluation model based on the feedback data of the status monitoring unit. When the performance indicators are detected to be below expectations, trigger the strategy regeneration mechanism to form a complete closed-loop self-learning and continuous optimization process.

[0069] Through the aforementioned algorithm, this unit achieves intelligent resource scheduling and parameter adaptation under multiple spatiotemporal constraints, providing reliable technical support for industrial-grade multi-source point cloud data fusion systems.

[0070] The parameter distribution unit is responsible for distributing the processing parameters and scheduling instructions output by the strategy generation unit to each execution node in the system. This unit encapsulates the resource scheduling strategy into control instructions and sends them to the edge sensor nodes to guide them in adjusting the data acquisition and transmission priorities; at the same time, it synchronizes the point cloud processing parameters to the edge fusion module to ensure the consistency of algorithm parameters on both the edge and edge sides, and ensures that all nodes complete parameter updates in a timely manner through an acknowledgment mechanism.

[0071] The aforementioned status monitoring unit continuously monitors the operating status of each node and functional module in the system, as well as changes in the network environment. This unit tracks the online status of sensor nodes, data transmission quality, edge processing progress, and clock synchronization stability in real time. When a node anomaly or performance degradation is detected, it immediately sends an alarm to the policy generation unit, providing the system with closed-loop control and adaptive adjustment capabilities.

[0072] Through specific experiments, in a 100m*100m industrial test scene equipped with three Velodyne VLP-16 laser radars and a 5G private network, the multi-source point cloud data fusion system based on the end-edge information collaborative model of the application is run with the parameters of a target downsampling density of 15%, a Gaussian curvature initial threshold of 0.05, and a maximum number of ICP iterations of 100. When the system starts working, the protocol management module configures the protocol, sends a delay measurement package, calculates the time delay of communication between each sensor, and initializes the end-edge information collaborative model with this information, and accesses the device. The edge gateway receives the clock signal from the 5G base station and realizes accurate clock synchronization from the base station to the industrial gateway through the 5G industrial gateway and the SIB9 time synchronization mechanism of 3GPP R16. After that, the time synchronization module receives and distributes the clock synchronization signal of the base station based on the collaborative model, ensuring the time consistency of all sensor data. In this way, the sensor clock is calibrated, and the sensor starts to monitor and collect point cloud data. On the device side, the data is preprocessed, including data cleaning, coordinate transformation and scale calibration, missing value filling, and generation of preprocessed point cloud data. Then, feature extraction, weight calculation and downsampling are performed to generate compressed and optimized data packets. The edge end collects data from all sensors, aligns the data, and fuses the point cloud data through the point cloud registration and feature alignment fusion technology based on the ICP algorithm of the Gaussian curvature, to ensure the consistency of the data in the timestamp and spatial position. Finally, the generated fusion point cloud data is transmitted to the front end for display, providing technical support for industrial automation and intelligent manufacturing. The entire process relies on the dynamic optimization of the end-edge information collaborative model implementation strategy, continuously adjusts the resource scheduling and processing parameters through multi-objective decision and closed-loop feedback mechanism, and guarantees the optimal performance of the system in the variable industrial environment.

[0073] Compared with the existing method, the average time synchronization error is reduced from 125.5us to 45.2us through the boundary clock synchronization mechanism; the single-node data transmission volume is reduced from 15.8MB / s to 5.1MB / s through the curvature weight dynamic downsampling and entropy encoding compression; the point cloud registration error is reduced from 0.085m to 0.021m, and the end-to-end total delay is reduced from 240.9ms to 130.9ms by combining the improved ICP algorithm based on the Gaussian curvature, which verifies the significant improvement of the application in synchronization accuracy, transmission efficiency and fusion speed, as shown in Table 1.

[0074] Table 1 Comparison of technical characteristics

[0075] Compared with the prior art, the application realizes significant improvement of synchronization accuracy through the technology based on 5G time service and boundary clock construction in the time synchronization link, fundamentally guarantees the spatio-temporal consistency of multi-source data; in the data transmission link, through the introduction of dynamic downsampling based on curvature weight and entropy encoding compression technology, the network bandwidth occupation is significantly reduced while maintaining the key geometric features of point cloud; in the point cloud registration link, the ICP algorithm is improved by adopting the key point screening based on Gaussian curvature, which effectively improves the registration accuracy and fusion efficiency; overall, through the construction of end-edge information collaborative model for dynamic resource scheduling, the cooperation efficiency between nodes is effectively improved, and the end-to-end processing speed of the system is greatly improved.

[0076] The system can realize high-precision and high-efficiency point cloud data synchronization and fusion in the field of industrial automation and intelligent manufacturing. In the protocol configuration module, the system adopts a high-precision time synchronization protocol through dynamic scheduling of the end-edge information collaborative model, completes master-slave port definition, clock synchronization protocol configuration and delay measurement, generates a boundary clock structure and accurately calculates communication delay. Compared with the fixed and inflexible protocol configuration scheme in the prior art, the system can adapt to variable industrial environments and provide more accurate time synchronization basis. The time synchronization module relies on the unified scheduling of the end-edge collaborative model to obtain accurate time through a time service synchronization mechanism. Through steps such as establishing a stable communication connection, clock signal analysis, clock synchronization distribution and real-time monitoring compensation, high-precision clock synchronization is realized from the base station to the gateway and then to the sensor device. Compared with the low-precision synchronization mechanism of the prior art, the system significantly improves the alignment accuracy of timestamps and ensures the synchronization of sensor data. The point cloud data downsampling and compression module effectively reduces the data volume while ensuring data quality through data preprocessing, feature extraction, weight calculation and entropy encoding-based data compression. Compared with the simple and crude data compression method of the prior art, the system reduces bandwidth occupation for data transmission while maintaining data integrity through dynamic parameter adjustment. The edge-side data fusion module processes and fuses data from different sensors in real time through data analysis, alignment, interpolation and point cloud registration based on the ICP algorithm of Gaussian curvature. Compared with the prior art, the system significantly improves response speed and decision-making real-time through a closed-loop optimization mechanism, can quickly respond and provide immediate decision support, and is suitable for industrial applications with high real-time requirements.

[0077] Compared with the prior art, the system has significant advantages in reliability, efficiency, real-time performance and generalization. In terms of reliability, the system constructs a closed-loop fault-tolerant mechanism through an end-edge information collaboration model, combines high-precision time synchronization and multiple data alignment strategies, and continuously ensures stable operation of the system in complex industrial environments. The prior art is limited by synchronization accuracy and insufficient robustness of the fusion algorithm, is easily affected by environmental interference and load fluctuations, and leads to unstable data fusion results. In terms of efficiency, the system realizes dynamic downsampling and entropy encoding compression based on curvature weight under the scheduling of the end-edge information collaboration model, significantly reduces data transmission volume, and improves bandwidth utilization efficiency. The prior art has a rough data processing mechanism, occupies a large bandwidth during transmission, and is inefficient, making it difficult to apply to bandwidth-limited industrial scenarios. In terms of real-time performance, the system relies on the dynamic prediction and scheduling capabilities of the end-edge collaboration model, combines an improved ICP registration algorithm based on Gaussian curvature, realizes rapid acquisition, transmission and fusion of multi-source point cloud data, and fully meets the high real-time performance requirements of industrial applications. The prior art is limited by processing delay and often cannot provide timely data fusion results. In terms of generalization, the system has excellent system expansion and environmental adaptation capabilities through the parameter self-adaptation and resource dynamic scheduling mechanism of the end-edge information collaboration model, so that the protocol configuration and processing flow can flexibly adapt to multiple sensors and diversified industrial scenarios. The prior art has a rigid architecture and fixed parameters, making it difficult to support heterogeneous devices and diverse applications, and severely limiting its deployment range in actual industrial environments.

[0078] The above specific embodiments can be adjusted in different ways by those skilled in the art without departing from the principles and purposes of the present application. The protection scope of the present application is subject to the claims and is not limited by the above specific embodiments. Each implementation within the scope is subject to the constraints of the present application.

Claims

1. A multi-source point cloud data fusion system based on an edge-end information collaboration model, characterized in that, include: The system comprises a protocol configuration module, a time synchronization module, a point cloud data downsampling and compression module, and an edge-side data fusion module. Specifically: The protocol configuration module establishes a time synchronization architecture by importing configurations, establishes connections, calculates the time delay of data transmission from each sensor, and constructs an edge-end information collaboration model including an information aggregation unit, a strategy generation unit, a parameter distribution unit, and a status monitoring unit. Based on the configuration information, the edge-end information collaboration model constructs the overall topology and performs global resource scheduling. The time synchronization module establishes connections based on the collaboration model, receives, parses, and distributes clock synchronization signals for calibration based on the time synchronization architecture, achieving time synchronization from the 5G base station to the edge and then to the sensor devices. The point cloud data downsampling and compression module performs downsampling and entropy encoding on the collected point cloud data by evaluating point weights, compressing the data while maintaining data quality. The edge-side data fusion module receives compressed data from the sensors, performs point cloud registration using an iterative nearest-point algorithm based on Gaussian curvature, and generates an accurate 3D point cloud model integrating information from all sensors.

2. The multi-source point cloud data fusion system based on the edge-end information collaboration model according to claim 1, characterized in that, The aforementioned end-edge information collaboration model receives initialization parameters and network topology information from the protocol configuration module, clock offsets of each node from the time synchronization module, dynamically generates resource scheduling strategies and data processing parameters and distributes them to the edge-side data fusion module, while dynamically adjusting the scheduling strategy to ensure synchronization and collaboration between the end-side and the edge-side in terms of timing, resources and data processing flow.

3. The multi-source point cloud data fusion system based on the edge-end information collaboration model according to claim 1 or 2, characterized in that, The protocol configuration module includes: a boundary clock construction unit, a delay measurement and parameter recording unit, and a collaborative model construction unit. Specifically: the boundary clock construction unit uses the Precision Time Protocol (PTP) to define master and slave ports, generating a boundary clock structure with the 5G gateway as the master port and each sensor as a slave port. It then configures a clock synchronization protocol between the master and slave ports to ensure accurate distribution of clock signals to all sensors. During this process, the end-edge information collaborative model loads and parses externally provided configuration files to construct a DeviceConfig structure list. The delay measurement and parameter recording unit uses the IEEE 1588 protocol to measure the transmission delay between ports by sending delay measurement packets, synchronizing all clocks with the most accurate clock in the distributed network. The collaborative model construction unit initializes the end-edge collaborative model based on the boundary clock structure and delay measurement results. The configuration file contains the basic structure of the network, specifically including: information on all sensor nodes, the network address of the edge server, the master clock source, and data processing parameters; In the DeviceConfig structure list, each structure corresponds to a device node and a GlobalParams object, storing global parameters such as downsampling default parameters, compression algorithm selection, and the maximum number of iterations of the ICP registration algorithm. The edge-end information collaboration model performs network discovery and latency measurement based on the parsed DeviceConfig structure list.

4. The multi-source point cloud data fusion system based on the edge-end information collaboration model according to claim 1 or 2, characterized in that, The time synchronization module includes a communication link unit, a decoding unit, a distribution unit, and a monitoring unit. Specifically: the communication link unit establishes a communication link with the base station and listens for synchronization signals from SIB9 synchronization information blocks containing clock synchronization information received from the base station, extracting precise clock synchronization signals from the SIB9 synchronization information blocks; the decoding unit decodes the received SIB9 information blocks, extracting the timestamp and clock accuracy parameters, and maps the decoded global timestamp to the local timestamp to ensure time consistency while the communication link unit evaluates the accuracy of the synchronization signal to determine the accuracy of synchronization; the distribution unit establishes a PTP clock synchronization protocol and sends the parsed clock signal to each sensor device slave node through a configured distribution mechanism to confirm synchronization through feedback; the monitoring unit's synchronization status monitoring phase involves continuously monitoring the time accuracy and synchronization status of the sensor devices and recording key events and status changes during the synchronization process, facilitating fault diagnosis and historical analysis.

5. The multi-source point cloud data fusion system based on the edge-end information collaboration model according to claim 1 or 2, characterized in that, The edge-side data fusion module includes a parsing unit, an alignment unit, and a data fusion unit. The parsing unit parses the compressed point cloud data and its additional information received from the sensors and generates structured format data. The alignment unit adjusts the timestamps in the structured format data based on the time correction calculated by the system's time synchronization mechanism, aligning all sensor data to the same time reference. It also synchronizes the coordinate system of the structured format data, unifying all point cloud data into the same coordinate system to ensure consistency in timestamps and spatial location. The data fusion unit uses a Gaussian curvature-based ICP algorithm for point cloud registration. By utilizing the property that Gaussian curvature remains invariant during rigid body transformation, it estimates the Gaussian curvature of each point in the registered point cloud and filters out non-critical points, noise points, and outliers by setting a threshold. Then, it uses ICP to register the point cloud containing only critical points, ensuring accurate spatial correspondence between point cloud data from different sensors or different time points and eliminating deviations caused by differences in sensor position and attitude.

6. The multi-source point cloud data fusion system based on the edge-end information collaboration model according to claim 1 or 2, characterized in that, The point cloud data downsampling and compression module includes: a data preprocessing unit, a point cloud data downsampling unit, and a point cloud data compression unit. The data preprocessing unit uses a statistical filter to identify and remove outliers, calculates transformation parameters representing overall rotation, translation, and scaling from the point cloud data, and compresses the data to reduce storage space. The point cloud data downsampling unit calculates the geometric features of each point and then calculates the weight of each point based on the features and intensity values ​​for downsampling. The point cloud data compression unit further encapsulates the compressed point cloud blocks or objects obtained through data space transformation, difference information extraction, and entropy-based data compression to obtain standardized data packets.

7. The multi-source point cloud data fusion system based on the edge-end information collaboration model according to claim 3, characterized in that, The aforementioned synchronization of all clocks with the most accurate clock specifically includes: 1) Periodically send sync messages through the master clock and record the precise sending time of each sync message leaving the master clock. Then, the master clock will send the precise time. Encapsulate it into a Follow_Up message and send it to the slave clock; 2) Record the precise arrival time of messages from the clock. It also sends a Delay_Req message and records the exact sending time. The master clock records the precise time when the Delay_Req message arrives. And send information carrying a precise timestamp The Delay_Resp message is sent to the slave clock; 3) Calculate the time deviation and transmission delay between the master and slave clocks and output them to the edge information collaborative model for clock calibration. Specifically: time deviation... One-way transmission delay or ; The edge server acts as the master clock or boundary clock, actively sending PTP synchronization messages to all identified sensor nodes. While calculating the one-way path delay to each sensor using the exchanged timestamps, the edge server also makes a preliminary estimate of the clock offset of each sensor using the exchanged timestamps. The measurement results are stored in the NodeLatency structure list. The initialization of the edge-end collaboration model refers to: constructing a core instance of the collaboration model using a DeviceConfig list, a NodeLatency list, and a GlobalParams object; this model object maintains several core components in memory: a dictionary `devices` storing the corresponding DeviceConfig information, a dictionary `latencies` storing the corresponding NodeLatency information, including dynamically updated latency and offset, an initialized time synchronization engine, and a parameter distribution service; the time synchronization engine is bound to a network port to continuously run the PTP protocol's master clock or boundary clock logic and maintain a dynamically updated ClockOffsetTable; the parameter distribution service encapsulates the edge-side-related parameters in GlobalParams into structured command messages and actively sends them to all relevant sensor nodes via the network.

8. The multi-source point cloud data fusion system based on the edge-end information collaboration model according to claim 6, characterized in that, The alignment specifically includes: Step i: Restore to the original sensor coordinate system. Point cloud data is converted to a new coordinate system during preprocessing to save space. First, it needs to be restored to the original coordinate system. Let... It is the processed point cloud data. and It is the global rotation matrix and translation vector applied during preprocessing, specifically: ; Step ii: Convert the point cloud data to a standard coordinate system and obtain sensor data. Rotation matrix in standard coordinate system Translation vector Specifically: ; Considering the different sampling rates of different sensors, point cloud data interpolation is performed for sensors with low sampling rates. Specifically, nearest neighbor search is used for point cloud data. Each point in , find it in The nearest point in Based on the distance and time between them Compared to and proportion To calculate interpolation points Perform point cloud data interpolation to realize sensor exist and Between New point cloud data is generated continuously; The data fusion specifically includes: Step ① For the existing point cloud data set from multiple sensors , Indicates the first Extract two sets of point cloud data from the point cloud data of each sensor. Find the overlapping area between the two sets of data coordinate systems, extract the two sets of sub-data, and perform registration based on these two sets of data, making the source point cloud... The target point cloud is Overlapping areas ,in: Indicates the distance between two points. This is a preset threshold used to determine the proximity between points, thus extracting two sets of sub-point cloud data describing the same region. ; Step ② Calculate the multi-scale Gaussian curvature at each point in the equation: at point... of Calculate Gaussian curvature for each of the three neighborhood radii. And take their weighted average as the final curvature: ,in, yes The eigenvalues ​​of the covariance matrix of the local neighborhood. These are the coefficients of the quadratic form in that neighborhood. ; Step ③ Use the strategy to generate the curvature weight threshold issued by the unit. Filter out key points and ; Step 4: Perform feature-enhanced ICP registration using the selected keypoints, pairing each keypoint with... Calculate feature similarity weights ,in For the local feature descriptor of a point, the objective formula is: , and These are corresponding point pairs during the registration process; Step ⑤ introduces a progressive registration mechanism, evaluating the rate of change of registration error after each iteration. ,when Terminate the iteration early to avoid overfitting; Step 6: Based on the generated transformation matrix and The two sets of data are merged and the original two sets of data are replaced, specifically as follows: The new point cloud generated by fusing the two sets of data is: ; Step 7: Repeat steps 1-6 until only one set of point cloud data remains in the collection. .

9. The multi-source point cloud data fusion system based on the edge-end information collaboration model according to claim 6, characterized in that, The statistical filter described above applies to each point. Calculate its nearest Average distance of neighbors The distance is compared with the global average distance, and points that exceed the average distance by two to three times are considered outliers and removed. The data compression specifically includes: Step 1: Calculate the centroid of the point cloud. , It is the arithmetic mean of the point cloud coordinates, where: It is the first point cloud The coordinates of the points It is the total number of points in the point cloud; Step 2: Construct the covariance matrix of the point cloud. , For the covariance matrix Perform eigenvalue decomposition to obtain eigenvalues. and the corresponding feature vector , This constructs the rotation matrix. , ; Step 3: Calculate the new data coordinates using the following formula. ,in: These are the original coordinates. These are the transformed coordinates; Step 4: Calculate the scaling difference between the local point cloud data scale and the standard scale. Scale standardization is performed on point cloud data to eliminate scale differences between different sensors; Step 5: Identify and fill in missing values ​​using the nearest neighbor interpolation method. For each missing point... , find it nearest neighbor Calculate its mean coordinates , Afterwards, The value assigned to This generates preprocessed point cloud data. The encapsulation mentioned above refers to: for each compressed point cloud block or object, the timestamp of its acquisition. After being appended as a key attribute, the compressed data and the additional clock information are encapsulated into a standardized data packet. Specifically, a structure containing a header and a data payload is constructed. The header contains metadata and clock information, while the data payload contains the compressed point cloud data. Finally, it is organized into a series of data units that conform to a specific protocol standard to ensure data integrity and readability while transmitting the data to the edge.

10. An application architecture based on the system described in any one of claims 1-9, characterized in that, include: The system is structured from bottom to top as follows: a storage layer, a business layer, and an application layer. The storage layer stores fused point cloud data, system logs, and device information. The application layer uses the React framework and the Ant Design component library to retrieve point cloud data and node status information from the edge data fusion module via HTTP requests and displays the data. The business layer of the multi-source point cloud data fusion system imports configurations through the protocol configuration module to establish a time synchronization architecture. After establishing connections and calculating the time delay of data transmission from each sensor, it constructs an edge-end information collaboration model containing an information aggregation unit, a strategy generation unit, a parameter distribution unit, and a status monitoring unit. This model is then used to establish the edge-end information collaboration mechanism. Based on the configuration information, an overall topology structure is constructed, and global resource scheduling is performed. A connection is established on the basis of the collaborative model through a time synchronization module. Based on the time synchronization architecture, clock synchronization signals are received, parsed, and distributed for calibration, achieving time synchronization from the 5G base station to the edge and sensor devices. A point cloud data downsampling and compression module performs downsampling and entropy encoding on the collected point cloud data by evaluating the weights of the points, compressing the data while maintaining data quality. An edge-side data fusion module receives compressed data from the sensors, performs point cloud registration using an iterative nearest-point algorithm based on Gaussian curvature, and generates an accurate 3D point cloud model integrating information from all sensors. The strategy generation unit dynamically generates resource scheduling strategies and data processing parameters based on the global status data provided by the information aggregation unit. Its execution process includes the following steps: Step 1: Receive and integrate the operating status parameters from various functional modules in the system in real time to construct a global state vector. Specifically, this includes: clock offset from the time synchronization module. Synchronization error Data quality score from the point cloud data downsampling and compression module Compared with the original data generation rate Current registration error from the edge-side data fusion module Error with target and the network topology from the protocol configuration module With equipment configuration parameters By establishing a unified state information database, a complete global view can be provided for collaborative decision-making within the system. Step 2: Based on the system's real-time requirements and resource constraints, establish a multi-dimensional constraint model: ,in: To handle deadlines, The fusion error threshold, The constraint model, which takes into full account the stringent requirements for timing consistency, fusion accuracy, and resource utilization in industrial applications, is the total system bandwidth. Step 3: Use the improved NSGA-II algorithm to solve the multi-objective optimization problem, and construct the optimization problem: Specifically, the process includes: first, initializing the population using the Latin hypercube sampling method to ensure a uniform distribution of initial solutions in the decision space; then, performing fast non-dominated sorting to divide the population into multiple non-dominated levels, calculating the crowding distance for each solution, and then using a binary tournament selection method to select parent individuals based on the non-dominated level and crowding distance; subsequently, performing simulated binary crossover and polynomial mutation operations, in which an adaptive mutation operator is introduced to dynamically adjust the mutation probability according to population diversity. ,in The initial mutation probability, Distance representing the average crowding level of the population. The maximum congestion distance is determined; a dynamic constraint processing mechanism is adopted to prioritize satisfying system constraints during the non-dominated sorting process; after multiple generations of evolution, the Pareto optimal solution set with uniform distribution and good convergence is output. Step 4: Select the optimal solution from the Pareto front using the weighted compromise method and calculate key operating parameters: First, calculate the priority score of each data stream. ,in The weight coefficients are dynamically adjusted through an online reinforcement learning mechanism. To adapt to different business scenario requirements; then, an improved proportional fair scheduling algorithm is used to allocate bandwidth. ,in An adaptive exponential weight is used to dynamically adjust the network congestion level; finally, edge processing parameters, including curvature weight threshold, are dynamically generated. Downsampling rate And the compression level adaptively selected based on network status and edge load using a predefined decision matrix; Step 5: Encapsulate the generated resource scheduling strategy and data processing parameters into standardized control instructions and distribute them to each execution node through the parameter distribution unit; at the same time, build a strategy effect evaluation model based on the feedback data of the status monitoring unit. When the performance indicators are detected to be below expectations, trigger the strategy regeneration mechanism to form a complete closed-loop self-learning and continuous optimization process. The parameter distribution unit is responsible for distributing the processing parameters and scheduling instructions output by the strategy generation unit to each execution node in the system. This unit encapsulates the resource scheduling strategy into control instructions and sends them to the edge sensor nodes to guide them to adjust the data acquisition and transmission priorities. At the same time, it synchronizes the point cloud processing parameters to the edge fusion module to ensure the consistency of algorithm parameters on both the edge and edge sides, and ensures that all nodes complete parameter updates in a timely manner through an acknowledgment mechanism. The status monitoring unit continuously monitors the operating status of each node and functional module in the system and changes in the network environment. The unit tracks the online status of sensor nodes, data transmission quality, edge processing progress and clock synchronization stability in real time. When a node abnormality or performance degradation is detected, it immediately sends an alarm to the policy generation unit, providing the system with closed-loop control and adaptive adjustment capabilities.

Images