Big Data-Based Geographic Mapping Information Data Acquisition Methods and Systems

Abstract

This invention discloses a method and system for geographic mapping information acquisition based on big data, belonging to the field of geographic mapping information acquisition technology. By establishing an environmental disturbance data acquisition module, it actively collects non-topographic meteorological interference factors such as wind speed (Wv), relative humidity (Hr), precipitation particle size (Dp), and air pressure (Pa). Furthermore, it introduces a quantification mechanism for disturbance intensity values, breaking the limitation of traditional mapping systems that "only rely on laser body signals to identify anomalies." This enables the system to perceive and understand natural disturbance backgrounds, maintaining the stability of mapping signal judgment even under strong disturbance weather conditions. Through clustering and probabilistic modeling to identify anomalous samples, a noise identification function that can be used for real-time matching is constructed. This allows the system to make accurate judgments based on pattern similarity when facing nonlinear interference such as signal drift and echo abrupt changes, improving the reliability and intelligence level of noise identification.

Description

Technical Field

[0001] This invention relates to the field of geographic surveying and mapping information acquisition technology, specifically to a geographic surveying and mapping information data acquisition method and system based on big data. Background Technology

[0002] In modern surveying and mapping operations, high-density point cloud acquisition equipment is widely used for 3D terrain reconstruction, urban modeling, and resource surveys, gradually evolving from static drawings to dynamic data. Especially in practical applications, such as geological disaster monitoring, urban renewal modeling, and road surveying, higher demands are placed on the density, accuracy, and anti-interference capabilities of point cloud data. However, in these outdoor dynamic operations, laser signals are highly susceptible to external environmental factors such as wind speed fluctuations, rain particles, and sudden changes in air humidity, resulting in a large number of outliers in the surveying and mapping data, becoming a significant technical obstacle to the reliability and continuity of the system.

[0003] Currently, in practical geographic mapping systems, the processing of noise generated by dynamic environmental interference mainly relies on simple filtering at the signal end or threshold-based rule judgment. This depends on empirically preset parameters to remove laser point clouds, lacking the ability to model the coupling between environmental conditions and laser signal behavior. This approach has two main drawbacks: first, it cannot accurately identify signal abrupt changes caused by "atypical disturbances," such as fine droplet interference at the beginning of rainfall or humid-thermal resonance disturbances, which may be misjudged as normal; second, under conditions of strong environmental disturbance, in order to maintain overall data cleanliness, a large number of edge or low-intensity points are often discarded, resulting in decreased data integrity and affecting the accuracy of subsequent modeling.

[0004] The root cause of these defects lies in the lack of a complete environment-signal correlation modeling mechanism in current surveying and mapping systems. Specifically, it fails to construct a dynamic model capable of sensing the intrinsic relationship between changes in environmental parameters such as wind, rain, and humidity and the response of laser signals. Especially under severe weather disturbances, such as sudden storms, periods of active heat and moisture exchange, or regional precipitation disturbances, laser signals exhibit spatially abnormally sparse density, rapid intensity fluctuations, and imbalanced echo time delays. This ultimately leads to structural anomalies in the terrain model, such as cracks, collapses, faults, or false elevations. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the present invention provides a method and system for collecting geographic surveying and mapping information based on big data, which solves the problems mentioned in the background technology.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solution: a geographic surveying and mapping information data acquisition system based on big data, including an environmental disturbance data acquisition module, a noise pattern construction and database training module, a real-time laser signal acquisition and anomaly detection module, a multi-dimensional screening and point cloud purification module, and a point cloud fusion and terrain reconstruction module;

[0007] The environmental disturbance data acquisition module collects non-terrain signal interference sources during the mapping process, fits them into an environmental dataset HW, and calculates the disturbance intensity value Eenv.

[0008] The noise pattern construction and database training module uses the disturbance intensity value Eenv as a basis to construct and map noise event patterns, and constructs a noise probability model through clustering and outlier statistics to output the noise identification function Pno;

[0009] The real-time laser signal acquisition and anomaly detection module acquires real-time laser signals during surveying operations and combines them with the noise identification function Pno to obtain the noise label value Fabn.

[0010] The multidimensional screening and point cloud purification module analyzes the disturbance intensity value Eenv and the noise label value Faban to obtain the intensity rejection value Mcr and remove noise points.

[0011] The point cloud fusion and terrain reconstruction module integrates the noise-removed data to construct a 3D geographic model and obtain the model consistency index Cnet.

[0012] Preferably, the environmental disturbance data acquisition module includes a disturbance signal raw acquisition unit and a normalized intensity modeling unit;

[0013] The disturbance signal acquisition unit collects non-terrain signal interference sources through sensors, including wind speed Wv, relative humidity Hr, precipitation particle size Dp, and air pressure Pa;

[0014] Among them, wind speed Wv was collected by an ultrasonic anemometer; relative humidity Hr was collected by a humidity-sensitive capacitive sensor.

[0015] Precipitation particle size Dp was collected and analyzed in real time using an optical raindrop spectrometer; air pressure Pa was obtained using a miniature piezoelectric barometer.

[0016] Outliers in wind speed Wv, relative humidity Hr, precipitation particle size Dp, and air pressure Pa were processed using the local Z-value filtering method and fitted to the original dataset YW.

[0017] Preferably, the normalized intensity modeling unit normalizes the original dataset YW to obtain the environmental dataset HW;

[0018] The environment dataset HW is obtained using the following formula:

[0019] ;

[0020] In the formula, HWo represents the o-th data in the environmental dataset HW, YWo represents the o-th data in the original dataset YW, maxYWo represents the peak value of the o-th data in the original dataset YW, and minYWo represents the valley value of the o-th data in the original dataset YW.

[0021] The environmental dataset HW was analyzed to obtain the wind-humidity coupling factor F1 and the particle size-pressure coupling suppression factor F2, and then combined to calculate the disturbance intensity value Eenv.

[0022] The method for obtaining the rheumatoid coupling factor F1 is as follows: First, calculate the rate of change of wind speed Wv and relative humidity Hr, that is, the rate of change of these two quantities per unit time, and multiply them to obtain the linkage intensity of rheumatoid changes; then take the absolute value to eliminate the influence of direction, add 1, and take the natural logarithm to obtain the final rheumatoid coupling factor F1.

[0023] The particle size-pressure coupling suppression factor F2 is obtained as follows: the precipitation particle size Dp is used as the numerator; then the pressure Pa is added by 1 and the natural logarithm is taken as the denominator; finally, the numerator and denominator are divided to obtain the particle size-pressure coupling suppression factor F2.

[0024] The disturbance intensity value Eenv is obtained using the following formula:

[0025] ;

[0026] In the formula, E represents a non-zero constant.

[0027] Preferably, the noise pattern construction and database training module includes a perturbation feature extraction unit and a clustering modeling and probability recognition parameter construction unit;

[0028] The disturbance feature extraction unit extracts the laser echo intensity hisIr and echo time delay hisTd for each measurement point in the historical laser mapping data, and introduces the disturbance intensity value Eenv at the corresponding time point to construct a ternary disturbance feature vector V=(hisIr, hisTd, Eenv).

[0029] The extracted laser echo intensity hisIr and echo time delay hisTd are normalized, and a standard signal deviation factor Sdev is defined.

[0030] The standard signal deviation factor Sdev is obtained using the following formula:

[0031] ;

[0032] In the formula, Sdev(i) represents the standard signal deviation factor of the i-th mapping sample point, hisIr(i) represents the laser echo intensity of the i-th mapping sample point, hisTd represents the echo time delay of the i-th mapping sample point, μIr represents the average value of historical laser echo intensity, σIr represents the standard deviation of historical laser echo intensity, μTd represents the historical mean of echo time delay, and σTd represents the standard deviation of echo time delay.

[0033] Preferably, the probability identification parameter construction unit performs unsupervised clustering analysis on the ternary perturbation feature vector V=(hisIr, hisTd, Eenv) of all surveying sample points. By using the K-Medoids clustering method, it identifies typical noise event patterns, establishes a noise probability model, and obtains the noise identification function Pno.

[0034] Each cluster center represents a typical type of abnormal mapping behavior.

[0035] The noise identification function Pno is obtained using the following formula:

[0036] ;

[0037] In the formula, exp represents the exponential function, tanh represents the hyperbolic tangent function, Eenv(i) represents the disturbance intensity value of the i-th survey sample point, and Dclus(i) represents the Euclidean distance from the i-th survey sample point to its cluster center.

[0038] The steps to obtain the Euclidean distance Dclus(i) from the i-th survey sample point to its cluster center are as follows:

[0039] Stable cluster centers are obtained by using the K-Medoids clustering method, which is robust to outliers; the cluster affiliation label Li of each sample point and the cluster center Ck (c1, c2, c3) of each cluster are obtained.

[0040] For each survey sample point i, set a ternary perturbation feature vector Vi = (hisIr(i), hisTd(i), Eenv(i)), find its category k, and calculate its three-dimensional spatial distance from the cluster center Ck(c1, c2, c3); the formula is as follows:

[0041] ;

[0042] Preferably, the real-time laser signal acquisition and anomaly detection module includes a laser signal real-time acquisition and normalization processing unit and a noise function fusion and anomaly marking unit;

[0043] The laser signal real-time acquisition and normalization processing unit acquires real-time laser signals during the surveying operation through the detector and high-frequency timer at the laser receiver, including the current echo intensity nIr and the current echo time delay nTd, and performs normalization processing.

[0044] The noise function fusion and anomaly labeling unit calculates the standard offset Xde of the current sampling point in the signal space based on the acquired current echo intensity nIr and current echo time delay nTd.

[0045] The standard offset Xde is obtained by summing the square of the current echo intensity nIr and the square of the current echo time delay nTd, and then taking the square root.

[0046] The obtained standard offset degree Xde is combined with the noise identification function Pno to calculate the noise label value Fabn and mark the noise points.

[0047] The noise label value Fann is obtained as follows: First, the noise identification function Pno of the survey sample points is introduced, then multiplied by the adjustment coefficient and incremented by one as a weight amplification term to obtain the influence factor; finally, the influence factor is multiplied by the standard offset degree Xde of the survey sample points to obtain the noise label value Fann.

[0048] The noise points are marked as follows:

[0049] When the noise label value Fabn≤2, it means that the survey sample points are within a reasonable fluctuation range and will not be labeled;

[0050] When the noise label value Fahn > 2, it indicates that the survey sample point is a noise point and is marked as a candidate rejection point.

[0051] Preferably, the multidimensional screening and point cloud purification module performs combined analysis on the disturbance intensity value Eenv and the noise label value Faban to calculate and obtain the intensity rejection value Mcr;

[0052] The intensity rejection value Mcr is obtained using the following formula:

[0053] ;

[0054] In the formula, Mcr(i) represents the intensity removal value of the i-th survey sample point, Faban(i) represents the acoustic marker value of the i-th survey sample point, Eenv(i) represents the disturbance intensity value of the i-th survey sample point, and e represents a constant;

[0055] Based on the obtained intensity rejection value Mcr, a rejection and retention mechanism is performed on the candidate rejection points:

[0056] When the intensity rejection value Mcr > 0.3, it indicates that the survey sample point is an overload anomaly point and is directly rejected without retention; the coordinates and timestamp of the rejected point are recorded.

[0057] When the intensity rejection value Mcr ≤ 0.3, it indicates that the survey sample point is a low-confidence outlier and no processing is performed; such points are still retained in the point cloud structure, but are marked as "disturbed points".

[0058] Preferably, the point cloud fusion and terrain reconstruction module includes a point cloud fusion and 3D modeling unit and a consistency evaluation and parameter feedback unit;

[0059] The point cloud fusion and 3D modeling unit interpolates overloaded outlier points to obtain interpolated reconstructed points (pill), retains low-confidence outlier points, and marks normal retained points (Pok).

[0060] The method for obtaining the interpolation reconstruction point (pill) is as follows: First, find the neighboring points within the spatial range of the survey sample point and calculate the average Euclidean distance between the neighboring points and the current survey sample point; then, calculate the inverse distance weight based on the distance between each neighboring point and the current survey sample point; finally, use the weights to perform a weighted average of the coordinate values ​​of the neighboring points to obtain the interpolation reconstruction point (pill).

[0061] The interpolated reconstruction point pill and the normal preserved point Pok are fused to construct a highly continuous point cloud field, and a voxel grid filtering algorithm is used to generate a unified terrain grid.

[0062] Preferably, the consistency assessment and parameter feedback unit constructs a three-dimensional geographic model on the basis of a unified terrain grid and obtains the model consistency index Cnet;

[0063] The model consistency metric Cnet is obtained using the following formula:

[0064] ;

[0065] In the formula, N represents the total number of sample points, and pEenv represents the average perturbation intensity;

[0066] Analyze the obtained model consistency index Cnet to determine the effectiveness of the removal process:

[0067] When 0 < model consistency index Cnet < 0.5, it indicates that the removal effect is normal and the model is stable;

[0068] When 0.5 ≤ model consistency index Cnet < 1, it indicates that the removal effect is abnormal. The noisy pattern construction and database training module is returned to update the number of clusters and increase the training sample window to remove samples again.

[0069] The data acquisition method for geographic mapping information based on big data includes the following steps:

[0070] Step 1: The environmental disturbance data acquisition module collects non-terrain signal interference sources during the surveying process, fits them into an environmental dataset HW, and calculates and obtains the disturbance intensity value Eenv.

[0071] Step 2: The noise pattern construction and database training module uses the disturbance intensity value Eenv as a basis to construct a noise event pattern and, through clustering and outlier statistics, constructs a noise probability model to output the noise identification function Pno.

[0072] Step 3: The real-time laser signal acquisition and anomaly detection module acquires the real-time laser signal during the surveying operation and combines it with the noise identification function Pno to obtain the noise label value Fabian.

[0073] Step 4: The multidimensional screening and point cloud purification module analyzes the disturbance intensity value Eenv and the noise label value Faban to obtain the intensity rejection value Mcr, and removes the noise points.

[0074] Step 5: The point cloud fusion and terrain reconstruction module integrates the noise-removed data to construct a 3D geographic model and obtain the model consistency index Cnet.

[0075] This invention provides a method and system for collecting geographic surveying and mapping information based on big data, which has the following beneficial effects:

[0076] (1) When the system is running, by setting up an environmental disturbance data acquisition module, it actively collects non-topographic meteorological disturbance factors such as wind speed Wv, relative humidity Hr, precipitation particle size Dp and air pressure Pa, and introduces a quantification mechanism for disturbance intensity values, breaking the limitation of the traditional surveying and mapping system that "only relies on laser body signals to identify anomalies", thus enabling the system to perceive and understand the background of natural disturbances, and maintain the stability of surveying and mapping signal judgment under strong disturbance weather.

[0077] By using clustering and probabilistic modeling to identify anomalous samples, a noise identification function for real-time matching is constructed. This enables the system to make accurate judgments based on pattern similarity when facing nonlinear interference such as signal drift and echo abrupt changes, improving the reliability and intelligence of noise identification. By introducing a joint noise labeling and disturbance intensity removal mechanism, the system can dynamically balance noise removal and information preservation, avoiding false removal or preservation due to overly conservative or aggressive rules. Furthermore, an interpolation reconstruction mechanism repairs locally missing areas, improving the overall smoothness and reliability of the 3D terrain model while maintaining spatial density continuity.

[0078] (2) By setting up a disturbance signal acquisition unit, an external physical disturbance signal acquisition channel independent of terrain was introduced into the surveying and mapping data processing system for the first time. This channel includes key meteorological interference factors such as wind speed, relative humidity, precipitation particle size, and air pressure. This enables the system to acquire the environmental disturbance status of the surveying and mapping site in real time, realizing a structural shift from "relying solely on laser echo signals for self-judgment" to "actively sensing the natural disturbance background." By introducing a local Z-value filtering algorithm to preprocess the disturbance signal source, the system can identify and eliminate outliers caused by extreme mutations or random errors during the original signal sampling stage, preventing them from being misjudged or amplified in subsequent modeling, thereby improving the basic reliability of the overall data quality. This approach performs "removal-smoothing-unification" processing on the noise structure from the source, providing a cleaner and more stable data input base for subsequent disturbance modeling and point cloud anomaly identification.

[0079] (3) By setting up a disturbance feature linkage extraction unit, two key features in the laser mapping signal—echo intensity and time delay—are combined with the external disturbance intensity value to form a triplet model for the first time, thereby fully expressing the optical response characteristics and environmental background state of the measurement point at the feature level. This linkage mechanism effectively makes up for the unidirectional anomaly judgment logic of the traditional system that "only relies on laser echo intensity", and can more accurately identify signal distortion caused by disturbances such as wind, rain, and humidity, and establish a quantitative connection bridge between environmental response and mapping data.

[0080] (4) By retaining low-confidence outliers and interpolating and reconstructing high-intensity noise points, and then integrating them into a unified point cloud field, and using voxel filtering to generate a continuous terrain mesh, modeling problems such as point cloud holes, faults, and excessive sparsity can be effectively avoided. At the same time, the final effect is evaluated and a feedback mechanism is triggered through the model consistency index Cnet, which dynamically optimizes the recognition function and training window, thereby constructing a self-repairing and continuously optimizing closed-loop modeling process.

[0081] The entire process does not rely on sensor hardware in any special way. It is built entirely on existing mapping laser systems and meteorological sensors. It is optimized only through data mining and processing strategies. Without increasing additional costs, it can significantly improve the quality of geographic mapping data and has strong engineering application implementation capabilities and promotion value. Attached Figure Description

[0082] Figure 1 This is a flowchart illustrating the data acquisition method for geographic surveying and mapping information based on big data according to the present invention.

[0083] Figure 2 This is a schematic diagram illustrating the steps of the geographic surveying and mapping information data acquisition system based on big data according to the present invention;

[0084] Figure 3 This is a schematic diagram of the process for obtaining the consistency index of the model in this invention;

[0085] Figure 4 This is a trend chart of the model consistency index of the present invention. Detailed Implementation

[0086] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0087] Example 1:

[0088] This invention provides a geographic mapping information data acquisition system based on big data. Please refer to [link / reference]. Figures 1 to 4 It includes an environmental disturbance data acquisition module, a noise pattern construction and database training module, a real-time laser signal acquisition and anomaly detection module, a multi-dimensional screening and point cloud purification module, and a point cloud fusion and terrain reconstruction module.

[0089] The environmental disturbance data acquisition module collects non-terrain signal interference sources during the mapping process, fits them into an environmental dataset HW, and calculates the disturbance intensity value Eenv.

[0090] The noise pattern construction and database training module uses the disturbance intensity value Eenv as a basis to construct and map noise event patterns, and constructs a noise probability model through clustering and outlier statistics to output the noise identification function Pno;

[0091] The real-time laser signal acquisition and anomaly detection module acquires real-time laser signals during surveying operations and combines them with the noise identification function Pno to obtain the noise label value Fabn.

[0092] The multidimensional screening and point cloud purification module analyzes the disturbance intensity value Eenv and the noise label value Faban to obtain the intensity rejection value Mcr and remove noise points.

[0093] The point cloud fusion and terrain reconstruction module integrates the noise-removed data to construct a 3D geographic model and obtain the model consistency index Cnet.

[0094] In this embodiment, by establishing an environmental disturbance data acquisition module, non-topographic meteorological disturbance factors such as wind speed Wv, relative humidity Hr, precipitation particle size Dp, and air pressure Pa are actively collected. A quantification mechanism for disturbance intensity values ​​is introduced, breaking the limitation of traditional surveying and mapping systems that "only rely on laser body signals to identify anomalies". This enables the system to perceive and understand the background of natural disturbances and maintain the stability of surveying and mapping signal judgment even under strong disturbance weather.

[0095] By using clustering and probabilistic modeling to identify anomalous samples, a noise identification function for real-time matching is constructed. This enables the system to make accurate judgments based on pattern similarity when facing nonlinear interference such as signal drift and echo abrupt changes, improving the reliability and intelligence of noise identification. By introducing a joint noise labeling and disturbance intensity removal mechanism, the system can dynamically balance noise removal and information preservation, avoiding false removal or preservation due to overly conservative or aggressive rules. Furthermore, an interpolation reconstruction mechanism repairs locally missing areas, improving the overall smoothness and reliability of the 3D terrain model while maintaining spatial density continuity.

[0096] This invention introduces a consistency evaluation index in the point cloud fusion and terrain reconstruction modules to provide a "reverse evaluation" of the modeling results on the front-end recognition strategy, enabling the system to possess posterior learning capabilities. Whether the elimination strategy is excessive or the model has converged is no longer determined by human experience, but rather by the model's own output structural stability index. This achieves automatic adjustment of system parameters and recognition algorithms, as well as adaptive updating of the strategy, ensuring broad applicability in different survey areas and under different weather conditions.

[0097] Example 2:

[0098] This embodiment is an explanation based on Embodiment 1. Please refer to it. Figure 3 Specifically: the environmental disturbance data acquisition module includes a disturbance signal raw acquisition unit and a normalized intensity modeling unit;

[0099] The disturbance signal acquisition unit collects non-terrain signal interference sources through sensors, including wind speed Wv, relative humidity Hr, precipitation particle size Dp, and air pressure Pa;

[0100] Among them, wind speed Wv was collected by an ultrasonic anemometer; relative humidity Hr was collected by a humidity-sensitive capacitive sensor.

[0101] Precipitation particle size Dp was collected and analyzed in real time using an optical raindrop spectrometer; air pressure Pa was obtained using a miniature piezoelectric barometer.

[0102] Outliers in wind speed Wv, relative humidity Hr, precipitation particle size Dp, and air pressure Pa were processed using the local Z-value filtering method and fitted to the original dataset YW.

[0103] The normalized intensity modeling unit normalizes the original dataset YW to obtain the environmental dataset HW;

[0104] The environment dataset HW is obtained using the following formula:

[0105] ;

[0106] In the formula, HWo represents the o-th data in the environmental dataset HW, YWo represents the o-th data in the original dataset YW, maxYWo represents the peak value of the o-th data in the original dataset YW, and minYWo represents the valley value of the o-th data in the original dataset YW.

[0107] The environmental dataset HW was analyzed to obtain the wind-humidity coupling factor F1 and the particle size-pressure coupling suppression factor F2, and then combined to calculate the disturbance intensity value Eenv.

[0108] The method for obtaining the wind-humidity coupling factor F1 is as follows: First, calculate the rate of change of wind speed Wv and relative humidity Hr, multiply them, and obtain the linkage intensity of wind-humidity changes; then take the absolute value to eliminate the influence of direction, add 1, and take the natural logarithm to obtain the final wind-humidity coupling factor F1.

[0109] The particle size-pressure coupling suppression factor F2 is obtained as follows: the precipitation particle size Dp is used as the numerator; then the pressure Pa is added by 1 and the natural logarithm is taken as the denominator; finally, the numerator and denominator are divided to obtain the particle size-pressure coupling suppression factor F2.

[0110] The disturbance intensity value Eenv is obtained using the following formula:

[0111] ;

[0112] In the formula, E represents a non-zero constant.

[0113] In this embodiment, by setting up a disturbance signal raw acquisition unit, an external physical disturbance signal acquisition channel independent of terrain is introduced into the surveying and mapping data processing system for the first time. This includes key meteorological interference factors such as wind speed, relative humidity, precipitation particle size, and air pressure. This enables the system to acquire the environmental disturbance status of the surveying and mapping site in real time, achieving a structural shift from "relying solely on laser echo signals for self-judgment" to "actively sensing the natural disturbance background." By introducing a local Z-value filtering algorithm to preprocess the disturbance signal source, the system can identify and eliminate outliers caused by extreme mutations or random errors during the raw signal sampling stage, preventing them from being misjudged or amplified in subsequent modeling, thereby improving the basic reliability of the overall data quality. This approach performs "removal-smoothing-unification" processing on the noise structure at its source, providing a cleaner and more stable data input foundation for subsequent disturbance modeling and point cloud anomaly identification.

[0114] This embodiment constructs a wind-humidity coupling factor using a normalized intensity modeling unit, leveraging wind speed and humidity change rates. It also models precipitation particle size and air pressure together as a particle size suppression factor, ultimately fusing them into a disturbance intensity value. Compared to previous methods using a "single disturbance term + empirical weighting," this scheme dynamically models the intensity and direction of influence between different environmental disturbances through physically meaningful and structurally sound coupling logic. This enables the system to accurately determine the impact of complex weather conditions such as sudden storms, thermal and humid convergence, and fluctuating rainfall on laser ranging and echo stability, providing high-resolution and continuous disturbance perception indicators for the subsequent noise identification module.

[0115] Example 3:

[0116] This embodiment is an explanation based on Embodiment 2. Please refer to it. Figure 1 Specifically: the noise pattern construction and database training module includes a perturbation feature extraction unit and a clustering modeling and probability recognition parameter construction unit;

[0117] The disturbance feature extraction unit extracts the laser echo intensity hisIr and echo time delay hisTd for each measurement point in the historical laser mapping data, and introduces the disturbance intensity value Eenv at the corresponding time point to construct a ternary disturbance feature vector V=(hisIr, hisTd, Eenv).

[0118] The extracted laser echo intensity hisIr and echo time delay hisTd are normalized, and a standard signal deviation factor Sdev is defined.

[0119] The standard signal deviation factor Sdev is obtained using the following formula:

[0120] ;

[0121] In the formula, Sdev(i) represents the standard signal deviation factor of the i-th mapping sample point, hisIr(i) represents the laser echo intensity of the i-th mapping sample point, hisTd represents the echo time delay of the i-th mapping sample point, μIr represents the average value of historical laser echo intensity, σIr represents the standard deviation of historical laser echo intensity, μTd represents the historical mean of echo time delay, and σTd represents the standard deviation of echo time delay.

[0122] The probability identification parameter construction unit performs unsupervised clustering analysis on the ternary perturbation feature vector V=(hisIr, hisTd, Eenv) of all surveying sample points. By using the K-Medoids clustering method, it identifies typical noise event patterns, establishes a noise probability model, and obtains the noise identification function Pno.

[0123] The noise identification function Pno is obtained using the following formula:

[0124] ;

[0125] In the formula, exp represents the exponential function, tanh represents the hyperbolic tangent function, Eenv(i) represents the disturbance intensity value of the i-th survey sample point, and Dclus(i) represents the Euclidean distance from the i-th survey sample point to its cluster center.

[0126] In this embodiment, by setting up a disturbance feature linkage extraction unit, two key features in the laser mapping signal—echo intensity and time delay—are combined with the external disturbance intensity value to form a triplet model for the first time. This allows for a complete representation of the optical response characteristics and environmental background state of the measurement point at the feature level. This linkage mechanism effectively compensates for the unidirectional anomaly judgment logic of traditional systems that "only relies on laser echo intensity," enabling more accurate identification of signal distortion caused by disturbances such as wind, rain, and humidity, and establishing a quantitative connection bridge between environmental response and mapping data.

[0127] By introducing a standard signal deviation factor, the system can perform standardized and quantitative assessment of the signal deviation degree of each current mapping sample point based on the statistical patterns of historical echo signals. This mechanism can effectively avoid misjudgment and missed judgment caused by hard threshold settings. Especially under complex weather conditions, it can still distinguish signal anomalies of different degrees by gradient, providing more discriminative feature support for subsequent clustering modeling and recognition function construction.

[0128] In the clustering stage, this embodiment uses the more robust K-Medoids clustering algorithm, avoiding the sensitivity of traditional K-Means to extreme points. It can more stably identify historical outlier clusters with typical perturbation characteristics, thereby establishing a systematic noise event structure library. This structure can not only be used for data removal in the current task but also has the transferability capability of "time-crossing reuse" in future tasks, providing a technical foundation for the system to build a sustainably evolving environmental noise knowledge graph.

[0129] This embodiment ultimately constructs a continuous noise identification function based on disturbance intensity and clustering distance. Compared to previous judgment methods based on fixed rules or discrete classification, this function can output corresponding noise probability values ​​after inputting different measurement point parameters, and has the characteristics of being differentiable, quantifiable, and optimizable. This mechanism provides core algorithmic support for the system to achieve end-to-end intelligent decision-making, and has the potential to self-update based on feedback information, meeting the broad adaptability requirements of surveying and mapping systems under various terrain and climate conditions.

[0130] Example 4:

[0131] This embodiment is an explanation based on Embodiment 3. Please refer to it. Figure 1 and Figure 3 Specifically: the real-time laser signal acquisition and anomaly detection module includes a real-time laser signal acquisition and normalization processing unit and a noise function fusion and anomaly marking unit;

[0132] The laser signal real-time acquisition and normalization processing unit acquires real-time laser signals during the surveying operation through the detector and high-frequency timer at the laser receiver, including the current echo intensity nIr and the current echo time delay nTd, and performs normalization processing.

[0133] The noise function fusion and anomaly labeling unit calculates the standard offset Xde of the current sampling point in the signal space based on the acquired current echo intensity nIr and current echo time delay nTd.

[0134] The standard offset Xde is obtained by summing the square of the current echo intensity nIr and the square of the current echo time delay nTd, and then taking the square root.

[0135] The obtained standard offset degree Xde is combined with the noise identification function Pno to calculate the noise label value Fabn and mark the noise points.

[0136] The noise label value Fann is obtained as follows: First, the noise identification function Pno of the survey sample points is introduced, then multiplied by the adjustment coefficient and incremented by one as a weight amplification term to obtain the influence factor; finally, the influence factor is multiplied by the standard offset degree Xde of the survey sample points to obtain the noise label value Fann.

[0137] The noise points are marked as follows:

[0138] When the noise label value Fabn≤2, it means that the survey sample points are within a reasonable fluctuation range and will not be labeled;

[0139] When the noise label value Fahn > 2, it indicates that the survey sample point is a noise point and is marked as a candidate rejection point.

[0140] The multidimensional screening and point cloud purification module performs combined analysis on the disturbance intensity value Eenv and the noise label value Faban to calculate and obtain the intensity rejection value Mcr.

[0141] The intensity rejection value Mcr is obtained using the following formula:

[0142] ;

[0143] In the formula, Mcr(i) represents the intensity removal value of the i-th survey sample point, Faban(i) represents the acoustic marker value of the i-th survey sample point, Eenv(i) represents the disturbance intensity value of the i-th survey sample point, and e represents a constant;

[0144] Based on the obtained intensity rejection value Mcr, a rejection and retention mechanism is performed on the candidate rejection points:

[0145] When the intensity rejection value Mcr > 0.3, it indicates that the survey sample point is an overload anomaly point and is directly rejected without retention;

[0146] When the intensity rejection value Mcr ≤ 0.3, it indicates that the surveyed sample point is a low-confidence outlier and no processing is performed.

[0147] In this embodiment, a dedicated real-time laser signal acquisition and normalization processing unit is set up to synchronously acquire two core physical parameters of the laser signal—current echo intensity and time delay—using a high-frequency timing mechanism, and perform normalization processing at the acquisition end. This mechanism effectively suppresses data fluctuations caused by factors such as uneven sampling frequency, differences in terrain material, and abrupt changes in reflectivity, providing a standardized baseline range for subsequent anomaly identification and interference removal, and improving the standardization and robustness of the data preprocessing stage.

[0148] This embodiment proposes a fusion mechanism of "standard offset degree Xde + noise identification function Pno", which innovatively links the spatial offset degree of the laser signal with the environmental noise probability scoring function to obtain a continuously quantifiable noise label value Fabian, enabling gradient scoring and labeling of the anomaly degree of each measurement point. This not only improves the sensitivity of anomaly point identification, but also lays the foundation for differentiated processing of anomalies of different levels.

[0149] This embodiment jointly models the aforementioned noise marker value with the disturbance intensity value from the environmental disturbance intensity module, calculates the intensity rejection value Mcr, and realizes multi-source feature fusion judgment for laser measurement points. This mechanism abandons the previous "either-or" judgment method and introduces a "rejection and retention hierarchical mechanism": for strong interference points, they are directly rejected; for slightly disturbed points, they are retained and enter the subsequent repair link, avoiding accidental damage to sparse and effective data points and ensuring the accuracy of point cloud reconstruction.

[0150] Example 5:

[0151] This embodiment is an explanation based on Embodiment 4. Please refer to it. Figure 3 and Figure 4 Specifically: the point cloud fusion and terrain reconstruction module includes a point cloud fusion and 3D modeling unit and a consistency assessment and parameter feedback unit;

[0152] The point cloud fusion and 3D modeling unit interpolates overloaded outlier points to obtain interpolated reconstructed points (pill), retains low-confidence outlier points, and marks normal retained points (Pok).

[0153] The method for obtaining the interpolation reconstruction point (pill) is as follows: First, find the neighboring points within the spatial range of the survey sample point and calculate the average Euclidean distance between the neighboring points and the current survey sample point; then, calculate the inverse distance weight based on the distance between each neighboring point and the current survey sample point; finally, use the weights to perform a weighted average of the coordinate values ​​of the neighboring points to obtain the interpolation reconstruction point (pill).

[0154] The interpolated reconstruction point pill and the normal preserved point Pok are fused to construct a highly continuous point cloud field, and a voxel grid filtering algorithm is used to generate a unified terrain grid.

[0155] The consistency assessment and parameter feedback unit constructs a three-dimensional geographic model based on a unified terrain grid and obtains the model consistency index Cnet.

[0156] The model consistency metric Cnet is obtained using the following formula:

[0157] ;

[0158] In the formula, N represents the total number of sample points, and pEenv represents the average perturbation intensity;

[0159] Analyze the obtained model consistency index Cnet to determine the effectiveness of the removal process:

[0160] When 0 < model consistency index Cnet < 0.5, it indicates that the removal effect is normal and the model is stable;

[0161] When 0.5 ≤ model consistency index Cnet < 1, it indicates that the removal effect is abnormal. The noisy pattern construction and database training module is returned to update the number of clusters and increase the training sample window to remove samples again.

[0162] In this embodiment, traditional methods often directly discard outliers, leading to structural defects in the mapping data such as "holes," "faults," or "sparse jumps." This embodiment innovatively employs a dual-path processing strategy of "interpolation reconstruction points + normal retention points": overloaded outliers are not directly discarded but are used to generate fitted points through interpolation reconstruction; while low-confidence outliers are retained in situ. The spatial fusion of these two types of points effectively compensates for problems such as insufficient data density, blurred boundaries, and occlusion-related missing measurements, significantly enhancing the spatial continuity and morphological integrity of the point cloud field.

[0163] In this embodiment, during the interpolation calculation process, a highly adaptable local reconstruction function is constructed by calculating the inverse weight of the Euclidean distance between outliers and their surrounding neighboring points. This makes the interpolation results more closely resemble the actual terrain boundary trends, and is particularly suitable for situations such as regional data collapse caused by rain and snow interference and abnormal missing building edge points. This mechanism avoids the "structural smoothing" problem caused by blind mean interpolation, and significantly improves the detail accuracy and geometric realism of the reconstructed point cloud.

[0164] After interpolation and fusion, the system introduces a voxel mesh filtering algorithm to simplify and rasterize the point cloud, quickly generating a unified terrain mesh that meets the requirements of 3D modeling. Compared with traditional methods that directly model from the original point cloud and suffer from redundant points leading to rendering delays, this mechanism can effectively reduce data volume and compress processing costs, while ensuring the complete preservation of the terrain surface's geometric topology, thus improving the efficiency and rendering quality of digital geographic modeling.

[0165] This embodiment constructs a model consistency index, Cnet, to evaluate the stability and internal noise removal effect of the fused terrain model. If the evaluation value is abnormal, the system will trigger a feedback logic, automatically adjust the clustering strategy, expand the sample window, and restart the noise identification process, realizing a closed-loop mechanism with "evaluation-feedback-optimization". This index-driven intelligent control strategy effectively avoids the problems of "irreparable" and "unaccountable" results after one round of processing in traditional models, ensuring the adaptive and repairable capability of the model generation results.

[0166] Example 6:

[0167] For data collection methods for geographic mapping information based on big data, please refer to [link / reference]. Figure 2 Specifically, it includes the following steps:

[0168] Step 1: The environmental disturbance data acquisition module collects non-terrain signal interference sources during the surveying process, fits them into an environmental dataset HW, and calculates and obtains the disturbance intensity value Eenv.

[0169] Step 2: The noise pattern construction and database training module uses the disturbance intensity value Eenv as a basis to construct a noise event pattern and, through clustering and outlier statistics, constructs a noise probability model to output the noise identification function Pno.

[0170] Step 3: The real-time laser signal acquisition and anomaly detection module acquires the real-time laser signal during the surveying operation and combines it with the noise identification function Pno to obtain the noise label value Fabian.

[0171] Step 4: The multidimensional screening and point cloud purification module analyzes the disturbance intensity value Eenv and the noise label value Faban to obtain the intensity rejection value Mcr, and removes the noise points.

[0172] Step 5: The point cloud fusion and terrain reconstruction module integrates the noise-removed data to construct a 3D geographic model and obtain the model consistency index Cnet.

[0173] In this embodiment, multiple environmental parameters, such as wind speed, humidity, precipitation particle size, and air pressure, are introduced at the initial stage of surveying and mapping. A highly targeted and resolving disturbance intensity calculation mechanism is constructed, enabling multi-dimensional modeling and dynamic quantification of natural factors such as rain, snow, and sudden airflow. This mechanism overcomes the shortcomings of traditional surveying and mapping systems that passively accept and compensate for environmental disturbances, providing a scientific basis for subsequent noise removal. By constructing a ternary vector model of disturbance intensity and historical laser signal characteristics, and combining it with K-Medoids clustering for pattern recognition, this method generates a high-precision noise recognition function based on massive historical surveying and mapping data. This significantly improves the accuracy of identifying complex disturbances such as low-intensity raindrop scattering, signal drift, and intermittent reflection errors. Compared to traditional removal mechanisms using single thresholds or simple filtering, this method has stronger adaptability and scenario versatility.

[0174] This method, during real-time laser signal acquisition, not only performs standard offset analysis of the signal but also introduces a noise identification function for real-time matching, outputting a noise marker value (Fabn). The elimination intensity index, calculated by combining the disturbance intensity and the noise marker value, allows for precise elimination or retention based on a threshold, breaking away from the previous crude strategy of "elimination equals discard" and improving the flexibility and fidelity of data processing.

[0175] By retaining low-confidence outliers and interpolating and reconstructing high-intensity noise points, and then fusing them into a unified point cloud field, coupled with voxel filtering to generate a continuous terrain mesh, modeling problems such as point cloud holes, faults, and excessive sparsity can be effectively avoided. Simultaneously, the model consistency index Cnet is used to evaluate the final results and trigger a feedback mechanism, dynamically optimizing the recognition function and training window, thus constructing a self-healing and continuously optimizing closed-loop modeling process.

[0176] The entire process does not rely on sensor hardware in any special way. It is built entirely on existing mapping laser systems and meteorological sensors. It is optimized only through data mining and processing strategies. Without increasing additional costs, it can significantly improve the quality of geographic mapping data and has strong engineering application implementation capabilities and promotion value.

[0177] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A geographic surveying and mapping information data acquisition system based on big data, characterized in that: It includes an environmental disturbance data acquisition module, a noise pattern construction and database training module, a real-time laser signal acquisition and anomaly detection module, a multi-dimensional screening and point cloud purification module, and a point cloud fusion and terrain reconstruction module; The environmental disturbance data acquisition module collects non-terrain signal interference sources during the surveying process, fits them into an environmental dataset, and calculates the disturbance intensity value. The noise pattern construction and database training module constructs a noise event pattern based on the disturbance intensity value, and constructs a noise probability model to output a noise identification function through clustering and outlier statistics. The noise pattern construction and database training module includes a perturbation feature extraction unit and a clustering modeling and probability recognition parameter construction unit; The disturbance feature extraction unit extracts the laser echo intensity hisIr and echo time delay hisTd for each measurement point in the historical laser mapping data, and introduces the disturbance intensity value Eenv at the corresponding time point to construct a ternary disturbance feature vector V=(hisIr, hisTd, Eenv). The extracted laser echo intensity hisIr and echo time delay hisTd are normalized, and a standard signal deviation factor Sdev is defined to obtain the signal. The standard signal deviation factor Sdev is obtained using the following formula: ； In the formula, Sdev(i) represents the standard signal deviation factor of the i-th mapping sample point, hisIr(i) represents the laser echo intensity of the i-th mapping sample point, hisTd(i) represents the echo time delay of the i-th mapping sample point, μIr represents the average value of historical laser echo intensity, σIr represents the standard deviation of historical laser echo intensity, μTd represents the historical mean of echo time delay, and σTd represents the standard deviation of echo time delay; The probability identification parameter construction unit performs unsupervised clustering analysis on the ternary perturbation feature vector V=(hisIr, hisTd, Eenv) of all surveying sample points. By using the K-Medoids clustering method, it identifies typical noise event patterns, establishes a noise probability model, and obtains the noise identification function Pno. The noise identification function Pno is obtained by the following formula: ； In the formula, exp represents the exponential function, tanh represents the hyperbolic tangent function, Eenv(i) represents the disturbance intensity value of the i-th survey sample point, and Dclus(i) represents the Euclidean distance from the i-th survey sample point to its cluster center; The real-time laser signal acquisition and anomaly detection module acquires real-time laser signals during surveying operations and combines them with a noise identification function to obtain noise marker values; The multidimensional screening and point cloud purification module analyzes the disturbance intensity value and noise marker value, obtains the intensity rejection value, and removes the noise points; The point cloud fusion and terrain reconstruction module integrates the noise-removed data to construct a 3D geographic model and obtain model consistency indicators.

2. The geographic mapping information data acquisition system based on big data according to claim 1, characterized in that: The environmental disturbance data acquisition module includes a disturbance signal raw acquisition unit and a normalized intensity modeling unit; The disturbance signal acquisition unit collects non-terrain signal interference sources through sensors, including wind speed Wv, relative humidity Hr, precipitation particle size Dp, and air pressure Pa; Among them, wind speed Wv was collected by an ultrasonic anemometer; relative humidity Hr was collected by a humidity-sensitive capacitive sensor. Precipitation particle size Dp was collected and analyzed in real time using an optical raindrop spectrometer; air pressure Pa was obtained using a miniature piezoelectric barometer. Outliers in wind speed Wv, relative humidity Hr, precipitation particle size Dp, and air pressure Pa were processed and removed using the local Z-value filtering method, and then fitted to the original dataset YW.

3. The geographic mapping information data acquisition system based on big data according to claim 2, characterized in that: The normalized intensity modeling unit normalizes the original dataset YW to obtain the environmental dataset HW; The environment dataset HW is obtained using the following formula: ； In the formula, HWo represents the o-th data in the environmental dataset HW, YWo represents the o-th data in the original dataset YW, maxYWo represents the peak value of the o-th data in the original dataset YW, and minYWo represents the valley value of the o-th data in the original dataset YW. The environmental dataset HW was analyzed to obtain the wind-humidity coupling factor F1 and the particle size-pressure coupling suppression factor F2, and then combined to calculate the disturbance intensity value Eenv. The method for obtaining the wind-humidity coupling factor F1 is as follows: First, calculate the rate of change of wind speed Wv and relative humidity Hr, multiply them, and obtain the linkage intensity of wind-humidity changes; then take the absolute value to eliminate the influence of direction, add 1, and take the natural logarithm to obtain the final wind-humidity coupling factor F1. The particle size-pressure coupling suppression factor F2 is obtained as follows: the precipitation particle size Dp is used as the numerator; then the pressure Pa is added by 1 and the natural logarithm is taken as the denominator; finally, the numerator and denominator are divided to obtain the particle size-pressure coupling suppression factor F2. The disturbance intensity value Eenv is obtained using the following formula: ； In the formula, E represents a non-zero constant.

4. The geographic mapping information data acquisition system based on big data according to claim 1, characterized in that: The real-time laser signal acquisition and anomaly detection module includes a real-time laser signal acquisition and normalization processing unit and a noise function fusion and anomaly marking unit; The laser signal real-time acquisition and normalization processing unit acquires real-time laser signals during the surveying operation through the detector and high-frequency timer at the laser receiver, including the current echo intensity nIr and the current echo time delay nTd, and performs normalization processing. The noise function fusion and anomaly labeling unit calculates the standard offset Xde of the current sampling point in the signal space based on the acquired current echo intensity nIr and current echo time delay nTd. The standard offset Xde is obtained by summing the square of the current echo intensity nIr and the square of the current echo time delay nTd, and then taking the square root. The obtained standard offset degree Xde is combined with the noise identification function Pno to calculate the noise label value Fabn and mark the noise points. The noise label value Fann is obtained as follows: First, the noise identification function Pno of the survey sample points is introduced, then multiplied by the adjustment coefficient and incremented by one as a weight amplification term to obtain the influence factor; finally, the influence factor is multiplied by the standard offset degree Xde of the survey sample points to obtain the noise label value Fann. The noise points are marked as follows: When the noise label value Fabn≤2, it means that the survey sample points are within a reasonable fluctuation range and will not be labeled; When the noise label value Fahn > 2, it indicates that the surveyed sample point is a noise point and is marked as a candidate to be removed.

5. The geographic mapping information data acquisition system based on big data according to claim 4, characterized in that: The multidimensional screening and point cloud purification module performs combined analysis on the disturbance intensity value Eenv and the noise label value Faban to calculate and obtain the intensity rejection value Mcr. The intensity rejection value Mcr is obtained using the following formula: ； In the formula, Mcr(i) represents the intensity removal value of the i-th mapping sample point, Faban(i) represents the noise label value of the i-th mapping sample point, Eenv(i) represents the disturbance intensity value of the i-th mapping sample point, and e represents a constant; Based on the obtained intensity rejection value Mcr, a rejection and retention mechanism is performed on the candidate rejection points: When the intensity rejection value Mcr > 0.3, it indicates that the survey sample point is an overload anomaly point and is directly rejected without retention; When the intensity rejection value Mcr ≤ 0.3, it indicates that the surveyed sample point is a low-confidence outlier and no processing is performed.

6. The geographic mapping information data acquisition system based on big data according to claim 5, characterized in that: The point cloud fusion and terrain reconstruction module includes a point cloud fusion and 3D modeling unit and a consistency evaluation and parameter feedback unit; The point cloud fusion and 3D modeling unit interpolates overloaded outlier points to obtain interpolated reconstructed points (pill), retains low-confidence outlier points, and marks normal retained points (Pok). The method for obtaining the interpolation reconstruction point (pill) is as follows: First, find the neighboring points within the spatial range of the survey sample point and calculate the average Euclidean distance between the neighboring points and the current survey sample point; then, calculate the inverse distance weight based on the distance between each neighboring point and the current survey sample point; finally, use the weights to perform a weighted average of the coordinate values ​​of the neighboring points to obtain the interpolation reconstruction point (pill). The interpolated reconstruction point pill and the normal preserved point Pok are fused to construct a highly continuous point cloud field, and a voxel grid filtering algorithm is used to generate a unified terrain grid.

7. The geographic mapping information data acquisition system based on big data according to claim 6, characterized in that: The consistency assessment and parameter feedback unit constructs a three-dimensional geographic model based on a unified terrain grid and obtains the model consistency index Cnet. The model consistency metric Cnet is obtained using the following formula: ； In the formula, N represents the total number of sample points, and pEenv represents the average perturbation intensity; Analyze the obtained model consistency index Cnet to determine the effectiveness of the removal process: When 0 < model consistency index Cnet < 0.5, it indicates that the removal effect is normal and the model is stable; When 0.5 ≤ model consistency index Cnet < 1, it indicates that the removal effect is abnormal. The noisy pattern construction and database training module is returned to update the number of clusters and increase the training sample window, and the removal is performed again.

8. A method for collecting geographic surveying and mapping information based on big data, applied to the geographic surveying and mapping information collection system based on big data as described in any one of claims 1 to 7, characterized in that: Includes the following steps: Step 1: The environmental disturbance data acquisition module collects non-terrain signal interference sources during the surveying process, fits them into an environmental dataset HW, and calculates and obtains the disturbance intensity value Eenv. Step 2: The noise pattern construction and database training module uses the disturbance intensity value Eenv as a basis to construct a noise event pattern and, through clustering and outlier statistics, constructs a noise probability model to output the noise identification function Pno. Step 3: The real-time laser signal acquisition and anomaly detection module acquires the real-time laser signal during the surveying operation and combines it with the noise identification function Pno to obtain the noise label value Fabian. Step 4: The multidimensional screening and point cloud purification module analyzes the disturbance intensity value Eenv and the noise label value Faban to obtain the intensity rejection value Mcr, and removes the noise points. Step 5: The point cloud fusion and terrain reconstruction module integrates the noise-removed data, constructs a 3D geographic model, and obtains the model consistency index Cnet.

Images