A three-dimensional modeling method and system

Abstract

The application discloses a kind of three-dimensional modeling method and system, it is related to intelligent perception and three-dimensional reconstruction technical field, the present application is prior with BIM, visible evaluation point cloud is sampled to geometric entity, and under the sensor constraint such as field of view, ranging, incidence angle and overlap, construct graph structure environment state;On this basis, introduce the deep reinforcement learning agent that is pre-trained by synthesized scene and is migrated by domain randomization, according to the closed loop of observation-decision-execution-update, the scanning parameter is selected on-line, and after each step scanning, re-estimate visibility and coverage, combined with on-line optimization, the unexecuted sequence is adaptively modified;Compared with the method of only offline global optimization, the present application effectively compresses search space by candidate station pre-generation and visibility gating, forms continuous adjustable trade-off between coverage, point cloud quality and operation time with multi-objective reward;Face temporary occlusion, site accessibility and registration overlap degree Engineering constraints, strategy can be dynamically replanned in execution period.

Description

Technical Field

[0001] This invention relates to the field of intelligent sensing and 3D reconstruction technology, and in particular to a 3D modeling method and system. Background Technology

[0002] In the operation and maintenance management of civil infrastructure such as bridges, tunnels, and large stadiums, 3D modeling based on laser scanning has become a key means of obtaining high-precision geometric data. Point cloud data collected by laser scanners can construct digital models that reflect the actual structural state, providing a basis for inspection, analysis, and renovation. In recent years, BIM-based scanning path planning methods have been gradually applied. This method improves the integrity and efficiency of data collection by converting building information models into point cloud scenes and using multi-objective optimization algorithms to determine the distribution of scanning stations.

[0003] In existing technologies, the scanning path planning problem is an NP-hard problem, and the computational complexity increases sharply with the scale of the scene. Dynamic factors in the field environment, such as temporary facilities, vegetation obstruction, and equipment movement, cause coverage blind spots in the pre-calculated ideal path during actual execution. In multi-objective optimization, the trade-offs between different indicators are difficult to model accurately, and the optimization results are sensitive to initial conditions and may get trapped in local optima. In addition, most existing methods rely on offline computation and lack the ability to adapt to environmental changes during the scanning process.

[0004] To address these issues, some existing solutions improve optimization algorithms and introduce prior knowledge; others employ multi-objective evolutionary algorithms to balance metrics such as coverage, point cloud quality, and scanning time; still others combine point cloud visibility analysis and terrain segmentation techniques to narrow the search space for candidate stations; and some methods identify key structural regions through semantic segmentation and give these regions higher priority in path planning. However, these improvements still struggle to effectively handle dynamic changes in the field environment and ensure full-view coverage in complex structures. Summary of the Invention

[0005] In view of the aforementioned existing problems, the present invention is proposed.

[0006] This invention provides a 3D modeling method and system to solve the problems of offline planning in dealing with dynamic occlusion, difficulty in balancing multiple objectives, and high search dimensionality.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0008] In a first aspect, embodiments of the present invention provide a three-dimensional modeling method, comprising:

[0009] Step S1: Obtain the Building Information Model (BIM) of the target infrastructure, triangulate and sample the geometric entities of the BIM, and generate a point cloud scene for visibility assessment.

[0010] Step S2: Generate a set of candidate scanning stations in the work space, and under sensor constraints, calculate the visibility relationship between the candidate stations and scene sampling points based on ray projection to obtain an environmental state representation;

[0011] Step S3: Based on the environmental state representation, the first scanning action is output using the policy network of the pre-trained deep reinforcement learning agent, wherein the deep reinforcement learning agent includes a policy network and is configured to output the next station and at least one scanning parameter.

[0012] Step S4: Perform a scanning operation to register and fuse the newly acquired point cloud with the existing point cloud to form an updated point cloud;

[0013] Step S5: Based on the updated point cloud, re-evaluate the visibility and coverage, construct a real-time point cloud density map and / or semantic feature map, and use them as proxy input to obtain subsequent scanning actions;

[0014] Step S6: Output the final 3D model when the termination condition is met; otherwise, return to step S4 and repeat the process.

[0015] As a preferred embodiment of the three-dimensional modeling method described in this invention, step S1 includes:

[0016] Extract component geometry from BIM;

[0017] The components are triangulated and their surfaces are uniformly sampled according to a set step size;

[0018] The boundary extraction process removes internal points to simulate a scannable surface. This boundary extraction includes local principal direction estimation and micro-segmentation based on principal component analysis, as well as plane fitting based on random sample consistency and point projection to extract boundary points.

[0019] In a preferred embodiment of the three-dimensional modeling method described in this invention, the environmental state representation in step S2 is encoded as a graph structure.

[0020] Nodes represent candidate stations and include spatial coordinates and attitude information. Edges represent the visibility relationships between stations or between a station and a scene area, and are accompanied by visibility scores. The visibility scores are used to characterize the coverage and occlusion degree under sensor constraints.

[0021] The steps for defining the visibility score are as follows:

[0022] S21, in the candidate station With scene sampling points Ray projection is performed on the surface to obtain gating results for field of view, range, incident angle, occlusion and minimum overlap, which are used as Boolean masks for subsequent scoring;

[0023] S22, when the ray satisfies the gating condition, the geometric quality and information gain are weighted to obtain the point-level fraction:

[0024] ,

[0025] in, Indicates position Observation point Point-level visibility score, Subscript for candidate station locations As the index of the scene sampling point, For minimum overlap gate, superscript Indicates overlap. For the position The degree of overlap with the expected amount of existing point clouds, The minimum overlap threshold, To cover the door, the superscript Indicates occlusion. For field of view gate, superscript Represents field-of-view, For optical axis and connection line The included angle, The half-angle threshold of the field of view. For distance measuring gates, superscript Indicates range, To measure distance, For maximum distance measurement, For the angle of incidence goal, superscript Indicates the incidence angle. Angle of incidence The minimum incident angle threshold, This is the information gain trade-off coefficient. For geometric mass, To normalize information gain, For indicator functions, take the condition if it is true. Otherwise take ;

[0026] In the formula, geometric mass is defined as:

[0027] ,

[0028] in, For distance attenuation scale, The incident angle sensitivity index;

[0029] S23, using a weighted sum to synthesize the coverage gain and occlusion penalty into a position scalar, is used for node weights or edge weights in graph structures:

[0030] ,

[0031] in, For the position scalar indicators For coverage and occlusion weights and , This represents the total number of sampling points. For the set of sampling points, For the position pass Gated measurable subset Its base.

[0032] As a preferred embodiment of the three-dimensional modeling method described in this invention, candidate stations are generated based on the rasterization of the work area and accessibility constraints, the accessibility constraints including equipment safety buffer distance, ground flatness and restricted area shielding; and the area near the key structural region is sampled more densely according to semantic priority.

[0033] As a preferred embodiment of the three-dimensional modeling method described in this invention, the deep reinforcement learning agent is pre-trained through a synthetic point cloud scene and the agent behavior is optimized using a multi-objective reward function. The reward function comprehensively considers coverage gain, point cloud quality, and scanning time / energy consumption, and achieves transfer to the real scene through domain randomization and fine-tuning with a small amount of real data.

[0034] The multi-objective reward function is defined as follows:

[0035] S31, at each decision step The coverage gain, point cloud quality, time / energy consumption, and constraint penalties are combined into a scalar reward:

[0036] ,

[0037] in, For steps The reward and These are the weights for coverage, quality, time / energy consumption, and penalty, respectively. To cover gain fraction, Point cloud quality score, The time / energy cost score. To restrict the points awarded for violations, This is a termination condition; a positive value is taken when the task is terminated and the task threshold is met; otherwise, a negative value is taken when the task terminates due to timeout or excessive energy consumption. The value is determined by the task strategy table;

[0038] S32, with a new set of visible and gated sampling points. Measure coverage gains and maintain consistency with information gain:

[0039] ,

[0040] in, For steps Coverage score, For steps Add a new set of sampling points that pass the gate. Its base, For steps Selected station index This is the trade-off coefficient between geometric quality and information gain. For point In position The geometric mass term, Normalized information gain; It is obtained by gating based on field of view, range, angle of incidence, and occlusion.

[0041] S33 combines density, incident angle, ranging noise, and registration overlap into a normalized mass fraction:

[0042] ,

[0043] in, , ,

[0044] in, For quality fraction, And the four together are There are four weights. Density fraction, This represents the actual number of sampling points in the current statistical unit (surface patch / voxel) within step t. Indicates the nominal density of the task. For the fraction of the angle of incidence, take the fraction of the angle of incidence relative to the return point. average, , The distance measurement noise fraction, This represents the standard deviation of the average ranging within a step based on the noise model. For reference standard deviation, For distance measurement Noise prediction at that time The registration overlap score with the existing point cloud is based on the overlap ratio of the corresponding voxel / feature pair;

[0045] S34, with step duration and energy consumption forming a weighted cost score:

[0046] ,

[0047] in, For the sake of the score, and Weighted by time and energy consumption, For steps The time required For reference duration, For steps Energy consumption, For reference energy consumption; the ratio of the two items is greater than Cut off at time ;

[0048] S35 aggregates positive and negative constraints of various types using hinge functions:

[0049] ,

[0050] in, As a penalty for low scores, For a constraint set, To constrain The weight, For steps State-action pair To constrain Signed violation, Positive part operator; Terminating term Employ an indicative threshold strategy: when cumulative coverage or quality reaches a threshold... or Positive rewards are given when the cumulative duration or energy consumption reaches the limit. or A negative reward is given when the event is passively terminated, and all four termination thresholds mentioned above are preset positive numbers.

[0051] As a preferred embodiment of the three-dimensional modeling method described in this invention, the agent's observation input includes a real-time point cloud density map and / or a semantic feature map. The density map is generated by projecting or voxelizing the workspace at a fixed resolution to count the number of coverages, and the semantic feature map is generated by a point cloud segmentation model. The policy network employs a convolutional neural network and / or a graph neural network to process the above observations and output the next station and at least one scanning parameter.

[0052] As a preferred embodiment of the three-dimensional modeling method described in this invention, the visibility relationship is recalculated based on the updated point cloud after each scan step, and an online optimization algorithm is used to correct the unexecuted scan sequence to minimize the coverage blind zone and operation time. The online optimization algorithm is any one of a greedy algorithm, a genetic algorithm, or a local search algorithm.

[0053] Secondly, the present invention provides a three-dimensional modeling system, comprising,

[0054] The BIM processing module is used to convert BIM into point cloud scenes for visibility assessment.

[0055] The state construction module is used to generate candidate stations and calculate visibility relationships to form an environmental state representation.

[0056] The decision agent module, which includes a policy network submodule and a reward calculation submodule, is used to output scanning decisions based on real-time observations.

[0057] The scanning control module is used to perform scanning, complete point cloud registration and fusion, and trigger online sequence correction.

[0058] The beneficial effects of this invention are as follows: This invention uses BIM as a priori, samples geometric entities as visibility assessment point clouds, and constructs a graph-structured environmental state under the constraints of sensors such as field of view, ranging, incident angle, and overlap; on this basis, a deep reinforcement learning agent pre-trained by the synthetic scene and subjected to domain randomization transfer is introduced to select the station position and scanning parameters online in a closed loop of observation-decision-execution-update, and re-evaluates visibility and coverage after each scan step, and adaptively corrects the unexecuted sequence by combining online optimization. Compared to methods that only perform offline global optimization, this invention effectively compresses the search space through candidate site pre-generation and visibility gating, and forms a continuously adjustable trade-off between coverage, point cloud quality, and operation time with multi-objective rewards. Faced with engineering constraints such as temporary occlusion, site accessibility, and registration overlap, the strategy can dynamically replan during execution to reduce blind spots and redundancy. In complex structures (such as bridges, tunnels, and stadiums), the method drives the encryption of key areas and the explicit expression of quality constraints with semantic priority, ensuring the acquisition of high-quality point clouds that meet the needs of detection and evaluation within a limited time, thereby improving the overall acquisition efficiency, robustness, and modeling effect, and is suitable for rapid engineering deployment and large-scale operation and maintenance scenarios. Attached Figure Description

[0059] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation on the scope of this application.

[0060] Figure 1 This is a flowchart illustrating the 3D modeling method in the embodiment.

[0061] Figure 2 This is a schematic diagram of the framework of the 3D modeling system in the embodiment.

[0062] Figure 3This is a point cloud rendering based on echo intensity / reflectivity in the example.

[0063] Figure 4 This is a comparison image of multi-station point cloud registration before and after the implementation example.

[0064] Figure 5 This is the real-time point cloud density map in the embodiment.

[0065] Figure 6 This is a schematic diagram of a triangular mesh model generated from point clouds in an embodiment. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0067] All terms used in this application (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0068] For example, the terms “first” and “second” used in this application are only used to distinguish and describe similar objects, to differentiate the first object from another object, and are not used to describe a specific order or sequence, nor should they be interpreted as indicating or implying relative importance.

[0069] Example 1:

[0070] This embodiment proposes a three-dimensional modeling method, combined with... Figure 1 As shown, data is acquired using TLS on a laser scanner, including:

[0071] Step S1: Obtain the Building Information Model (BIM) of the target infrastructure, triangulate and sample the geometric entities of the BIM, and generate a point cloud scene for visibility assessment.

[0072] The data acquired by the TLS laser scanner includes at least: three-dimensional point cloud coordinates (X, Y, Z) calculated from ranging and angles, echo intensity and / or reflectivity, timestamps, and echo sequence number / scanning channel identifier; and sensor extrinsic pose information related to the current station position. When the TLS integrates an imaging unit, it also acquires RGB texture information registered with the point cloud.

[0073] Pose information is obtained through at least one of target / total station, IMU / encoder, or SLAM alignment; scanning parameters include at least one of angular resolution, vertical / horizontal stepping, laser power and / or exposure, and number of scan cycles.

[0074] In this embodiment, the Building Information Model (BIM) can be a structural model, electromechanical model, or integrated model from the design or operation and maintenance phases. A version containing the geometric and positional relationships of components is preferred to ensure that large areas of voids do not appear after triangulation. During triangulation, a sampling step size equivalent to the angular resolution of a laser scanner at the nominal measurement distance is used for uniform surface sampling, ensuring that the generated point cloud scene has a density not lower than the expected value in actual operation. Typically, 5cm to 10cm can be used for outdoor bridges and stadium facades, and 2cm to 5cm for tunnels and interiors. During sampling, surfaces with inconsistent normal vector directions are unified in one go to prevent misjudgments in subsequent gating based on the incident angle. When there are non-load-bearing components or components used only for functional representation in the BIM, these components can be optionally marked as invisible or excluded from sampling before generating the point cloud scene to reduce the number of invalid rays. If some components in the BIM are missing or their positions are changed by later maintenance, the existing point cloud blocks on site are used to locally supplement the area, prioritizing the continuity of the measured surface. The missing parts are treated as occlusions in subsequent visibility calculations.

[0075] Step S2: Generate a set of candidate scanning stations in the work space, and under sensor constraints (field of view, maximum ranging, incident angle threshold), calculate the visibility relationship between candidate stations and scene sampling points based on ray projection to obtain an environmental state representation;

[0076] Specifically, the workspace can be determined based on the grid range of the building information model or the pre-set safety distance extended outward from the outer bounding box. The commonly used extension is 0.5m to 2m, which is used to reserve space for tripod placement, mobile equipment passage, and safety buffer. Sensor constraints are determined according to the instruction manual of the laser scanner used. The half-angle of the field of view is generally taken as the equipment's announced value or slightly less than the announced value to allow for mechanical installation errors. The maximum distance is taken as the stable working distance of the equipment under the selected reflectivity. The minimum incident angle threshold is selected based on the diffuse reflectance and surface roughness of the material of the measured component. For metal or polished components, the threshold is appropriately increased to suppress low-quality points. When ray projection is performed, the coordinate system and units consistent with the point cloud scene are used to keep the coordinate origin consistent with the BIM coordinates and avoid coordinate drift during multiple reassessments of visibility. When there are temporary facilities in the workspace that are not modeled in the BIM, the area can be optionally marked as a permanent occlusion area in the environmental status representation, and its subordinate sampling points will not participate in the coverage statistics in this task.

[0077] Step S3: Based on the environmental state representation, the policy network of the pre-trained deep reinforcement learning agent outputs the first scanning action. The deep reinforcement learning agent includes the policy network and is configured to output the next station and at least one scanning parameter.

[0078] Step S4: Perform a scanning operation to register and fuse the newly acquired point cloud with the existing point cloud to form an updated point cloud;

[0079] The raw data acquired by TLS is converted into timestamped point cloud frames via the device driver or SDK. Point attributes include (but are not limited to) 3D coordinates, echo intensity / reflectivity, and echo number / channel identifier; when an imaging unit is configured, aligned RGB values ​​are also included. The system synchronously records the station's extrinsic pose for registration and fusion. The noise standard deviation can be fitted based on the device's nominal value or on-site calibration and used for calculating the quality score.

[0080] Step S5: Based on the updated point cloud, re-evaluate the visibility and coverage, construct a real-time point cloud density map and / or semantic feature map, and use them as proxy input to obtain subsequent scanning actions;

[0081] Step S6: When the termination condition (coverage, point cloud quality, or operation time threshold) is met, output the final 3D model; otherwise, return to step S4 and repeat the process.

[0082] In one embodiment, step S1 includes:

[0083] Extract component geometry from BIM;

[0084] The components are triangulated and their surfaces are uniformly sampled according to a set step size;

[0085] The boundary extraction process removes internal points to simulate a scannable surface. Boundary extraction includes local principal direction estimation and micro-segmentation based on principal component analysis, and plane fitting and point projection based on random sample consistency to extract boundary points.

[0086] In one embodiment, the environment state representation of step S2 is encoded as a graph structure:

[0087] Nodes represent candidate stations and include spatial coordinates and pose information. Edges represent the visibility relationships between stations or between a station and the scene area, and are accompanied by visibility scores. The visibility scores are used to characterize the coverage and occlusion degree under sensor constraints.

[0088] For example, the set of scene sampling points can be regarded as the set of all sampled face points after triangulation. In implementation, it can be divided into several subsets according to components, floors or sections for piecewise computation. Candidate scanning stations are represented in three-dimensional position plus attitude angle in the same coordinate system. When the gating result of a station and all sampling points is negative, the station can be marked as an inefficient station in the current iteration and removed from the graph structure to reduce the state dimension of subsequent reinforcement learning. Occlusion determination adopts the same spatial resolution and step size as ray casting, and the first collision is used as the basis for occlusion. When there are floating point errors or missed detections caused by sparse point cloud scene, it can be treated as no occlusion, but an occlusion penalty factor is applied in subsequent station-level synthesis. For key components that need to ensure measurement accuracy, their sampling points can be weighted before the visibility score is calculated to ensure that they are covered by more stations in the graph structure.

[0089] The steps to define a visibility score are as follows:

[0090] S21, in the candidate station With scene sampling points Ray projection is performed on the surface to obtain gating results for field of view, range, incident angle, occlusion and minimum overlap, which are used as Boolean masks for subsequent scoring;

[0091] S22, when the ray satisfies the gating condition, the geometric quality and information gain are weighted to obtain the point-level fraction:

[0092] ,

[0093] in, Indicates position Observation point Point-level visibility score, Subscript for candidate station locations As the index of the scene sampling point, For minimum overlap gate, superscript Indicates overlap. For the position The degree of overlap with the expected amount of existing point clouds, The minimum overlap threshold, To cover the door, the superscript Indicates occlusion. For field of view gate, superscript Represents field-of-view, For optical axis and connection line The included angle, The half-angle threshold of the field of view. For distance measuring gates, superscript Indicates range, To measure distance, For maximum distance measurement, For the angle of incidence goal, superscript Indicates the incidence angle. Angle of incidence The minimum incident angle threshold, This is the information gain trade-off coefficient. For geometric mass, To normalize information gain, For indicator functions, take the condition if it is true. Otherwise take ;

[0094] In the formula, geometric mass is defined as:

[0095] ,

[0096] in, For distance attenuation scale, The incident angle sensitivity index;

[0097] S23, using a weighted sum to synthesize the coverage gain and occlusion penalty into a position scalar, is used for node weights or edge weights in graph structures:

[0098] ,

[0099] in, For the position scalar indicators For coverage and occlusion weights and , This represents the total number of sampling points. For the set of sampling points, For the position pass Gated measurable subset Its base;

[0100] Similarly, coverage weight and occlusion weight can be allocated in engineering according to the task objectives. When the main goal is complete modeling, the coverage weight can be 0.6 to 0.8, and the occlusion weight can be 0.4 to 0.2 accordingly. When the main goal is rapid inspection, the coverage weight can be reduced to around 0.5 to obtain a faster scanning path. The information gain trade-off coefficient should be selected between 0.3 and 0.7. When the expected new information on site is limited and more of it is for patching holes or enhancing overlap, this coefficient can be lowered to rely more on the geometric quality term. The distance attenuation scale should not be less than the typical distance from the actual average station position to the measured surface, such as in a tunnel field. The field of view can be 3m to 5m wide, and the outer side of the venue or bridge can be 8m to 15m wide. The incident angle sensitivity index can be smoothly adjusted between 0 and 3 to meet the requirements of most materials. The minimum overlap threshold, field of view half angle threshold, maximum distance measurement, and minimum incident angle threshold can be set separately according to semantic partitions. For example, stricter overlap and incident angle requirements can be applied to high-value areas such as tunnel arches and bridge nodes, while more lenient requirements can be applied to unobstructed flat walls. When the number of effective measurable subsets of a certain station is less than 5% of the total number of sampling points, the station can be marked as a non-preferred station and its selection probability can be reduced in subsequent optimizations.

[0101] It can be configured to partition according to scene semantics. It can be obtained by reducing the entropy normalization of occupancy or semantic uncertainty;

[0102] Specifically, this section defines a continuous link from ray-level gating to point-level scoring and then to station-level synthesis. Point-level scores are used to prune feasible observations through gating products, and then the degree of measurability and value is expressed by a convex combination of geometric quality and information gain, which facilitates physical parameter tuning of distance and angle sensitivity. Station-level scalars place coverage gains and occlusion penalties on the same scale, and through the coordinated measurement of normalized global coverage and local occlusion of gating subsets, stations with high coverage and low occlusion are given priority in ranking. The weights are summed to one, which is conducive to smoothly switching the focus between different operational objectives.

[0103] In one embodiment, candidate stations are generated based on the rasterization of the work area and accessibility constraints, including equipment safety buffer distance, ground flatness, and no-entry zone shielding. Furthermore, dense sampling is performed near key structural areas (such as bridge piers, arch ribs, tunnel linings, and node connections) according to semantic priority. Optionally, semantic priority can be directly derived from the component category field of the building information model, or it can be obtained from an existing point cloud semantic segmentation model on a preliminary coarse scan of the point cloud. When the two do not conflict, the higher priority takes precedence. Dense sampling can be achieved by shortening the grid side length based on the original rasterization. The number of candidate stations is reduced to half or one-third of the original number, making the distribution of candidate stations around key structures denser, thus providing more action options for these areas in subsequent reinforcement learning. If temporary stands, barriers, or construction machinery exist inside the venue, making some semantically important areas inaccessible for a period of time, the semantic labels are retained in the candidate stations of that area, but the accessibility is marked as inaccessible. When the status update reflects that the area has been cleared, the stations can be automatically restored. When the input BIM model lacks semantic categories, it can degenerate into a geometric priority scheme based on height, normal, or spatial neighborhood.

[0104] Meanwhile, the deep reinforcement learning agent is pre-trained through a synthetic point cloud scene and its behavior is optimized using a multi-objective reward function. The reward function comprehensively considers coverage gain, point cloud quality, and scanning time / energy consumption, and achieves transfer to the real scene through domain randomization and fine-tuning with a small amount of real data.

[0105] Furthermore, during the construction of the synthetic point cloud scene, noise levels, occlusion density, reflectivity distribution, and the number of candidate stations can be randomly perturbed. This allows the policy network to see multiple task scales and occlusion patterns during the training phase. The typical number of training rounds can be set to several thousand to tens of thousands, with the maximum number of decision steps per round corresponding to the actual task's duration. During the transfer phase, a small amount of field-collected point cloud data can be used to fine-tune the policy. The field data can only cover a portion of the area, and rapid adaptation can be achieved by keeping the reward function structure unchanged and only adjusting the coverage weight and time penalty weight. When it is found during fine-tuning that the registration overlap of the field point cloud is consistently lower than the preset minimum overlap threshold, the termination term of that round can be set to a negative value to encourage the policy to generate more overlapping station combinations. If training resources are limited, the policy network can also be pre-trained only on the synthetic scene, and a larger time penalty can be used during online deployment to suppress unnecessary exploration.

[0106] The multi-objective reward function is defined as follows:

[0107] S31, at each decision step The coverage gain, point cloud quality, time / energy consumption, and constraint penalties are combined into a scalar reward:

[0108] ,

[0109] in, For steps The reward and These are the weights for coverage, quality, time / energy consumption, and penalty, respectively. To cover gain fraction, Point cloud quality score, The score represents the time / energy cost (the higher the cost, the higher the score). To restrict the points awarded for violations, This is a termination condition; a positive value is taken when the task is terminated and the task threshold is met; otherwise, a negative value is taken when the task terminates due to timeout or excessive energy consumption. The value is determined by the task strategy table;

[0110] S32, with a new set of visible and gated sampling points. Measure coverage gains and maintain consistency with information gain:

[0111] ,

[0112] in, For steps Coverage score, For steps Add a new set of sampling points that pass the gate. Its base, For steps Selected station index This is the trade-off coefficient between geometric quality and information gain. For point In position The geometric mass term, Normalized information gain; It is obtained by gating based on field of view, range, angle of incidence, and occlusion.

[0113] S33 combines density, incident angle, ranging noise, and registration overlap into a normalized mass fraction:

[0114] ,

[0115] in, , ,

[0116] in, For quality fraction, And the four together are There are four weights. Density fraction, This represents the actual number of sampling points in the current statistical unit (surface patch / voxel) within step t. Indicates the nominal density of the task. For the fraction of the angle of incidence, take the fraction of the angle of incidence relative to the return point. average, , The distance measurement noise fraction, This represents the standard deviation of the average ranging within a step based on the noise model. For reference standard deviation, For distance measurement Noise prediction at that time The registration overlap score with the existing point cloud is based on the overlap ratio of the corresponding voxel / feature pair;

[0117] S34, with step duration and energy consumption forming a weighted cost score:

[0118] ,

[0119] in, For the sake of the score, and Weighted by time and energy consumption, For steps The time required For reference duration, For steps Energy consumption, For reference energy consumption; the ratio of the two items is greater than It can be cut off to ;

[0120] S35 aggregates positive and negative constraints of various types using hinge functions:

[0121] ,

[0122] in, As a penalty for low scores, The constraint set includes field of view, maximum range, minimum angle of incidence, minimum overlap, safety buffer, no-entry zone, etc. To constrain The weight, For steps State-action pair To constrain Signed violations (e.g.) , (This represents the minimum angle of incidence actually observed at the selected station in step t). Positive part operator; Terminating term Employ an indicative threshold strategy: when cumulative coverage or quality reaches a threshold... or Positive rewards are given when the cumulative duration or energy consumption reaches the limit. or A negative reward is given when the event is passively terminated, where all four termination thresholds mentioned above are preset positive numbers;

[0123] In this embodiment, the coverage termination threshold can be set between 85% and 95% according to the task accuracy requirements, the quality termination threshold can be selected between 0.75 and 0.9, the time termination threshold can be estimated according to the acceptable operation time on site, for example, it can be set to 30 min to 60 min for routine bridge inspection, and 15 min to 30 min for continuous tunnel sections. The energy consumption termination threshold can be set to 70% to 90% of the nominal single battery life of the equipment to leave a safety margin. When any of the termination thresholds is reached in advance, this embodiment will prioritize ending the current round with a positive termination item. If the threshold is not reached but the time or energy consumption reaches the upper limit, it will end with a negative termination item. If the scanning action cannot continue in a certain round due to site congestion, communication abnormality or point cloud registration failure, the round will be conservatively terminated based on the current coverage area and the time consumed, and the state cache of the most recent success will be restored in the next round.

[0124] Specifically, the reward design connects coverage, quality, and cost objectives with a weighted, unified scale, facilitating continuous adjustment across different task preferences. Coverage scores directly statistically represent newly visible points and are coupled with the aforementioned information gain and geometric quality, taking into account both the area and value seen. Quality scores express the combined state of sampling density, incident angle, ranging noise, and registration overlap using a four-factor structure, providing a clear adjustment feel through interpretable weights. Time and energy consumption are combined using a normalized linear combination, reflecting both movement and scanning timing as well as device power consumption. Constraint penalties use hinged aggregation to generate continuous penalty signals for exceeding limits, helping the strategy develop robust behavior near the boundaries. Termination criteria are based on coverage / quality achievement and time / energy consumption limits, ensuring consistency between learning and task objectives.

[0125] Furthermore, the agent's observation inputs include real-time point cloud density maps and / or semantic feature maps. The density map is generated by projecting the workspace at a fixed resolution or by statistically counting the number of voxels, while the semantic feature map is generated by a point cloud segmentation model. The policy network employs convolutional neural networks and / or graph neural networks to process the above observations and output the next station location and at least one scanning parameter. For example, the real-time point cloud density map can be projected onto a horizontal or vertical plane, and the resolution should be matched to the actual station spacing and the size of the measured component, generally set to 0.1m to 0.25m per grid. The semantic feature map can undergo channel pruning before input, retaining only those relevant to the scanning decision. The component categories are optimized to reduce the computational load of the policy network. The convolutional network is used to extract local spatial patterns, and the graph network is used to propagate visibility and occlusion information between candidate stations. The outputs of both are concatenated at the decision layer to form the final action probability or action vector. When deployed on edge terminals or robot platforms with limited computing resources, only the convolutional branches can be retained and compensated with a fixed candidate station ranking table to ensure that the single-step decision time is controlled within the order of hundreds of milliseconds. If the real-time point cloud density map has short-term holes due to sensor jitter or communication delay, the most recent effective density map is used as input, and the time penalty is reduced in the reward function to avoid policy misjudgment.

[0126] After each scan step, the visibility relationship is recalculated based on the updated point cloud, and an online optimization algorithm is used to correct the unexecuted scan sequence to minimize the coverage blind zone and operation time. The online optimization algorithm can be any one of the greedy algorithm, genetic algorithm or local search algorithm.

[0127] Similarly, in actual operation, different online optimization algorithms can be selected according to the task scale. When the number of candidate stations is within dozens, a greedy algorithm can be used to obtain good coverage improvement. When the number of candidate stations reaches hundreds or even thousands, a genetic algorithm or a local search algorithm can be used to avoid getting trapped in local optima. The online optimization uses the proposed sequence of the current reinforcement learning agent as the initial solution and only adjusts the unexecuted parts to ensure that the executed parts are not destroyed. When the online optimization fails to find a better solution within the limited computation time, the system maintains the original scanning sequence and continues to execute to ensure that the job is not interrupted. If a station is detected as unreachable or blocked by on-site safety rules before execution, the station is removed from the current sequence during the online optimization stage and reordered according to the visibility score of the remaining stations.

[0128] Within the same operation time, the solution of this invention improves the newly added visible coverage and effective overlap, and reduces registration error and occlusion blind spots; see also Figures 3-6 : Figure 3 The point cloud quality distribution is shown, tinted by echo intensity / reflectivity. Figure 4 This demonstrates the effect of multi-site point cloud registration from misalignment to alignment. Figure 5 This is a real-time point cloud density map. Figure 6 The diagram shows a triangular mesh model generated from point clouds, which visually verifies the comprehensive benefits of the method in terms of coverage, quality, and efficiency.

[0129] Example 2:

[0130] This embodiment proposes a 3D modeling system that can implement the 3D modeling method described in Embodiment 1, combined with... Figure 2 As shown, the system includes:

[0131] The BIM processing module is used to convert BIM into point cloud scenes for visibility assessment.

[0132] The state construction module is used to generate candidate stations and calculate visibility relationships to form an environmental state representation.

[0133] The decision agent module, which includes a policy network submodule and a reward calculation submodule, is used to output scanning decisions based on real-time observations.

[0134] The scanning control module is used to perform scanning, complete point cloud registration and fusion, and trigger online sequence correction.

[0135] The scanning control module can be implemented on an industrial computer, a ruggedized workstation, or an embedded platform with a real-time operating system. It transmits scanning commands and returns point cloud data to the laser scanner via a wired or wireless data link. If the transmission fails, it automatically retryes at least three times and records the current station as an abnormal station. Point cloud registration prioritizes the registration method with the same overlap as the existing point cloud. After successful registration, the updated point cloud is written to the environmental state cache so that the latest point cloud can be used directly during subsequent visibility reassessment. When the scanning control module detects that the number or quality of the point cloud collected at the current station is lower than the minimum threshold set by the task, it can trigger a temporary rescan or request the decision agent to reselect adjacent stations to ensure the integrity of the final model. After the online sequence correction is triggered, the scanning control module issues control commands according to the new station order, while retaining the previous sequence as a fallback plan to deal with network interruptions or on-site emergencies.

[0136] This embodiment also provides a computer device, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the three-dimensional modeling method proposed in the above embodiment.

[0137] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0138] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, it implements the three-dimensional modeling method proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0140] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of this application and form different embodiments. For example, all the embodiments above can be used in any combination. The information disclosed in this background section is intended only to enhance the understanding of the general background of this application and should not be construed as an admission or in any way implying that such information constitutes prior art known to those skilled in the art.

Claims

1. A three-dimensional modeling method based on data acquisition using a laser scanner TLS, characterized in that, include: Step S1: Obtain the Building Information Model (BIM) of the target infrastructure, triangulate and sample the geometric entities of the BIM, and generate a point cloud scene for visibility assessment. Step S2: Generate a set of candidate scanning stations in the work space, and under sensor constraints, calculate the visibility relationship between the candidate stations and scene sampling points based on ray projection to obtain an environmental state representation; Step S3: Based on the environmental state representation, the first scanning action is output using the policy network of the pre-trained deep reinforcement learning agent, wherein the deep reinforcement learning agent includes a policy network and is configured to output the next station and at least one scanning parameter. Step S4: Perform a scanning operation to register and fuse the newly acquired point cloud with the existing point cloud to form an updated point cloud; Step S5: Based on the updated point cloud, re-evaluate the visibility and coverage, construct a real-time point cloud density map and / or semantic feature map, and use them as proxy input to obtain subsequent scanning actions; Step S6: Output the final 3D model when the termination condition is met; otherwise, return to step S4 and repeat the process. The environmental state representation is encoded as a graph structure: Nodes represent candidate stations and include spatial coordinates and attitude information. Edges represent the visibility relationships between stations or between a station and a scene area, and are accompanied by visibility scores. The visibility scores are used to characterize the coverage and occlusion degree under sensor constraints. The deep reinforcement learning agent is pre-trained using a synthetic point cloud scene and its behavior is optimized using a multi-objective reward function. The reward function comprehensively considers coverage gain, point cloud quality, and scanning time / energy consumption, and achieves transfer to the real scene through domain randomization and fine-tuning with a small amount of real data.

2. The three-dimensional modeling method as described in claim 1, characterized in that, Step S1 includes: Extract component geometry from BIM; The components are triangulated and their surfaces are uniformly sampled according to a set step size; The boundary extraction process removes internal points to simulate a scannable surface. This boundary extraction includes local principal direction estimation and micro-segmentation based on principal component analysis, as well as plane fitting based on random sample consistency and point projection to extract boundary points.

3. The three-dimensional modeling method as described in claim 1, characterized in that, The steps for defining the visibility score are as follows: S21, in the candidate station With scene sampling points Ray projection is performed on the surface to obtain gating results for field of view, range, incident angle, occlusion and minimum overlap, which are used as Boolean masks for subsequent scoring; S22, when the ray satisfies the gating condition, the geometric quality and information gain are weighted to obtain the point-level fraction: ， in, Indicates position Observation point Point-level visibility score, Subscript for candidate station locations As the index of the scene sampling point, For minimum overlap gate, superscript Indicates overlap. For the position The degree of overlap with the expected amount of existing point clouds, The minimum overlap threshold, To cover the door, the superscript Indicates occlusion. For field of view gate, superscript Represents field-of-view, For optical axis and connection line The included angle, The half-angle threshold of the field of view. For distance measuring gates, superscript Indicates range, To measure distance, For maximum distance measurement, For the angle of incidence goal, superscript Indicates the incidence angle. Angle of incidence The minimum incident angle threshold, This is the information gain trade-off coefficient. For geometric mass, To normalize information gain, For indicator functions, take the condition if it is true. Otherwise take .

4. The three-dimensional modeling method as described in claim 3, characterized in that, In the step of defining the visibility score, the geometric quality is defined as follows: ， in, For distance attenuation scale, The incident angle sensitivity index; The steps for defining visibility scores also include: S23, using a weighted sum to synthesize the coverage gain and occlusion penalty into a position scalar, is used for node weights or edge weights in graph structures: ， in, For the position scalar indicators For coverage and occlusion weights and , This represents the total number of sampling points. For the set of sampling points, For the position pass Gated measurable subset Its base.

5. The three-dimensional modeling method as described in claim 1, characterized in that, Candidate stations are generated based on the gridding of the work area and accessibility constraints, including equipment safety buffer distance, ground flatness, and restricted area shielding; and sampling is performed near key structural areas based on semantic priority.

6. The three-dimensional modeling method as described in claim 1, characterized in that, The multi-objective reward function is defined as follows: S31, at each decision step The coverage gain, point cloud quality, time / energy consumption, and constraint penalties are combined into a scalar reward: ， in, For steps The reward and These are the weights for coverage, quality, time / energy consumption, and penalty, respectively. To cover gain fraction, Point cloud quality score, The time / energy cost score. To restrict the scoring of violations, This is a termination condition; a positive value is taken when the task is terminated and the task threshold is met; otherwise, a negative value is taken when the task terminates due to timeout or excessive energy consumption. The value is determined by the task strategy table; S32, with a new set of visible and gated sampling points. Measure coverage gains and maintain consistency with information gain: ， in, For steps Coverage score, For steps Add a new set of sampling points that pass the gate. Its base, For steps Selected station index This is the trade-off coefficient between geometric quality and information gain. For point In position The geometric mass term, Normalized information gain; It is obtained by screening based on field of view, range, angle of incidence, and gating of obstruction.

7. The three-dimensional modeling method as described in claim 6, characterized in that, The steps for setting the multi-objective reward function also include: S33 combines density, incident angle, ranging noise, and registration overlap into a normalized mass fraction: ， in, , , in, For quality fraction, And the four together are There are four weights. Density fraction, This represents the actual number of sampling points in the current statistical unit within step t. Indicates the nominal density of the task. For the fraction of the angle of incidence, take the fraction of the angle of incidence relative to the return point. average, , The distance measurement noise fraction, This represents the standard deviation of the average ranging within a step based on the noise model. For reference standard deviation, For distance measurement Noise prediction at that time The registration overlap score with the existing point cloud is based on the overlap ratio of the corresponding voxel / feature pair; S34, with step duration and energy consumption forming a weighted cost score: ， in, For the sake of the score, and Weighted by time and energy consumption, For steps The time required For reference duration, For steps Energy consumption, For reference energy consumption; the ratio of the two items is greater than Cut off at time ; S35 aggregates positive and negative constraints of various types using hinge functions: ， in, As a penalty for low scores, For a constraint set, To constrain The weight, For steps State-action pair To constrain Signed violation, Positive part operator; Terminating term Employ an indicative threshold strategy: when cumulative coverage or quality reaches a threshold... or Positive rewards are given when the cumulative duration or energy consumption reaches the limit. or A negative reward is given when the event is passively terminated, and all four termination thresholds mentioned above are preset positive numbers.

8. The three-dimensional modeling method as described in claim 1, characterized in that, The agent's observation inputs include a real-time point cloud density map and / or a semantic feature map. The density map is generated by projecting or voxelizing the workspace at a fixed resolution to count the number of coverages, and the semantic feature map is generated by a point cloud segmentation model. The policy network uses a convolutional neural network and / or a graph neural network to process the above observations and output the next station and at least one scanning parameter.

9. The three-dimensional modeling method as described in claim 1, characterized in that, After each scan step, the visibility relationship is recalculated based on the updated point cloud, and an online optimization algorithm is used to correct the unexecuted scan sequence to minimize the coverage blind zone and operation time. The online optimization algorithm can be any one of a greedy algorithm, a genetic algorithm, or a local search algorithm.

10. A three-dimensional modeling system, based on the three-dimensional modeling method according to any one of claims 1 to 9, characterized in that, include: The BIM processing module is used to convert BIM into point cloud scenes for visibility assessment. The state construction module is used to generate candidate stations and calculate visibility relationships to form an environmental state representation. The decision agent module, which includes a policy network submodule and a reward calculation submodule, is used to output scanning decisions based on real-time observations. The scanning control module is used to perform scanning, complete point cloud registration and fusion, and trigger online sequence correction.

Images