Three-dimensional modeling method and system

Abstract

The invention discloses a three-dimensional modeling method and system, and relates to the technical field of intelligent perception and three-dimensional reconstruction, and the method comprises the steps: taking BIM as a priori, sampling a geometric entity as a visibility evaluation point cloud, and constructing a graph structure environment state under the constraints of a field of view, distance measurement, an incident angle, overlapping and other sensors; on the basis, a deep reinforcement learning agent which is pre-trained by a synthetic scene and subjected to domain randomization migration is introduced, stations and scanning parameters are selected online according to an observation-decision-execution-update closed loop, visibility and coverage are re-estimated after each step of scanning, and self-adaptive correction is carried out on an unexecuted sequence in combination with online optimization; compared with an off-line global optimization method, the method has the advantages that a search space is effectively compressed through candidate station pre-generation and visibility gating, and continuous and adjustable balance is formed among coverage, point cloud quality and operation time by multi-target awards; for engineering constraints such as temporary shielding, site reachability, registration overlapping degree and the like, the strategy can be dynamically replanned in an execution period.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent sensing and three-dimensional reconstruction, and in particular to a three-dimensional modeling method and system. BACKGROUND

[0002] In the operation and maintenance management of civil infrastructure such as bridges, tunnels and large venues, three-dimensional modeling based on laser scanning has become a key means to obtain high-precision geometric data. Point cloud data collected by a laser scanner can construct a digital model reflecting the actual structure state, providing a basis for detection, analysis and reconstruction. In recent years, scanning path planning methods based on BIM have been gradually applied. This method converts the building information model into a point cloud scene and uses a multi-objective optimization algorithm to determine the scanning station distribution, improving the completeness and efficiency of data collection.

[0003] In the prior art, the scanning path planning problem is an NP-hard problem, and the computational complexity increases sharply with the size of the scene. Dynamic factors in the field environment, such as temporary facilities, vegetation obstruction and equipment movement, cause the ideal path calculated in advance to have blind areas when actually executed. The trade-off relationship between different indicators in multi-objective optimization is difficult to accurately model, and the optimization result is sensitive to the initial conditions, which may fall into a local optimum. In addition, most existing methods rely on offline calculation and lack the ability to adapt to changes in the environment during scanning.

[0004] To address these issues, some existing solutions improve optimization algorithms and introduce prior knowledge. Some solutions use decomposition multi-objective evolutionary algorithms to balance coverage, point cloud quality and scanning time. Some solutions combine point cloud visibility analysis and terrain segmentation techniques to narrow down the search space for candidate stations. Some methods identify key structural areas through semantic segmentation and give these areas higher priority in path planning. However, these improvements still struggle to effectively handle dynamic changes in the field environment and ensure full-angle coverage in complex structures. SUMMARY

[0005] In view of the above existing problems, the present application is proposed.

[0006] The present application provides a three-dimensional modeling method and system to solve the problems of offline planning difficulty in dealing with dynamic obstruction, multi-objective trade-off difficulty and high search dimension.

[0007] To solve the above technical problems, the present application provides the following technical solutions:

[0008] In a first aspect, the present application provides a three-dimensional modeling method, comprising:

[0009] Step S1, obtain the building information model BIM of the target infrastructure, triangulate the geometric entities of the BIM and sample to generate a point cloud scene for visibility evaluation;

[0010] Step S2, generate a set of candidate scanning stations in the working space, and based on ray casting, calculate the visibility relationship between the candidate stations and the scene sampling points under the constraint of the sensor to obtain an environment state representation;

[0011] Step S3, based on the environment state representation, output a first scanning action using the policy network of a pre-trained deep reinforcement learning agent, the deep reinforcement learning agent comprising a policy network and being configured to output a next station and at least one scanning parameter;

[0012] Step S4, perform a scanning operation, register and fuse the newly collected point cloud with the existing point cloud to form an updated point cloud;

[0013] Step S5, re-estimate the visibility and coverage according to the updated point cloud, construct a real-time point cloud density map and / or semantic feature map as an agent input to obtain a subsequent scanning action;

[0014] Step S6, output a final three-dimensional model when the termination condition is met, otherwise return to step S4 for cyclic execution.

[0015] As a preferred scheme of the three-dimensional modeling method, step S1 comprises:

[0016] extracting the component geometry in the BIM;

[0017] triangulating the component and uniformly sampling the surface at a set step;

[0018] removing internal points to simulate a scannable surface by boundary extraction, the boundary extraction comprising local principal direction estimation and micro-section segmentation based on principal component analysis, and plane fitting and boundary point projection based on random sample consistency to extract boundary points.

[0019] As a preferred scheme of the three-dimensional modeling method, the environment state representation of step S2 is encoded as a graph structure:

[0020] The node represents the candidate station and contains spatial coordinates and attitude information, the edge represents the visibility relationship between the stations or between the station and the scene region, and is accompanied by a visibility score, the visibility score being used to represent the coverage and occlusion degree under the constraint of the sensor;

[0021] The definition step of the visibility score is as follows:

[0022] S21, at the candidate station and the scene sampling point The ray casting is performed to obtain the field-of-view, range, incidence angle, occlusion, and minimum overlap gating results as Boolean masks for subsequent scoring.

[0023] S22, when the ray satisfies the gating, the geometric quality and information gain are weighted to obtain a point-level score:

[0024] ,

[0025] wherein, denotes the station position observation point point-level visibility score, is the candidate station position index, is the scene sampling point index, is the minimum overlap gate, the superscript denotes overlap, is the expected overlap of the station position with the existing point cloud, is the minimum overlap threshold, is the occlusion gate, the superscript denotes occlusion, is the field-of-view gate, the superscript denotes field-of-view, is the angle between the optical axis and the connecting line , is the field-of-view half-angle threshold, is the range gate, the superscript denotes range, is the measured distance, is the maximum range, is the incidence angle gate, the superscript denotes incidence angle, is the incidence angle, is the minimum incidence angle threshold, is the information gain weighting coefficient, is the geometric quality term, is the normalized information gain, is an indicator function, and if the condition is true, otherwise;

[0026] In the formula, the geometric quality is defined as:

[0027] ,

[0028] wherein, is the distance attenuation scale, is the incidence angle sensitivity index;

[0029] S23, using a weighted sum to synthesize the coverage gain and occlusion penalty into a position scalar, is used for node weights or edge weights in graph structures:

[0030] ,

[0031] in, For the position scalar indicators For coverage and occlusion weights and , This represents the total number of sampling points. For the set of sampling points, For the position pass Gated measurable subset Its base.

[0032] As a preferred embodiment of the three-dimensional modeling method described in this invention, candidate stations are generated based on the rasterization of the work area and accessibility constraints, the accessibility constraints including equipment safety buffer distance, ground flatness and restricted area shielding; and the area near the key structural region is sampled more densely according to semantic priority.

[0033] As a preferred embodiment of the three-dimensional modeling method described in this invention, the deep reinforcement learning agent is pre-trained through a synthetic point cloud scene and the agent behavior is optimized using a multi-objective reward function. The reward function comprehensively considers coverage gain, point cloud quality, and scanning time / energy consumption, and achieves transfer to the real scene through domain randomization and fine-tuning with a small amount of real data.

[0034] The multi-objective reward function is defined as follows:

[0035] S31, at each decision step The coverage gain, point cloud quality, time / energy consumption, and constraint penalties are combined into a scalar reward:

[0036] ,

[0037] in, For steps The reward and These are the weights for coverage, quality, time / energy consumption, and penalty, respectively. For coverage gain fraction, Point cloud quality score, The time / energy cost score. To restrict the points awarded for violations, This is a termination condition; a positive value is taken when the task is terminated and the task threshold is met; otherwise, a negative value is taken when the task terminates due to timeout or excessive energy consumption. The value is determined by the task strategy table;

[0038] S32, new visible and gated sample points set Metric coverage gain, consistent with information gain:

[0039] ,

[0040] where, is the coverage score at step , is the new gated sample points set at step , is the cardinality of it, is the selected station index at step , is the trade-off coefficient between geometric quality and information gain, is the geometric quality term at station , , is the normalized information gain; filtered by FOV, ranging, incidence angle and occlusion gating;

[0041] S33, density, incidence angle, ranging noise and registration overlap are integrated into the normalized quality score:

[0042] ,

[0043] where, , ,

[0044] where, is the quality score, and the sum of the four is , is the four weights, is the density score, denotes the actual number of sampled points in the current statistical unit (surface patch / voxel) at step t, denotes the nominal density of the task, is the incidence angle score, taking the average of the incidence angles of the returned points, , is the ranging noise score, denotes the average ranging standard deviation at step based on the noise model, is the reference standard deviation, is the noise prediction when the ranging is , is the registration overlap score with the point cloud, based on the overlap ratio of the corresponding voxel / feature pairs;

[0045] S34, the weighted cost score is composed of the step length and energy consumption:

[0046] ,

[0047] wherein, is a cost score, and is a time and energy weight, is a step time, is a reference time length, is a step energy consumption, is a reference energy consumption; the ratio of the two is greater than when truncated to ;

[0048] S35, aggregate the positive violation of each constraint with a hinge function:

[0049] ,

[0050] wherein, is a penalty score, is a constraint set, is a weight of the constraint , is a state-action pair of the step , is a signed violation of the constraint , is a positive part operator; the termination item uses an indicative threshold strategy: when the cumulative coverage or quality reaches the threshold or , a positive reward is given, when the cumulative time length or energy consumption reaches the upper limit or is passively terminated, a negative reward is given, wherein the above four termination thresholds are all preset positive numbers.

[0051] As a preferred scheme of the three-dimensional modeling method, wherein: the observation input of the agent includes a real-time point cloud density map and / or a semantic feature map, the density map is generated by projecting or voxelizing the working space at a fixed resolution to count the number of coverages, and the semantic feature map is generated by a point cloud segmentation model; the strategy network uses a convolutional neural network and / or a graph neural network to process the above observations and output the next station and at least one scanning parameter.

[0052] As a preferred scheme of the three-dimensional modeling method, wherein: after each step of scanning, the visibility relationship is recalculated based on the updated point cloud, and an online optimization algorithm is used to correct the unexecuted scanning sequence to minimize the coverage blind area and the working time, the online optimization algorithm is any one of a greedy algorithm, a genetic algorithm or a local search algorithm.

[0053] In a second aspect, the present application provides a three-dimensional modeling system, comprising,

[0054] a BIM processing module for converting BIM into a point cloud scene for visibility evaluation;

[0055] a state construction module for generating candidate station positions and calculating visibility relationships to form an environment state representation;

[0056] a decision agent module containing a policy network submodule and a reward calculation submodule for outputting scanning decisions according to real-time observations;

[0057] a scanning control module for executing scanning, completing point cloud registration fusion, and triggering online sequence correction.

[0058] The present application has the following advantages: the present application uses BIM as a priori, samples geometric entities into visibility evaluation point clouds, and constructs a graph structure environment state under sensor constraints such as field of view, ranging, incident angle, and overlap; on this basis, a deep reinforcement learning agent pre-trained on a synthesized scene and migrated through domain randomization is introduced, which selects station positions and scanning parameters in a closed loop of observation-decision-execution-update, and reevaluates visibility and coverage after each scan, and adaptively corrects the unexecuted sequence in combination with online optimization. Compared with the method of only offline global optimization, the present application effectively compresses the search space through candidate station position pre-generation and visibility gating, forms a continuous adjustable trade-off among coverage, point cloud quality, and operation time through multi-objective rewards, dynamically replans the strategy during the execution period in the face of engineering constraints such as temporary occlusion, site accessibility, and registration overlap, reduces blind areas and redundancies, and in complex structures such as bridges, tunnels, and stadiums, the method drives key area encryption and quality constraints with semantic priority, ensures that high-quality point clouds meeting detection and evaluation needs are obtained within a limited time, thereby improving overall acquisition efficiency, robustness, and modeling effect, and is suitable for engineering rapid deployment and large-scale operation scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope of the present application.

[0060] Figure 1 It is a flowchart of the three-dimensional modeling method in the embodiments.

[0061] Figure 2 It is a framework diagram of the three-dimensional modeling system in the embodiments.

[0062] Figure 3A point cloud rendering based on echo intensity / reflectivity in the embodiment.

[0063] Figure 4 A comparison chart before and after multi-station point cloud registration in the embodiment.

[0064] Figure 5 A real-time point cloud density chart in the embodiment.

[0065] Figure 6 A triangular mesh model generated from a point cloud in the embodiment. DETAILED DESCRIPTION

[0066] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0067] All the terms (including technical and scientific terms) used in the present application have the meanings generally understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted to have meanings consistent with the context of the present specification, and should not be interpreted in an idealized or overly formal manner.

[0068] For example, the terms "first", "second", etc. used in the present application are only used to distinguish similar objects, to distinguish the first object from another object, and are not used to describe a specific order or sequence, nor can be understood to indicate or imply relative importance.

[0069] Embodiment 1:

[0070] The present embodiment proposes a three-dimensional modeling method, which combines Figure 1 As shown in the figure, data is acquired based on a laser scanner TLS, including:

[0071] Step S1, acquiring a building information model BIM of a target infrastructure, triangulating and sampling the geometric entities of the BIM to generate a point cloud scene for visibility evaluation;

[0072] The data acquired by the laser scanner TLS includes at least: three-dimensional point cloud coordinates (X, Y, Z) obtained by range finding and angle solving, echo intensity and / or reflectivity, time stamp, and echo sequence number / scanning channel identifier; and sensor extrinsic pose information related to the current station. When the TLS is integrated with an imaging unit, RGB texture information for point cloud registration is also acquired.

[0073] The pose information is obtained by at least one of a target / total station, an IMU / encoder, or SLAM alignment; and the scan parameters include at least one of angular resolution, vertical / horizontal step, laser power and / or exposure, and number of scan circles.

[0074] In this embodiment, the building information model can be a structural model, a mechanical and electrical model, or a comprehensive model in the design stage or the operation and maintenance stage. Preferably, a version containing the geometric and positional relationship of components is selected to ensure that no large area of voids appears after triangulation. When triangulating, a sampling step corresponding to the angular resolution of the laser scanner at the nominal measurement distance is used to uniformly sample the surface, so that the generated point cloud scene has a density not lower than the actual operation expectation value. In typical cases, 5 cm to 10 cm can be taken for outdoor bridges and the outer facade of a venue, and 2 cm to 5 cm can be taken for tunnels and indoors. When sampling, the facets with inconsistent normal vectors are unified at one time to prevent misjudgment of the subsequent gating based on the incident angle. When there are non-load-bearing components or components used only for functional expression in the BIM, the components of this type can be optionally marked as invisible or not participating in sampling before generating the point cloud scene, so as to reduce the number of invalid rays. If some components are missing or their positions are changed by later maintenance in the BIM, the existing point cloud blocks in the field are used to locally supplement the points in the region, and the continuity of the measured surface is preferentially ensured. The missing part is treated as an occlusion in the subsequent visibility calculation.

[0075] In step S2, a set of candidate scanning station positions is generated in the operation space, and the visibility relationship between the candidate station positions and the scene sampling points is calculated based on ray casting under the constraints of the sensor (field of view, maximum range, and incident angle threshold), to obtain an environment state representation.

[0076] Specifically, the operation space can be determined by expanding the axis network range or the bounding box of the building information model outward by a preset safety distance. The commonly used expansion amount is 0.5 m to 2 m, which is used for tripod placement, mobile device passage, and safety buffer reserved positions. The sensor constraints are determined according to the specifications of the laser scanner used. The field of view half angle is generally taken as the announced value of the device or slightly smaller than the announced value to leave out mechanical installation errors. The maximum range is taken as the stable working distance of the device under the selected reflectivity. The minimum incident angle threshold is selected according to the diffuse reflection ability and surface roughness of the measured components. The threshold is appropriately increased for metal or polished components to suppress low-quality points. When ray casting, the same coordinate system and unit as the point cloud scene are used to keep the coordinate origin consistent with the BIM coordinate, so as to avoid coordinate drift when re-estimating the visibility multiple times. When there are temporary facilities in the operation space but not modeled in the BIM, the region can be optionally marked as a permanent occlusion region in the environment state representation, and the subordinate sampling points in the region do not participate in the coverage statistics in this task.

[0077] Step S3, output a first scanning action based on the environment state representation using a policy network of a pre-trained deep reinforcement learning agent, the deep reinforcement learning agent comprising the policy network and being configured to output a next station position and at least one scanning parameter;

[0078] Step S4, perform a scanning operation, and register and fuse the newly acquired point cloud with the existing point cloud to form an updated point cloud;

[0079] Raw data acquired by the TLS is converted into a time-stamped point cloud frame via a device driver or SDK, and point attributes include (but are not limited to) three-dimensional coordinates, echo intensity / reflectivity, echo number / channel identifier; when configured as an imaging unit, it also includes aligned RGB values. The system synchronously records the station position pose for registration and fusion. The noise standard deviation can be fitted according to the device nominal or field calibration, and used for the calculation of the quality score.

[0080] Step S5, re-estimate the visibility and coverage according to the updated point cloud, construct a real-time point cloud density map and / or semantic feature map as an agent input, and obtain a subsequent scanning action;

[0081] Step S6, output a final three-dimensional model when a termination condition (coverage, point cloud quality, or job duration threshold) is met, otherwise return to step S4 for loop execution;

[0082] In one embodiment, step S1 comprises:

[0083] Extracting the component geometry in the BIM;

[0084] Triangulating the component and uniformly sampling the surface at a set step;

[0085] Removing internal points to simulate a scannable surface by boundary extraction, which includes local principal direction estimation and micro-section segmentation based on principal component analysis, and plane fitting and boundary point extraction by projecting points based on random sample consistency;

[0086] In one embodiment, the environment state representation of step S2 is encoded as a graph structure:

[0087] The node represents a candidate station position and contains spatial coordinates and attitude information, and the edge represents the visibility relationship between station positions or between station positions and scene regions, and is accompanied by a visibility score, which is used to represent the coverage and occlusion degree under sensor constraints;

[0088] For example, the set of scene sampling points can be regarded as the set of all sampled face points after triangulation, and can be divided into several subsets for piecewise calculation according to components, floors or sections in implementation; the candidate scanning station is expressed by a three-dimensional position and an attitude angle in the same coordinate system, and when the gating result of the station and all sampling points is not passed, the station can be marked as an inefficient station in the current iteration and removed from the graph structure to reduce the state dimension of subsequent reinforcement learning; the occlusion judgment adopts the same spatial resolution and step length as the ray projection, and takes the first collision as the basis for the existence of occlusion, and when there is a missed detection caused by floating point error or sparse point cloud scene, it can be treated as not occluded, but an occlusion penalty factor is applied in subsequent station-level synthesis; for key components that need to ensure measurement accuracy, the sampling points can be weighted before visibility score calculation to ensure that they are covered by more stations in the graph structure.

[0089] The definition step of the visibility score is as follows:

[0090] S21, ray projection is performed on the candidate station and the scene sampling points to obtain the gating results of field of view, ranging, incident angle, occlusion and minimum overlap, which are used as Boolean masks for subsequent scoring;

[0091] S22, when the ray meets the gating, the point-level score is obtained by weighting the geometric quality and information gain:

[0092] ,

[0093] wherein, represents the point-level visibility score of the observation point of the station , is the index of the candidate station, is the index of the scene sampling point, is the minimum overlap gate, and the superscript represents overlap, is the expected overlap of the station with the point cloud, is the minimum overlap threshold, is the occlusion gate, and the superscript represents occlusion, is the field-of-view gate, and the superscript represents field-of-view, is the angle between the optical axis and the connecting line , is the field-of-view half-angle threshold, is the ranging gate, and the superscript represents range, is the measured distance, for maximum range, for incidence angle gate, superscript denotes incidence angle, for incidence angle, for minimum incidence angle threshold, for information gain trade-off coefficient, for geometric quality term, for normalized information gain, for indicator function, takes if condition holds, otherwise takes ;

[0094] wherein the geometric quality is defined as:

[0095] ,

[0096] wherein, is the distance decay scale, is the incidence angle sensitivity index;

[0097] S23, to combine the coverage gain and the occlusion penalty into a station scalar with a weighted sum, for node weight or edge weight of the graph structure:

[0098] ,

[0099] wherein, is the scalar index of the station, is the coverage and occlusion weight and , , is the total number of sampling points, is the set of sampling points, is the measurable subset of the station after gating, is the cardinality thereof;

[0100] ​Similarly, the coverage weight and the occlusion weight can be assigned according to the task purpose, when the complete modeling is given priority, the coverage weight can be taken as 0.6 to 0.8, and the occlusion weight can be taken as 0.4 to 0.2, when the fast inspection is given priority, the coverage weight can be reduced to about 0.5 to obtain a faster scanning path; the information gain weighting coefficient should be selected between 0.3 to 0.7, when the field is expected to add limited information and more is to fill holes or enhance overlap, the coefficient can be taken as low to rely more on the geometric quality item; the distance attenuation scale should not be less than the typical distance from the actual average station to the measured surface, for example, the tunnel scene can be taken as 3m to 5m, the venue or the outside of the bridge can be taken as 8m to 15m, and the incidence angle sensitivity index between 0 to 3 can be smoothly adjusted to meet the needs of most materials; the minimum overlap threshold, the field of view half angle threshold, the maximum ranging, and the minimum incidence angle threshold can be set according to the semantic partition, for example, the more stringent overlap and incidence angle requirements are taken for high value areas such as tunnel vaults and bridge joints, and the more relaxed requirements are taken for unobstructed flat walls; when the number of effective measurable subsets of a station is less than 5% of the total number of sampling points, the station can be marked as a non-optimal station and its selection probability can be reduced in subsequent optimization.

[0101] The minimum overlap threshold, the field of view half angle threshold, the maximum ranging, and the minimum incidence angle threshold can be set according to the semantic partition, The entropy reduction of occupancy or semantic uncertainty can be obtained by normalization;

[0102] Specifically, a continuous link from ray-level gating to point-level scoring, and then to station-level synthesis is defined here; the point-level score is expressed as the convex combination of geometric quality and information gain, which is the feasible observation after gating product clipping, so as to facilitate the physical parameterization of distance and angle sensitivity; the station-level scalar puts the coverage benefit and the occlusion rate penalty on the same scale, and cooperatively measures the global coverage and the local occlusion of the gated subset, so that the station with high coverage and low occlusion is superior in the ranking; the weights are in the form of one, which is beneficial to smooth switching between different operation targets;

[0103] In one embodiment, the candidate station positions are generated based on job area gridding and accessibility constraints, the accessibility constraints including a device safety buffer distance, ground flatness, and a no-entry area shield; and a key structure area (such as a bridge pier, an arch rib, a tunnel lining, and a node connection) near a semantic priority is densely sampled; optionally, the semantic priority can be directly derived from a component category field of a building information model, or can be obtained by an existing point cloud semantic segmentation model on a preliminary coarse scanning point cloud, and when the two do not conflict, the one with a higher priority is used as a criterion; the dense sampling can shorten the grid length to one half or one third of the original grid length based on the original gridding, so that the candidate station positions around the key structure are more densely distributed, thereby providing more optional actions for these areas in subsequent reinforcement learning; if there are temporary stands, isolation fences, or construction machinery inside the venue, causing some semantic important areas to be inaccessible for a period of time, the semantic label is maintained in the candidate station positions in the area, but the accessibility is marked as inaccessible, and when the state is updated to reflect that the area is empty, the part of the station positions can be automatically restored; when the input BIM model lacks semantic categories, the geometric priority scheme can be degraded according to the height, normal, or spatial neighborhood.

[0104] Meanwhile, the deep reinforcement learning agent is pre-trained via a synthetic point cloud scene, and a multi-objective reward function is used to optimize the agent behavior, the reward function comprehensively considering coverage gain, point cloud quality, and scanning time / energy consumption, and the migration to a real scene is realized through domain randomization and a small amount of real data fine-tuning;

[0105] Further, the synthetic point cloud scene can be randomly disturbed in noise level, occlusion body density, reflectivity distribution, and candidate station position number when constructed, so that the policy network can see multiple task scales and multiple occlusion modes in the training stage, and the typical number of training rounds can be set to several thousand to tens of thousands of rounds, and the maximum decision step number of each round corresponds to the actual task operation time; in the migration stage, a small amount of on-site collected point clouds can be used to fine-tune the strategy, and the on-site data can only cover part of the area, and the reward function structure is kept unchanged, and only the coverage weight and the time penalty weight are adjusted to realize rapid adaptation; when fine-tuning finds that the registration overlap degree of the on-site point cloud is lower than a preset minimum overlap threshold for a long time, the termination term of this round can be set to a negative value to promote the strategy to generate a more overlapping station position combination; if the training resources are limited, the policy network can also be pre-trained only on the synthetic scene, and a larger time penalty can be used in online deployment to suppress unnecessary exploration.

[0106] The multi-objective reward function is set as follows:

[0107] S31, in each decision step The coverage gain, point cloud quality, and time / energy consumption, and constraint penalty are combined into a scalar reward:

[0108] ,

[0109] where, is the reward of step , and are the weights of coverage, quality, time / energy and penalty, respectively, is the coverage gain score, is the point cloud quality score, is the time / energy cost score (the larger the cost, the larger the score), is the constraint violation score, is the termination term, taking positive value when the task is terminated and the threshold is reached, otherwise taking negative value when the task is terminated due to time / energy consumption, the numerical values are determined by the task policy table;

[0110] S32, the set of new visible and gated sampling points measure the coverage gain and keep consistency with information gain:

[0111] ,

[0112] where, is the coverage score of step , is the set of new gated sampling points of step , is the cardinality of it, is the selected station index of step , is the trade-off coefficient between geometric quality and information gain, is the geometric quality term of point at station , is the normalized information gain; filtered by field of view, ranging, incident angle and occlusion gating;

[0113] S33, integrate density, incident angle, ranging noise and registration overlap into normalized quality score:

[0114] ,

[0115] where, , ,

[0116] where, is the quality score, and the sum of the four is , are the four weights, is the density score, denotes the actual number of samples in the current statistical unit (surface patch / voxel) at step t, denotes the task nominal density, is the fraction of incidence angle, taking the incidence angle of the return point, is the average, , is the fraction of ranging noise, denotes the step-wise average ranging standard deviation based on the noise model, is the reference standard deviation, is the noise prediction when the ranging is , is the fraction of registration overlap with the current point cloud, based on the overlap ratio of corresponding voxel / feature pairs;

[0117] S34, the step duration and energy consumption constitute a weighted cost fraction:

[0118] ,

[0119] where, is the cost fraction, and are the time and energy consumption weights, is the time used by the step , is the reference duration, is the energy consumption of the step , is the reference energy consumption; when the ratio of the two is greater than , it can be truncated to ;

[0120] S35, the hinge function is used to aggregate the positive violation of each type of constraint:

[0121] ,

[0122] where, is the penalty fraction, is the constraint set (including field of view, maximum ranging, minimum incidence angle, minimum overlap, safety buffer, forbidden area, etc.), is the weight of the constraint , is the state-action pair of the step , is the signed violation of the constraint (e.g. , denotes the minimum incidence angle actually observed by the selected station at step t), is the positive part operator; the termination item uses an indicative threshold strategy: when the cumulative coverage or quality reaches the threshold or a positive reward is given when the accumulated length or energy consumption reaches an upper limit or a negative reward is given when the passive termination is reached, wherein the four termination thresholds are all positive numbers preset;

[0123] In this embodiment, the coverage termination threshold can be set to between 85% and 95% according to the task accuracy requirement, the quality termination threshold can be selected between 0.75 and 0.9, the time termination threshold can be estimated according to the acceptable working time on site, for example, the routine bridge inspection can be set to 30min to 60min, the continuous section of the tunnel can be set to 15min to 30min, and the energy consumption termination threshold can be valued at 70% to 90% of the nominal single battery endurance of the device to leave a safety margin; when any of the termination thresholds is reached in advance, the current round is ended with a positive termination item in this embodiment, and when the time or energy consumption reaches the upper limit, it is ended with a negative termination item; if the scanning action cannot be continued due to site congestion, communication abnormality or point cloud registration failure in a certain round, the current covered range and the time length consumed are used as the basis for conservative termination of the round, and the next round is restored to the latest successful state cache.

[0124] Specifically, the reward design connects the coverage, quality and cost three types of targets with a unified scale that can be weighted, facilitating continuous adjustment between different task preferences; the coverage score directly counts the newly visible points, and is coupled with the information gain and the geometric quality as mentioned above, taking into account the area seen and the value seen; the quality score expresses the comprehensive state of sampling density, incident angle, ranging noise and registration overlap with a four-factor structure, and obtains clear adjustment feel through interpretable weights; time and energy consumption are combined in a normalized linear combination, which can reflect the moving and scanning timing and also can depict the device power consumption; the constraint penalty uses a hinge aggregation, generating a continuous punishment signal for the out-of-bound amplitude, which is beneficial to the formation of robust behavior of the strategy near the boundary; the termination item takes the coverage / quality standard and the time / energy consumption upper limit as the criterion, making the learning goal consistent with the task goal;

[0125] Furthermore, the agent's observation inputs include real-time point cloud density maps and / or semantic feature maps. The density map is generated by projecting the workspace at a fixed resolution or by statistically counting the number of voxels, while the semantic feature map is generated by a point cloud segmentation model. The policy network employs convolutional neural networks and / or graph neural networks to process the above observations and output the next station location and at least one scanning parameter. For example, the real-time point cloud density map can be projected onto a horizontal or vertical plane, and the resolution should be matched to the actual station spacing and the size of the measured component, generally set to 0.1m to 0.25m per grid. The semantic feature map can undergo channel pruning before input, retaining only those relevant to the scanning decision. The component categories are optimized to reduce the computational load of the policy network. The convolutional network is used to extract local spatial patterns, and the graph network is used to propagate visibility and occlusion information between candidate stations. The outputs of both are concatenated at the decision layer to form the final action probability or action vector. When deployed on edge terminals or robot platforms with limited computing resources, only the convolutional branches can be retained and compensated with a fixed candidate station ranking table to ensure that the single-step decision time is controlled within the order of hundreds of milliseconds. If the real-time point cloud density map has short-term holes due to sensor jitter or communication delay, the most recent effective density map is used as input, and the time penalty is reduced in the reward function to avoid policy misjudgment.

[0126] After each scan step, the visibility relationship is recalculated based on the updated point cloud, and an online optimization algorithm is used to correct the unexecuted scan sequence to minimize the coverage blind zone and operation time. The online optimization algorithm can be any one of the greedy algorithm, genetic algorithm or local search algorithm.

[0127] Similarly, in actual operation, different online optimization algorithms can be selected according to the task scale. When the number of candidate stations is within dozens, a greedy algorithm can be used to obtain good coverage improvement. When the number of candidate stations reaches hundreds or even thousands, a genetic algorithm or a local search algorithm can be used to avoid getting trapped in local optima. The online optimization uses the proposed sequence of the current reinforcement learning agent as the initial solution and only adjusts the unexecuted parts to ensure that the executed parts are not destroyed. When the online optimization fails to find a better solution within the limited computation time, the system maintains the original scanning sequence and continues to execute to ensure that the job is not interrupted. If a station is detected as unreachable or blocked by on-site safety rules before execution, the station is removed from the current sequence during the online optimization stage and reordered according to the visibility score of the remaining stations.

[0128] Within the same operation time, the solution of this invention improves the newly added visible coverage and effective overlap, and reduces registration error and occlusion blind spots; see also Figures 3-6 : Figure 3 The point cloud quality distribution is shown, tinted by echo intensity / reflectivity. Figure 4 This demonstrates the effect of multi-site point cloud registration from misalignment to alignment.Figure 5 to generate a real-time point cloud density map, Figure 6 to illustrate the comprehensive benefits of the method in coverage, quality and efficiency.

[0129] Embodiment 2:

[0130] This embodiment proposes a three-dimensional modeling system, which can implement the three-dimensional modeling method described in embodiment 1, combined with Figure 2 As shown in the figure, the system comprises:

[0131] A BIM processing module for converting BIM into a point cloud scene for visibility evaluation;

[0132] A state construction module for generating candidate stations and calculating visibility relationships to form an environmental state representation;

[0133] A decision agent module containing a policy network submodule and a reward calculation submodule for outputting scanning decisions according to real-time observations;

[0134] A scanning control module for executing scanning, completing point cloud registration and fusion, and triggering online sequence correction;

[0135] Wherein, the scanning control module can be implemented on an industrial computer, a rugged workstation or an embedded platform with a real-time operating system, and it transmits scanning instructions and returns point cloud data to the laser scanner through wired or wireless data link. When transmission fails, it automatically retries no less than three times and records the current station as an abnormal station. Point cloud registration prefers to use the registration method consistent with the current point cloud overlap, and after successful registration, the updated point cloud is written into the environmental state cache, so that the latest point cloud can be directly used for subsequent visibility reevaluation. When the scanning control module detects that the number or quality of point clouds collected at the current station is lower than the minimum threshold set by the task, it can trigger a temporary rescan or request the decision agent to select a new adjacent station, thereby ensuring the integrity of the final model. After online sequence correction is triggered, the scanning control module issues control instructions according to the new station order, while retaining the previous sequence as a fallback solution to deal with network interruptions or unexpected events.

[0136] This embodiment also provides a computer device comprising a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to implement the three-dimensional modeling method proposed in the above embodiments.

[0137] The computer device can be a terminal, which includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved by WIFI, an operator network, NFC (Near Field Communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0138] The embodiment also provides a storage medium having a computer program stored thereon, the program being executed by a processor to implement the three-dimensional modeling method proposed in the above embodiment. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk or an optical disk.

[0139] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

[0140] Furthermore, to the extent that the terms "comprises", "comprising", "includes", "including" and "has" or any variation thereof are used in the specification and / or claims, these terms are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises, includes or has a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Additionally, to the extent that the terms "coupled" and "current-coupled" are used in the detailed description and / or claims, such terms are intended to mean that the devices are in electronic communication and / or signal communication, unless otherwise indicated. Also, where necessary, the use of the terms "a", "an" and "the" are intended to mean "one or more", unless otherwise indicated.

Claims

1. A three-dimensional modeling method based on data acquisition using a laser scanner TLS, characterized in that, include: Step S1: Obtain the Building Information Model (BIM) of the target infrastructure, triangulate and sample the geometric entities of the BIM, and generate a point cloud scene for visibility assessment. Step S2: Generate a set of candidate scanning stations in the work space, and under sensor constraints, calculate the visibility relationship between the candidate stations and scene sampling points based on ray projection to obtain an environmental state representation; Step S3: Based on the environmental state representation, the first scanning action is output using the policy network of the pre-trained deep reinforcement learning agent, wherein the deep reinforcement learning agent includes a policy network and is configured to output the next station and at least one scanning parameter. Step S4: Perform a scanning operation to register and fuse the newly acquired point cloud with the existing point cloud to form an updated point cloud; Step S5: Based on the updated point cloud, re-evaluate the visibility and coverage, construct a real-time point cloud density map and / or semantic feature map, and use them as proxy input to obtain subsequent scanning actions; Step S6: Output the final 3D model when the termination condition is met; otherwise, return to step S4 and repeat the process.

2. The three-dimensional modeling method as described in claim 1, characterized in that, Step S1 includes: Extract component geometry from BIM; The components are triangulated and their surfaces are uniformly sampled according to a set step size; The boundary extraction process removes internal points to simulate a scannable surface. This boundary extraction includes local principal direction estimation and micro-segmentation based on principal component analysis, as well as plane fitting based on random sample consistency and point projection to extract boundary points.

3. The three-dimensional modeling method as described in claim 1, characterized in that, The environment state representation in step S2 is encoded as a graph structure: Nodes represent candidate stations and include spatial coordinates and attitude information. Edges represent the visibility relationships between stations or between a station and a scene area, and are accompanied by visibility scores. The visibility scores are used to characterize the coverage and occlusion degree under sensor constraints. The steps for defining the visibility score are as follows: S21, in the candidate station With scene sampling points Ray projection is performed on the surface to obtain gating results for field of view, range, incident angle, occlusion and minimum overlap, which are used as Boolean masks for subsequent scoring; S22, when the ray satisfies the gating condition, the geometric quality and information gain are weighted to obtain the point-level fraction: ， in, Indicates position Observation point Point-level visibility score, Subscript for candidate station locations As the index of the scene sampling point, For minimum overlap gate, superscript Indicates overlap. For the position The degree of overlap with the expected amount of existing point clouds, The minimum overlap threshold, To cover the door, the superscript Indicates occlusion. For field of view gate, superscript Represents field-of-view, For optical axis and connection line The included angle, The half-angle threshold of the field of view. For distance measuring gates, superscript Indicates range, To measure distance, For maximum distance measurement, For the angle of incidence goal, superscript Indicates the incidence angle. Angle of incidence The minimum incident angle threshold, This is the information gain trade-off coefficient. For geometric mass, To normalize information gain, For indicator functions, take the condition if it is true. Otherwise take .

4. The three-dimensional modeling method as described in claim 3, characterized in that, In the step of defining the visibility score, the geometric quality is defined as follows: ， in, For distance attenuation scale, The incident angle sensitivity index; The steps for defining visibility scores also include: S23, using a weighted sum to synthesize the coverage gain and occlusion penalty into a position scalar, is used for node weights or edge weights in graph structures: ， in, For the position scalar indicators For coverage and occlusion weights and , This represents the total number of sampling points. For the set of sampling points, For the position pass Gated measurable subset Its base.

5. The three-dimensional modeling method as described in claim 1, characterized in that, Candidate stations are generated based on the gridding of the work area and accessibility constraints, including equipment safety buffer distance, ground flatness, and restricted area shielding; and sampling is performed near key structural areas based on semantic priority.

6. The three-dimensional modeling method as described in claim 1, characterized in that, The deep reinforcement learning agent is pre-trained using a synthetic point cloud scene and its behavior is optimized using a multi-objective reward function. The reward function comprehensively considers coverage gain, point cloud quality, and scanning time / energy consumption, and achieves transfer to the real scene through domain randomization and fine-tuning with a small amount of real data. The multi-objective reward function is defined as follows: S31, at each decision step The coverage gain, point cloud quality, time / energy consumption, and constraint penalties are combined into a scalar reward: ， in, For steps The reward and These are the weights for coverage, quality, time / energy consumption, and penalty, respectively. For coverage gain fraction, Point cloud quality score, The time / energy cost score. To restrict the points awarded for violations, This is a termination condition; a positive value is taken when the task is terminated and the task threshold is met; otherwise, a negative value is taken when the task terminates due to timeout or excessive energy consumption. The value is determined by the task strategy table; S32, with a new set of visible and gated sampling points. Measure coverage gains and maintain consistency with information gain: ， in, For steps Coverage score, For steps Add a new set of sampling points that pass the gate. Its base, For steps Selected station index This is the trade-off coefficient between geometric quality and information gain. For point In position The geometric mass term, Normalized information gain; It is obtained by screening based on field of view, range, angle of incidence, and gating of obstruction.

7. The three-dimensional modeling method as described in claim 6, characterized in that, The steps for setting the multi-objective reward function also include: S33 combines density, incident angle, ranging noise, and registration overlap into a normalized mass fraction: ， in, , , in, For quality fraction, And the four together are There are four weights. Density fraction, This represents the actual number of sampling points in the current statistical unit within step t. Indicates the nominal density of the task. For the fraction of the angle of incidence, take the fraction of the angle of incidence relative to the return point. average, , The distance measurement noise fraction, This represents the standard deviation of the average ranging within a step based on the noise model. For reference standard deviation, For distance measurement Noise prediction at that time The registration overlap score with the existing point cloud is based on the overlap ratio of the corresponding voxel / feature pair; S34, with step duration and energy consumption forming a weighted cost score: ， in, For the sake of the score, and Weighted by time and energy consumption, For steps The time required For reference duration, For steps Energy consumption, For reference energy consumption; the ratio of the two items is greater than Cut off at time ; S35 aggregates positive and negative constraints of various types using hinge functions: ， in, As a penalty for low scores, For a constraint set, To constrain The weight, For steps State-action pair To constrain Signed violation, Positive part operator; Termination term Employ an indicative threshold strategy: when cumulative coverage or quality reaches a threshold... or Positive rewards are given when the cumulative duration or energy consumption reaches the limit. or A negative reward is given when the event is passively terminated, and all four termination thresholds mentioned above are preset positive numbers.

8. The three-dimensional modeling method as described in claim 1, characterized in that, The agent's observation inputs include a real-time point cloud density map and / or a semantic feature map. The density map is generated by projecting or voxelizing the workspace at a fixed resolution to count the number of coverages, and the semantic feature map is generated by a point cloud segmentation model. The policy network uses a convolutional neural network and / or a graph neural network to process the above observations and output the next station and at least one scanning parameter.

9. The three-dimensional modeling method as described in claim 1, characterized in that, After each scan step, the visibility relationship is recalculated based on the updated point cloud, and an online optimization algorithm is used to correct the unexecuted scan sequence to minimize the coverage blind zone and operation time. The online optimization algorithm can be any one of a greedy algorithm, a genetic algorithm, or a local search algorithm.

10. A three-dimensional modeling system, based on the three-dimensional modeling method according to any one of claims 1 to 9, characterized in that, include: The BIM processing module is used to convert BIM into point cloud scenes for visibility assessment. The state construction module is used to generate candidate stations and calculate visibility relationships to form an environmental state representation. The decision agent module, which includes a policy network submodule and a reward calculation submodule, is used to output scanning decisions based on real-time observations. The scanning control module is used to perform scanning, complete point cloud registration and fusion, and trigger online sequence correction.

Images