Multi-source data acquisition device and method for three-dimensional modeling of ancient buildings

Abstract

The application discloses a multi-source data acquisition device and method for three-dimensional modeling of ancient buildings, and belongs to the field of three-dimensional space measurement. The device comprises the following steps: obtaining a wireless ranging signal by a mobile master and a positioning beacon to establish a relative coordinate system; obtaining rough point cloud data of the ancient building in the relative coordinate system, constructing a three-dimensional occupancy grid model, and calculating the uncertainty value of each grid; calculating the best observation pose based on the obstacle distribution, solving the generated mechanical arm deformation parameters and obstacle avoidance movement path; controlling the end of the mechanical arm to enter the complex gap area to collect high-definition images and depth data; and fusing the high-definition images and depth data to the three-dimensional occupancy grid model, and iteratively circulating until the requirements are met. The application can realize autonomous, complete and high-fidelity measurement of the complex profile of the ancient building by adaptively planning an exploration path to penetrate the complex structure.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional spatial measurement, and in particular to a multi-source data acquisition device and method for three-dimensional modeling of ancient buildings. Background Technology

[0002] Ancient buildings are important carriers of cultural heritage, carrying rich historical information and artistic value. In order to achieve digital archiving, restoration design and long-term monitoring of ancient buildings, multi-source data acquisition technologies such as 3D laser scanning and close-range photogrammetry are usually used to obtain the geometric structure and texture information of ancient buildings, and to construct high-precision real-scene 3D models.

[0003] In related technologies, Chinese invention patent application CN119068096A discloses a method for 3D digitization of ancient buildings by fusing multi-source surveying and mapping geographic information. This method uses a set of acquisition parameters and a set of reference occlusion areas to confirm that accurate image acquisition of ancient buildings can be performed. Based on the set of acquisition parameters and the reference coordinate system, the acquisition path and coordinates for acquiring images of ancient buildings are fitted. The target ancient building is photographed using an image acquisition device to obtain an image set. After confirming that the image set is a preset target image set, a first 3D point cloud map is generated using the target image set to characterize the outdoor features of the ancient building. A second 3D point cloud map is obtained to characterize the indoor features of the ancient building. The first and second 3D point cloud maps are fused to obtain an optimized 3D point cloud map. A model of the target ancient building is constructed based on the optimized 3D point cloud map.

[0004] Regarding the aforementioned technologies, data acquisition primarily relies on external aerial photography by aircraft and internal ground station scanning. The fusion of multi-source data focuses on stitching together macroscopic indoor and outdoor scenes. However, ancient buildings often have complex geometric structures, such as layered eaves, deep gaps in brackets, and narrow structural seams. Aircraft are limited by safe flight distances and cannot approach or enter such unstructured, narrow areas. Ground scanning equipment is also prone to blind spots due to line-of-sight obstruction. This results in incomplete acquisition of high-precision texture and geometric data for the complex and hidden parts of ancient buildings, making it difficult to guarantee precision and completeness. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides a multi-source data acquisition device and method for 3D modeling of ancient buildings. By adaptively planning and exploring paths to perform penetrating measurements on complex structures, it is possible to achieve autonomous, complete, and high-fidelity measurements of the complex outlines of ancient buildings.

[0006] The above objectives can be achieved through the following approach: A multi-source data acquisition device for 3D modeling of ancient buildings includes an environment initialization module for controlling a mobile mother machine to deploy ultra-wideband positioning beacons in the area of ​​the ancient building, acquiring wireless ranging signals through the mobile mother machine and the ultra-wideband positioning beacons, and establishing a relative coordinate system; a global cognition module for acquiring coarse point cloud data of the ancient building in the relative coordinate system, constructing a 3D occupancy grid model, and calculating uncertainty values ​​for each grid; a coupling decision module for extracting grids with uncertainty values ​​higher than a preset threshold as observation grids, calculating the optimal observation pose for the observation grids using obstacle distribution, and solving to generate deformation parameters and obstacle avoidance movement paths for the robotic arm; an adaptive execution module for driving the acquisition device along the obstacle avoidance movement path, controlling the robotic arm to execute the deformation parameters, allowing the end effector of the robotic arm to enter complex gap areas, and acquiring high-definition images and depth data; and an incremental closed-loop module for fusing the high-definition images and depth data into the 3D occupancy grid model, updating the uncertainty values, and repeatedly executing viewpoint calculation and acquisition until the uncertainty values ​​of all grids meet the requirements.

[0007] Optionally, the environment initialization module includes: a beacon deployment interaction unit, used to drive the mobile mothership to release ultra-wideband positioning beacons at intervals, and to activate two-way time-of-flight ranging interaction between the mobile mothership and the ultra-wideband positioning beacons; a distance matrix construction unit, used to obtain wireless ranging signals through the mobile mothership and the ultra-wideband positioning beacons to extract timestamp differences and calculate point-to-point Euclidean distances, and construct a distance observation matrix; and a coordinate system calculation unit, used to perform least-squares adjustment calculations on the distance observation matrix with the ultra-wideband positioning beacon as the origin and the ultra-wideband positioning beacon as the coordinate axis, and output a relative coordinate system.

[0008] Optionally, the global cognition module includes: a data mapping unit, used to activate the wide-angle scanning sensor of the mobile mother machine to perform panoramic data capture, and use the relative coordinate system to perform coordinate transformation to generate coarse point cloud data; a grid discretization unit, used to perform voxel segmentation based on the coarse point cloud data with a fixed step size, establish a mapping relationship between point cloud index and voxel address, and construct a three-dimensional occupied grid model; and a feature quantization calculation unit, used to perform least squares plane fitting based on the coarse point cloud data and the three-dimensional occupied grid model, calculate the root mean square of the fitting residual, construct an inverse proportional function relationship with the point cloud density, and solve to output the uncertainty value.

[0009] Optionally, the solution to output an uncertainty value includes: extracting a subset of the point cloud of the current grid based on the three-dimensional occupied grid model, counting the number of point clouds to obtain the point cloud density, and calculating the vertical distance from the subset of point clouds to the fitting plane to generate the root mean square of the fitting residual; using the root mean square of the fitting residual as the numerator data and the point cloud density as the denominator data, performing a weighted division to output an uncertainty value.

[0010] Optionally, the coupling decision module includes: a target indexing unit, used to perform a numerical comparison between the uncertainty value and the preset threshold, and if the conditions are met, add the grid coordinates to the observation queue; a line-of-sight analysis unit, used to perform ray projection detection from the center of the grid to be observed outward, identify the ray direction not blocked by the distribution of obstacles, and determine the optimal observation pose in combination with the sensor field of view; and a motion calculation unit, used to input the optimal observation pose to the robotic arm for inverse kinematics solution, calculate joint angles to generate deformation parameters, and construct an obstacle avoidance movement path connected to the optimal observation pose based on the current position of the mobile machine.

[0011] Optionally, the device further includes: extracting the uncertainty value as a gain index, extracting the length of the obstacle avoidance path as a cost index, performing a ratio calculation between the gain index and the cost index, and generating a collection priority coefficient.

[0012] Optionally, the adaptive execution module includes: a cooperative motion execution unit, used to synchronously execute the navigation command of the obstacle avoidance movement path and the joint action command of the deformation parameters according to the acquisition priority coefficient, so as to deliver the end effector of the robotic arm to the complex gap area; and a state lock triggering unit, used to monitor the end effector of the robotic arm to stabilize in order to lock the joint of the robotic arm, and synchronously trigger the image sensor and depth sensor to capture high-definition images and depth data.

[0013] Optionally, the incremental closed-loop module includes: a mesh state repair unit, used to fill empty data voxels in the three-dimensional occupied mesh model with the depth data, and to update the surface visual texture of the empty data voxels with the high-resolution image; and a cyclic decision triggering unit, used to search for residual meshes that exceed the limit in the uncertainty value in real time, and if there are residual meshes, to send a replanning instruction to the coupled decision module, otherwise to lock the three-dimensional occupied mesh model.

[0014] Based on the same inventive concept, this invention also provides a multi-source data acquisition method for 3D modeling of ancient buildings. The method includes: controlling a mobile mother machine to deploy ultra-wideband positioning beacons in the area of ​​the ancient building; acquiring wireless ranging signals through the mobile mother machine and the positioning beacons to establish a relative coordinate system; acquiring coarse point cloud data of the ancient building in the relative coordinate system, constructing a 3D occupancy grid model, and calculating uncertainty values ​​for each grid; extracting grids with uncertainty values ​​higher than a preset threshold as observation grids; calculating the optimal observation pose for the observation grids based on obstacle distribution, and calculating and generating deformation parameters and obstacle avoidance movement paths for a robotic arm; driving the acquisition device along the obstacle avoidance movement path, controlling the robotic arm to execute the deformation parameters, causing the end of the robotic arm to enter a complex gap area, and acquiring high-definition images and depth data; fusing the high-definition images and the depth data into the 3D occupancy grid model, updating the uncertainty values, and repeating viewpoint calculation and acquisition until the uncertainty values ​​of all grids meet the requirements.

[0015] Compared with the prior art, the present invention has the following advantages: 1. By constructing a dynamic cognitive scalar field to quantify and drive the acquisition process, a paradigm shift from passive global scanning to proactive intelligent exploration has been achieved; 2. By designing a multi-level collaborative execution architecture that includes a mobile mother machine, a robotic arm, and a detachable micro intelligent acquisition terminal, the limitations of traditional measurement equipment in terms of physical space accessibility are overcome.

[0016] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a framework diagram of a multi-source data acquisition device for three-dimensional modeling of ancient buildings according to an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of the structure of a multi-source data acquisition device for three-dimensional modeling of ancient buildings according to an embodiment of the present invention.

[0020] Figure 3This is a trend analysis graph showing the change of uncertainty value with point cloud density in an embodiment of the present invention.

[0021] Figure 4 This is a scatter plot of the gain cost distribution for the priority evaluation of the acquisition task in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Reference Figure 1 One embodiment of the present invention proposes a multi-source data acquisition device for three-dimensional modeling of ancient buildings. By adaptively planning and exploring paths, it can perform penetrating measurements on complex structures, enabling autonomous, complete and high-fidelity measurements of the complex outlines of ancient buildings.

[0024] like Figure 2 As shown, the apparatus in this embodiment specifically includes: The environment initialization module is used to control the mobile mother unit to deploy ultra-wideband positioning beacons in the ancient building area, and to obtain wireless ranging signals through the mobile mother unit and the ultra-wideband positioning beacons to establish a relative coordinate system. The global cognition module is used to acquire rough point cloud data of ancient buildings in the relative coordinate system, construct a three-dimensional occupancy grid model, and calculate the uncertainty value for each grid. The coupled decision module is used to extract the grids with uncertainty values ​​higher than a preset threshold as the grids to be observed, calculate the optimal observation pose for the grids to be observed using obstacle distribution, and solve to generate the deformation parameters and obstacle avoidance movement path of the robotic arm. An adaptive execution module is used to drive the acquisition device to travel along the obstacle avoidance movement path, control the robotic arm to execute the deformation parameters, and enable the end of the robotic arm to enter a complex gap area to acquire high-definition images and depth data. The incremental closed-loop module is used to fuse the high-definition image and the depth data into the three-dimensional occupied grid model, update the uncertainty value, and repeatedly perform viewpoint calculation and acquisition until the uncertainty value of all grids meets the requirements.

[0025] Optionally, the environment initialization module includes: The beacon deployment interaction unit is used to drive the mobile mother machine to release ultra-wideband positioning beacons at intervals and to activate two-way time-of-flight ranging interaction between the mobile mother machine and the ultra-wideband positioning beacons. The mobile mothership, using onboard mechanical delivery components such as electromagnetic chucks or mechanical grippers, deploys ultra-wideband (UWB) positioning beacons at physical intervals in unstructured areas of ancient buildings, such as courtyard centers and corridor corners, according to the coverage path. The physical interval is determined based on a signal attenuation model and the line-of-sight conditions of the surrounding environment, typically set to 30% to 50% of the effective communication radius to ensure network connectivity. Once deployment is complete, a handshake command is broadcast to all activated UWB beacons, configuring the mobile mothership as the ranging initiator and each UWB beacon as a ranging responder, initiating a two-way time-of-flight ranging protocol. This protocol requires at least one round-trip signal transmission between any two nodes in the network to ensure subsequent acquisition of distance constraint information for full connectivity.

[0026] For example, suppose the ancient building is an enclosed courtyard with four corners. The mobile mother unit moves along the diagonal of the courtyard, deploying a button-sized ultra-wideband beacon every 10 meters, for a total of four beacons, labeled as nodes A, B, C, and D. The mother unit is node M. After deployment, node M first sends a ranging request frame to node A. Node A receives and processes the frame and returns an acknowledgment frame. Subsequently, this interaction is performed sequentially between M and B, C, and D, as well as between A and B, C, and D, ensuring that a full mesh signal link is formed in the network.

[0027] The distance matrix construction unit is used to acquire wireless ranging signals through the mobile mother unit and the ultra-wideband positioning beacon to extract timestamp differences and calculate point-to-point Euclidean distances, thereby constructing a distance observation matrix. By monitoring the wireless ranging signal during the interaction process, a high-precision hardware timestamp is directly read from the register of the ultra-wideband communication chip. To eliminate errors caused by clock asynchrony between nodes, a two-way time-of-flight (TW-ToF) algorithm is used to calculate the point-to-point Euclidean distance. In this algorithm, the physical distance is derived from the product of the signal's propagation time in the air and the speed of light. To quantify this process, the following distance calculation formula is used: ; in, This represents the point-to-point Euclidean distance between any two nodes, and its physical unit is meters. This represents the speed at which electromagnetic waves propagate in air, and is typically taken as a value. meters per second; This represents the total round-trip time of the signal, which is the time difference between when the initiator sends the signal and when the initiator receives the response signal. This value is directly measured by the initiator's local clock counter. This represents the response delay, which is the time difference between when the responder receives the signal and when it sends the response signal. This value is a known inherent hardware processing delay, provided by the factory calibration parameters of the ultra-wideband chip. All calculated distance values ​​are filled into a symmetric two-dimensional array to construct the distance observation matrix. The diagonal elements of this matrix are 0, and the off-diagonal elements are... The measured distance between node i and node j is stored. .

[0028] For example, if the mobile mother machine M initiates ranging towards beacon A, the total round-trip time of the signal is measured. The hardware response delay of beacon A is known to be 67 nanoseconds. The time is 0.3 nanoseconds. Substituting into the formula, the net flight time of the signal in the air is 66.7 nanoseconds. Multiplying by the speed of light and dividing by 2, the physical distance between M and A is calculated to be approximately 10.0 meters.

[0029] The coordinate system calculation unit is used to perform least squares adjustment calculation on the distance observation matrix with the ultra-wideband positioning beacon as the coordinate origin and the ultra-wideband positioning beacon as the coordinate axis, and output the relative coordinate system.

[0030] To address the uncertainties in the translation and rotation of the coordinate system, a geometric constraint method is used to define the reference of the relative coordinate system: the position coordinates of the first deployed ultra-wideband positioning beacon are selected as... The origin of the coordinate system is used; the second beacon is selected to be located on the positive half of the X-axis, and its coordinates are set to... A third beacon is selected and located on the XOY plane. Based on this, a nonlinear least-squares optimization problem is constructed, aiming to find an optimal set of 3D coordinates that minimizes the sum of squared errors between the theoretical distance calculated from these coordinates and the measured distance in the distance observation matrix. This optimization process follows the following objective function: ; in, This represents the global residual sum of squares, used to characterize the degree of fit between the solved coordinates and the measured data; These represent the three-dimensional coordinate components of the i-th node to be solved; these are the variables that need to be updated iteratively. Represents the three-dimensional coordinate components of the j-th node; This represents the measured distance between node i and node j recorded in the distance observation matrix. Iterative solutions are used to solve for... Once the minimum value is reached, the precise three-dimensional coordinates of all beacons and moving mother machines in the relative coordinate system can be output.

[0031] For example, in 5 nodes (M, A, B, C, D), the distance observation matrix provides A distance edge constraint. Initialize the coordinates of point A as follows: The coordinates of point B are Then, the coordinates of C, D, and M are calculated iteratively. After 5 iterations, the objective function... The value converges to Level, at this point the coordinates of point M are This indicates that the mother machine is located in the center of the courtyard at a height of 1.5 meters, thus completing the establishment of the relative coordinate system.

[0032] Optionally, the global cognitive module includes: The data mapping unit is used to activate the wide-angle scanning sensor of the mobile mother machine to perform panoramic data capture and to perform coordinate transformation using the relative coordinate system to generate coarse point cloud data. First, a trigger command is sent to the wide-angle scanning sensor mounted on the mobile mothership to perform a full-field panoramic data capture. The raw data output by the sensor is a set of local point clouds based on the sensor's own coordinate system. The relative coordinate system parameters established by the environment initialization module are read to construct a rigid body transformation matrix. This matrix contains a rotation matrix and a translation vector, corresponding to the rotation angle and displacement distance of the mobile mothership relative to the origin, respectively. For each captured laser point, its local coordinate vector is multiplied by the rotation matrix on the left and the translation vector is added to calculate the absolute position of the point in the relative coordinate system. The set of all transformed points constitutes the coarse point cloud data.

[0033] The grid discretization unit is used to perform voxel segmentation based on the coarse point cloud data with a fixed step size, establish a mapping relationship between point cloud index and voxel address, and construct a three-dimensional occupied grid model. Unstructured coarse point cloud data is converted into a structured 3D occupancy mesh model for spatial indexing and state labeling. First, a fixed step size is set, typically based on the smallest component size of ancient buildings, such as 10 to 20 centimeters. Then, each point in the coarse point cloud data is traversed, and the voxel address index corresponding to that point is calculated using a floor function. The voxel address index consists of three integers, corresponding to the grid number along the X, Y, and Z axes. A hash table or multidimensional array structure is used to record the point cloud index under the corresponding voxel address, thus establishing a mapping relationship. All non-empty voxel sets and their internal index lists together constitute the 3D occupancy mesh model.

[0034] For example, assume the moving machine is located in a relative coordinate system. At this point, the wide-angle scanning sensor captured a point on the wall of the ancient building, whose coordinates in the sensor coordinate system are: If the machine is not rotating at this time, through coordinate transformation, the coordinates of this point in the relative coordinate system become... .

[0035] If the fixed step size is set to 0.2 meters, then the voxel address index corresponding to that point is calculated as: X-axis index Y-axis index Z-axis index This point is mapped to the address. In the voxel grid.

[0036] The feature quantization calculation unit is used to perform least squares plane fitting based on the coarse point cloud data and the three-dimensional occupied grid model, calculate the root mean square of the fitting residual, construct an inverse proportional function relationship in combination with the point cloud density, and solve to output the uncertainty value.

[0037] For each non-empty grid in the 3D occupied mesh model, all point cloud data mapped within that grid are extracted. Then, a least-squares plane fitting algorithm is executed. This algorithm uses Singular Value Decomposition (SVD) or Principal Component Analysis (PCA) to calculate an optimal fitting plane that minimizes the sum of squared distances from all points to that plane. Based on this optimal fitting plane, the perpendicular distances from all points within the grid to the plane are calculated, and the root mean square (RMS) values ​​of these distances are obtained, i.e., the root mean square of the fitting residuals. Simultaneously, the total number of point clouds within the grid is calculated as the point cloud density. Finally, to comprehensively evaluate geometric complexity and sampling adequacy, an inverse proportional function is constructed to calculate the uncertainty. This calculation follows the following uncertainty evaluation formula: , in, Representing the uncertainty value, it is a dimensionless scalar. The larger the value, the worse the data quality or the more complex the structure in that area, and the more necessary it is for a robotic arm to perform close-range fine scanning. The root mean square of the fitting residual is represented by meters. It is derived from the distance statistics from the point cloud subset to the best-fit plane and is used to characterize the roughness or dispersion of the object surface within the current grid. The point cloud density, which represents the number of laser points contained within a grid, is used to characterize the richness of data sampling. It is a very small positive constant, for example This is used to prevent calculation errors where the denominator is zero; This is a normalized weighting coefficient used to adjust the calculation results to a range of 0 to 1 or a specific order of magnitude. Its value is based on the empirical ratio of sensor ranging accuracy to grid volume, for example, a value of 1000. Figure 3 As shown, the uncertainty value decreases inversely with the increase of point cloud density under different root mean square fitting residual conditions, verifying the model's sensitivity to low-quality data.

[0038] For example, suppose we are evaluating a grid located under an eave, containing 50 points. After least-squares plane fitting, the points are found to be randomly arranged, and the root mean square of the fitting residuals is calculated. It is 0.05 meters. If a weighting coefficient is taken... ,neglect The uncertainty value of the grid is... In contrast, another mesh located on a flat wall also has a point cloud density of 50, but its root mean square of the fitting residual is only 0.005 meters, and its uncertainty is calculated to be... Clearly, the grid below the eaves... Much higher than the wall Based on this, the eaves area is determined to be a high uncertainty area, which requires key observation.

[0039] Optionally, the solution to output an uncertainty value includes: Based on the three-dimensional occupied grid model, extract the point cloud subset of the current grid, count the number of point clouds to obtain the point cloud density, and calculate the vertical distance of the point cloud subset to the fitting plane to generate the root mean square of the fitting residual. When extracting a subset of the point cloud from the current mesh based on a 3D occupancy mesh model, the hash index table of the 3D occupancy mesh model is traversed to locate the address of the voxel mesh to be processed. All point cloud indices stored at that address are read, and the corresponding 3D coordinate data set, i.e., the point cloud subset, is retrieved from memory. Subsequently, two parallel statistical calculations are performed. The first is direct counting, counting the total number of coordinate points contained in the point cloud subset, defining this as the point cloud density, denoted as . The second term is geometric fitting, which calculates the eigenvector corresponding to the smallest eigenvalue of the covariance matrix of the point cloud subset. This eigenvector is the normal vector of the fitted plane. Based on the determined equation of the fitted plane... Calculate the Euclidean perpendicular distance from each point in the subset to the plane. To quantify the overall dispersion, calculate the root mean square (RMSE) of these perpendicular distances, defining it as the root mean square of the fitted residuals, denoted as . The calculation follows the formula: , in, Represents the root mean square of the fitting residuals, which physically represents the dispersion of the point cloud distribution within the grid relative to the ideal plane, and is expressed in meters. The total number of points in a subset of the point cloud represents the point cloud density. This represents the vertical distance from the i-th point to the fitted plane, and its value is calculated by substituting the point coordinates into the plane equation.

[0040] When the mesh covers the smooth walls of an ancient building, Generally close to 0, Extremely small; when the mesh covers complex carvings or broken edges, the point cloud distribution is messy. The differences are significant. Significantly increased.

[0041] For example, suppose we are processing a mesh on the surface of an ancient building column, mesh ID: 1024. This mesh contains 10 laser points. After plane fitting, the distances from these 10 points to the plane are calculated as: 0.001 meters, -0.001 meters, 0.002 meters, etc. The square root of the sum of the squares of these 10 distance values ​​is then used to calculate... The value is 0.0015 meters. This extremely small value indicates that the surface of the area is very flat and the sensor noise level is low.

[0042] The root mean square of the fitted residuals is used as the numerator data, and the point cloud density is used as the denominator data. Weighted division is then performed to output the uncertainty value.

[0043] When performing the step of using the root mean square of the fitted residuals as the numerator and the point cloud density as the denominator, and then performing weighted division to output the uncertainty value, the aim is to construct a comprehensive evaluation index that allows regions with more complex surfaces or sparser data to receive higher scores. The floating-point arithmetic unit is invoked to execute the following weighted division logic: , in, Represents the uncertainty value of the final output, which is dimensionless; This is the root mean square of the fitting residuals calculated above; The point cloud density is the one mentioned above. Let be the numerical stability constant, and let its value be . Its function is to prevent the calculation error of dividing by zero when there are no points in the grid; The dimensional balance weighting coefficient is determined based on the ratio of the ranging accuracy of the wide-angle scanning sensor to the side length of the grid voxels, and is typically set to a value of [value missing]. This coefficient amplifies the tiny meter-level residual values, making them numerically comparable to the point cloud density, thereby ensuring the accuracy of the calculated values. The value has discriminative power. Uncertainty. It is directly proportional to the fitting error and inversely proportional to the sampling density.

[0044] Optionally, the coupled decision module includes: The target index unit is used to perform a numerical comparison between the uncertainty value and the preset threshold. If the conditions are met, the grid coordinates are added to the observation queue. The abstract model quality assessment results are translated into specific robot execution instructions, responsible for autonomously deciding "where to collect data," "in what posture to collect data," and "how to reach the collection point." This process traverses every grid in the 3D occupancy grid model, reading its uncertainty value. To ensure the adaptability of the selection criteria, the uncertainty values ​​of all non-empty grids in the 3D occupancy grid model are first statistically analyzed, a histogram of the numerical distribution is constructed, and the global uncertainty mean and standard deviation are calculated. Subsequently, the adaptive discrimination threshold is calculated using the following statistical formula: , in, The adaptive discrimination threshold is a dynamic boundary line that distinguishes between "qualified areas" and "areas requiring supplementary testing". Represents the global uncertainty mean, which is derived from the arithmetic mean of all grid uncertainty values ​​and reflects the overall scanning quality benchmark of the current scene; The standard deviation of global uncertainty reflects the degree of dispersion in data quality. The confidence interval coefficient is a dimensionless statistical constant, typically ranging from 1.0 to 2.0. Only grid cells with uncertainties significantly deviating from the overall average are considered anomalous or unobservable. If the uncertainty value of a certain grid cell is higher than the overall average, it indicates an anomaly. The center three-dimensional coordinates of the grid are then added to a dynamically updated queue of observations.

[0045] For example, suppose the uncertainty values ​​for 10,000 grid points are output. Calculate the mean of these values. Standard deviation Set the confidence interval coefficient. The calculated adaptive discrimination threshold At this point, a grid located in the gap of the bracket set is scanned, and its uncertainty value is... .because The grid cell was immediately marked as a target to be observed and added to the queue for processing; while the other grid cell located on the flat wall had a value of Although it is higher than the average, it does not exceed the threshold, so it is not included in the supplementary testing plan, thus saving computing resources.

[0046] The line-of-sight analysis unit is used to perform ray projection detection from the center of the grid to be observed outward, identify the direction of rays that are not blocked by the distribution of obstacles, and determine the optimal observation pose by combining the sensor field of view. A target mesh is extracted from the observation queue, and a virtual spherical sampling space is constructed with the geometric center of this mesh as the origin. Raycasting detection, a computer graphics algorithm designed to determine spatial visibility, is performed. Virtual rays are emitted in various sampling directions within the spherical space, using a 3D occupied mesh model as an obstacle distribution map. For each ray, it is checked whether it collides with a voxel marked as "occupied" along its path. If a collision occurs, the direction is marked as "occluded"; if no collision occurs and the ray length is within the effective depth of field of the sensor, the direction is marked as "feasible direction." Among all feasible directions, the optimal observation pose is determined by combining the field of view (FOV) parameters of the sensor mounted at the end of the robotic arm, such as 70 degrees horizontally and 55 degrees vertically, to maximize coverage of the target mesh and minimize the angle between the FOV and the normal vector of the mesh fitting plane.

[0047] For example, the center of the target grid is located at coordinates And it is known that the surface normal vector at that location points to The emitted rays were found to be blocked by the external scaffolding along the positive Y-axis. However, rays along the negative Y-axis, at a distance of 1.0 meter from the target... There are no obstacles between the object and the target. The location is assessed as follows: a distance of 1.0 meter falls within the sensor's optimal imaging range, and the line of sight is parallel to the surface normal vector. Therefore, the coordinates... and orientation The pose is defined as the optimal observation pose.

[0048] The motion calculation unit is used to input the optimal observation pose into the robotic arm for inverse kinematics solution, calculate joint angles to generate deformation parameters, and construct an obstacle avoidance movement path connected to the optimal observation pose based on the current position of the mobile machine.

[0049] First, it inputs the optimal observation pose into the inverse motion solver of the robotic arm. This solver, based on the link length parameters of the robotic arm, solves a set of nonlinear equations and outputs a sequence of target angles for each joint of the robotic arm to rotate; these angles are the deformation parameters. Simultaneously, based on the current position of the mobile mother machine, it uses the Fast Exploratory Random Tree (RRT) algorithm to perform path planning on a two-dimensional projected map of a three-dimensional occupied grid model. This planning treats the mobile mother machine as a rigid body and uses the base docking point corresponding to the optimal observation pose as the endpoint, generating a series of navigation waypoints to avoid obstacles, thus forming an obstacle-avoidance movement path.

[0050] Optionally, the device further includes: The uncertainty value is extracted as a gain index, and the length of the obstacle avoidance path is extracted as a cost index. The ratio of the gain index to the cost index is calculated to generate a collection priority coefficient.

[0051] After generating corresponding observation schemes for each grid in the observation queue, each scheme is immediately prioritized. The uncertainty value of each grid is directly extracted and defined as a gain index, denoted as . Simultaneously, the geometric length of the obstacle avoidance path paired with this grid is extracted. This length represents the Euclidean distance the mobile mother machine needs to travel from its current position to the optimal observation position, and it is directly defined as the cost index, denoted as . This quantifies the time and energy costs required to perform the data collection task. After obtaining these two key metrics, a ratio calculation is performed to generate the final data collection priority coefficient, denoted as... The calculation follows the priority evaluation formula: , in, This represents the collection priority coefficient, which is a dimensionless positive real number used for sorting. The larger the value, the higher the priority. The value of the gain index is directly equal to the uncertainty value. This represents the cost metric, specifically the total length of the obstacle avoidance path, measured in meters. This ratio effectively balances the gains from data collection with the costs of execution. For example... Figure 4 As shown, the distribution of the grid to be observed in the coordinate system of uncertainty gain and movement path cost is displayed. The darker the color, the higher the acquisition priority coefficient, which verifies the decision logic of prioritizing high-yield and low-cost targets.

[0052] For example, assume the data acquisition device is currently located in the center of a courtyard. Two grids to be observed are identified: Grid A is located at a stone carving on the ground 2 meters away from the device; due to its complex texture, its uncertainty value is... Grid B is located at the eaves and brackets 20 meters away from the device. Due to structural obstruction, its uncertainty value is also [value missing]. For grid A: Gain metric obstacle avoidance movement path length Meters. Substitute into the formula to calculate, priority coefficient. For grid B: Gain metric obstacle avoidance movement path length Meters. Substitute into the formula to calculate, priority coefficient. Obviously, Much larger Based on this, although the data quality at both locations is equally poor, the cost of collecting data from grid A is significantly lower. Therefore, the device will be prioritized to scan the stone carvings on the ground. After processing the nearby low-cost tasks, the device will then proceed to process the distant eaves.

[0053] Optionally, the adaptive execution module includes: The collaborative motion execution unit is used to synchronously execute the navigation command of the obstacle avoidance movement path and the joint action command of the deformation parameters according to the acquisition priority coefficient, so as to deliver the end effector of the robotic arm to the complex gap area. The process begins with task scheduling, which involves reading the task queue in real time and locking in the task with the highest collection priority coefficient. The observed grid is used as the current execution target. To improve operational efficiency, a cooperative control strategy involving in-journey deployment is adopted. It includes a dual-channel motion controller; the first channel is responsible for resolving the obstacle avoidance path, discretizing it into a series of time-stamped linear velocities. With angular velocity The commands are sent via the bus to the underlying differential driver of the mobile host machine; the second channel is responsible for resolving deformation parameters, namely the target angles of each joint calculated by inverse kinematics. This signal is converted into a pulse width modulation (PWM) signal or a position control command and sent to the servo motor driver of the robotic arm. These two channels are synchronized on a microsecond-level time base, ensuring that the robotic arm is fully deployed and at the optimal approach angle just as the moving machine reaches the target position.

[0054] For example, suppose a high-priority grid is selected located below the dovetail beam at the top of the gallery of an ancient building. Priority coefficients are collected. The obstacle avoidance path indicates that the mother machine needs to move forward 3.5 meters and turn 30 degrees to the left, while the deformation parameters require the robotic arm to change from a folded state to a "Z"-shaped extended state. The total time is calculated to be 10 seconds. From second 0 to second 8, the mother machine performs the movement, while the robotic arm only releases the brake to fine-tune the gravity balance; from second 8 to second 10, the mother machine performs the final fine parking adjustment, while the robotic arm quickly executes joint movements, precisely placing the sensor-equipped end into the shadowed gap under the dovetail beam at the 10-second mark.

[0055] The state lock trigger unit is used to monitor the end effector of the robotic arm to stabilize and lock the joints of the robotic arm, and simultaneously trigger the image sensor and depth sensor to capture high-definition images and depth data.

[0056] Upon receiving the feedback signal, the end effector's inertial measurement unit (IMU) is immediately activated. The three-axis acceleration data of the end effector is sampled at high frequency. And calculate the comprehensive vibration amplitude in real time. A strict stability threshold was set. The threshold value is calculated based on the image sensor's exposure time and the allowable pixel blur radius. The determination logic follows the following inequality: , in, This represents the current overall vibration amplitude, with physical units of 1. ; To ensure stability, a threshold value is set, for example, to [value]. The robotic arm is considered to be in a "stable" state only when the inequality holds continuously within a certain time window. Once the stability condition is met, two synchronous operations are immediately executed: First, a "position loop lock" command is sent to the servo motors of all joints of the robotic arm to rigidly lock the current posture; second, falling edge trigger signals are sent simultaneously to the image sensor and the depth sensor via hardware trigger lines.

[0057] For example, when the robotic arm's end effector just extends into the dougong (a type of bracket system), due to the sudden stop, the end effector generates a frequency of 5Hz and an amplitude of [missing value]. Aftershocks were detected. Therefore, we remain in a waiting state and do not take pictures. After approximately 400 milliseconds of damping decay, the overall vibration amplitude drops to... This process lasted for 200 milliseconds. At this point, the system was deemed "stable," the joint motor was immediately locked, and a trigger signal was sent. The image sensor and depth sensor simultaneously exposed within a microsecond delay after receiving the signal, capturing a high-resolution image without motion blur and a frame of precisely registered depth data. This mechanism ensures that clear, textured data can be obtained even inside poorly lit ancient buildings.

[0058] Optionally, the incremental closed-loop module includes: A mesh state repair unit is used to fill empty data voxels in the three-dimensional occupied mesh model with the depth data and update the surface visual texture of the empty data voxels with the high-resolution image. The precise pose of the robotic arm's end effector at the moment of acquisition is read, and the point cloud data in the depth sensor coordinate system is transformed to the global relative coordinate system using a rigid body transformation matrix. Then, ray projection is used to traverse these global point clouds, calculating the voxel coordinates of each point cloud within the 3D occupied mesh model. Voxels originally marked as "empty" or "unknown" are updated to "occupied," thus restoring the integrity of the geometric model. Next, the surface visual texture is updated. This requires solving the mapping problem from 3D voxels to 2D image pixels. Using the principle of a pinhole camera model, the center coordinates of each newly restored voxel are projected onto the synchronously captured high-resolution image plane to index the corresponding RGB color value.

[0059] The cyclic decision triggering unit is used to search for residual grids that exceed the limit in the uncertainty value in real time. If there are residual grids, a replanning instruction is sent to the coupled decision module; otherwise, the three-dimensional occupied grid model is locked.

[0060] After the mesh state repair is completed, a local update logic is triggered to recalculate the uncertainty values ​​of the newly repaired region and its neighboring meshes. Due to the introduction of new data, the point cloud density in these regions... Significantly increased, fitting residuals It usually converges, resulting in an uncertainty value. A significant decrease was observed. Subsequently, a global search was performed on the 3D occupied mesh model to determine if any residual meshes existed.

[0061] Residual meshes are defined as those whose uncertainty values ​​are still greater than the aforementioned adaptive discrimination threshold. The model is a grid. As long as any substandard "dead corner" exists in the model, a replanning instruction must be output to feed back the coordinates of the remaining grid to the coupled decision module, triggering a new round of viewpoint calculation and acquisition; only when the quality of all grids meets the standard is the task considered complete, the model locked and the final result output.

[0062] Based on the same inventive concept, this invention also provides a multi-source data acquisition method for three-dimensional modeling of ancient buildings, the method comprising: The mobile mother unit is controlled to deploy ultra-wideband positioning beacons in the ancient building area, and wireless ranging signals are obtained through the mobile mother unit and the positioning beacons to establish a relative coordinate system. Under the aforementioned relative coordinate system, coarse point cloud data of the ancient buildings are acquired, a three-dimensional occupancy grid model is constructed, and the uncertainty value is calculated for each grid. The grids with uncertainty values ​​higher than a preset threshold are extracted as the grids to be observed. The optimal observation pose is calculated for the grids to be observed based on the obstacle distribution, and the deformation parameters and obstacle avoidance movement path of the robotic arm are calculated and generated. The drive acquisition device travels along the obstacle avoidance movement path, and the control robot arm executes the deformation parameters, so that the end of the robot arm enters a complex gap area to acquire high-definition images and depth data; The high-resolution imagery and depth data are fused into the 3D occupancy grid model, the uncertainty values ​​are updated, and viewpoint calculation and acquisition are repeated until the uncertainty values ​​of all grids meet the requirements.

[0063] It should be noted that the formulas described above, through the principle of dimensional consistency and mathematical standardization methods (such as normalization, dimensionless parameter conversion, or unit device unification), can translate physical quantities with different properties into unitless standard values ​​or parameters that can be superimposed in the same dimension. This eliminates the interference of different dimensions on the computational logic, allowing the formulas to retain the original data distribution characteristics while possessing mathematical rationality and adaptability to objective laws. These are conventional technical methods and will not be elaborated further. The electrical connections between the various units described above do not necessarily represent direct or indirect connections; any indirect connection method is applicable to the embodiments of this invention as long as it achieves the purpose of this invention. The above descriptions are merely exemplary embodiments of this invention and should not be construed as limiting the scope of this invention.

[0064] All equivalent changes and modifications made in accordance with the teachings of this invention are still within the scope of this invention. Those skilled in the art will readily conceive of other embodiments of this invention upon considering the specification and the disclosure of practical truth. This application is intended to cover any variations, uses, or adaptations of this invention that follow the general principles of this invention and include common knowledge or conventional techniques in the art not described herein.

Claims

1. A multi-source data acquisition device for 3D modeling of ancient buildings, characterized in that, The device includes: The environment initialization module is used to control the mobile mother unit to deploy ultra-wideband positioning beacons in the ancient building area, and to obtain wireless ranging signals through the mobile mother unit and the ultra-wideband positioning beacons to establish a relative coordinate system. The global cognition module is used to acquire rough point cloud data of ancient buildings in the relative coordinate system, construct a three-dimensional occupancy grid model, and calculate the uncertainty value for each grid. The coupled decision module is used to extract the grids with uncertainty values ​​higher than a preset threshold as the grids to be observed, calculate the optimal observation pose for the grids to be observed using obstacle distribution, and solve to generate the deformation parameters and obstacle avoidance movement path of the robotic arm. An adaptive execution module is used to drive the acquisition device to travel along the obstacle avoidance movement path, control the robotic arm to execute the deformation parameters, and enable the end of the robotic arm to enter a complex gap area to acquire high-definition images and depth data. The incremental closed-loop module is used to fuse the high-definition image and the depth data into the three-dimensional occupied grid model, update the uncertainty value, and repeatedly perform viewpoint calculation and acquisition until the uncertainty value of all grids meets the requirements.

2. The multi-source data acquisition device for three-dimensional modeling of ancient buildings according to claim 1, characterized in that, The environment initialization module includes: The beacon deployment interaction unit is used to drive the mobile mother machine to release ultra-wideband positioning beacons at intervals and to activate two-way time-of-flight ranging interaction between the mobile mother machine and the ultra-wideband positioning beacons. The distance matrix construction unit is used to acquire wireless ranging signals through the mobile mother unit and the ultra-wideband positioning beacon to extract timestamp differences and calculate point-to-point Euclidean distances, thereby constructing a distance observation matrix. The coordinate system calculation unit is used to perform least squares adjustment calculation on the distance observation matrix with the ultra-wideband positioning beacon as the coordinate origin and the ultra-wideband positioning beacon as the coordinate axis, and output the relative coordinate system.

3. The multi-source data acquisition device for three-dimensional modeling of ancient buildings according to claim 1, characterized in that, The global cognition module includes: The data mapping unit is used to activate the wide-angle scanning sensor of the mobile mother machine to perform panoramic data capture and to perform coordinate transformation using the relative coordinate system to generate coarse point cloud data. The grid discretization unit is used to perform voxel segmentation based on the coarse point cloud data with a fixed step size, establish a mapping relationship between point cloud index and voxel address, and construct a three-dimensional occupied grid model. The feature quantization calculation unit is used to perform least squares plane fitting based on the coarse point cloud data and the three-dimensional occupied grid model, calculate the root mean square of the fitting residual, construct an inverse proportional function relationship in combination with the point cloud density, and solve to output the uncertainty value.

4. The multi-source data acquisition device for three-dimensional modeling of ancient buildings according to claim 3, characterized in that, The solution process, which outputs an uncertainty value, includes: Based on the three-dimensional occupied grid model, extract the point cloud subset of the current grid, count the number of point clouds to obtain the point cloud density, and calculate the vertical distance of the point cloud subset to the fitting plane to generate the root mean square of the fitting residual. The root mean square of the fitted residuals is used as the numerator data, and the point cloud density is used as the denominator data. Weighted division is then performed to output the uncertainty value.

5. The multi-source data acquisition device for three-dimensional modeling of ancient buildings according to claim 1, characterized in that, The coupled decision module includes: The target index unit is used to perform a numerical comparison between the uncertainty value and the preset threshold. If the conditions are met, the grid coordinates are added to the observation queue. The line-of-sight analysis unit is used to perform ray projection detection from the center of the grid to be observed outward, identify the direction of rays that are not blocked by the distribution of obstacles, and determine the optimal observation pose by combining the sensor field of view. The motion calculation unit is used to input the optimal observation pose into the robotic arm for inverse kinematics solution, calculate joint angles to generate deformation parameters, and construct an obstacle avoidance movement path connected to the optimal observation pose based on the current position of the mobile machine.

6. The multi-source data acquisition device for three-dimensional modeling of ancient buildings according to claim 1, characterized in that, The device further includes: The uncertainty value is extracted as a gain index, and the length of the obstacle avoidance path is extracted as a cost index. The ratio of the gain index to the cost index is calculated to generate a collection priority coefficient.

7. The multi-source data acquisition device for three-dimensional modeling of ancient buildings according to claim 6, characterized in that, The adaptive execution module includes: The collaborative motion execution unit is used to synchronously execute the navigation command of the obstacle avoidance movement path and the joint action command of the deformation parameters according to the acquisition priority coefficient, so as to deliver the end effector of the robotic arm to the complex gap area. The state lock trigger unit is used to monitor the end effector of the robotic arm to stabilize and lock the joints of the robotic arm, and simultaneously trigger the image sensor and depth sensor to capture high-definition images and depth data.

8. The multi-source data acquisition device for three-dimensional modeling of ancient buildings according to claim 1, characterized in that, The incremental closed-loop module includes: A mesh state repair unit is used to fill empty data voxels in the three-dimensional occupied mesh model with the depth data and update the surface visual texture of the empty data voxels with the high-resolution image. The cyclic decision triggering unit is used to search for residual grids that exceed the limit in the uncertainty value in real time. If there are residual grids, a replanning instruction is sent to the coupled decision module; otherwise, the three-dimensional occupied grid model is locked.

9. A multi-source data acquisition method for 3D modeling of ancient buildings, applied to the multi-source data acquisition device for 3D modeling of ancient buildings as described in any one of claims 1-8, characterized in that, The method includes: The mobile mother unit is controlled to deploy ultra-wideband positioning beacons in the ancient building area, and wireless ranging signals are obtained through the mobile mother unit and the positioning beacons to establish a relative coordinate system. Under the aforementioned relative coordinate system, coarse point cloud data of the ancient buildings are acquired, a three-dimensional occupancy grid model is constructed, and the uncertainty value is calculated for each grid. The grids with uncertainty values ​​higher than a preset threshold are extracted as the grids to be observed. The optimal observation pose is calculated for the grids to be observed based on the obstacle distribution, and the deformation parameters and obstacle avoidance movement path of the robotic arm are calculated and generated. The drive acquisition device travels along the obstacle avoidance movement path, and the control robot arm executes the deformation parameters, so that the end of the robot arm enters a complex gap area to acquire high-definition images and depth data; The high-resolution imagery and depth data are fused into the 3D occupancy grid model, the uncertainty values ​​are updated, and viewpoint calculation and acquisition are repeated until the uncertainty values ​​of all grids meet the requirements.

Images