Three-dimensional scene digital fusion method for internal and external field environmental characteristics of intelligent test

Abstract

The invention provides a three-dimensional scene digital fusion method for intelligent test internal and external field environment characteristics, and belongs to the technical field of three-dimensional scene digital fusion. According to the method, multi-source data is normalized through a data fusion algorithm, so that the data source of three-dimensional live-action construction is more sufficient and more comprehensive, the coverage integrity of a combat scene is improved, the surveying and mapping precision and modeling details are improved, the labor cost is further reduced, and the method is of great significance to three-dimensional live-action construction of internal and external fields.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional scene digital fusion, and in particular relates to a three-dimensional scene digital fusion method oriented to the characteristics of indoor and outdoor environments of intelligent experiments. Background Art

[0002] Different test objects in the indoor and outdoor fields will face cross-domain interactions, that is, interactions between objects in the physical field and objects in the virtual field. The test environment needs to have the ability to design, allocate, and implement interactions between objects in these different fields, combining various real, virtual, and constructed resources distributed around the world to build an integrated test environment. Based on this, it is necessary to establish a three-dimensional spatial model of the indoor and outdoor fields to support a more intuitive and convenient display of target scenes.

[0003] At present, the three-dimensional spatial modeling of the indoor and outdoor fields of the LVC joint test mostly adopts the spatial modeling technology of a single data source, including indoor and outdoor three-dimensional laser point cloud data modeling technology, indoor and outdoor depth image data modeling technology, indoor and outdoor CAD data modeling technology, indoor and outdoor BIM data modeling technology, indoor and outdoor panoramic image data modeling technology, etc.

[0004] The flow chart of the prior art is shown in the attached Figure 1 As shown in the figure, this technology is currently the most common modeling technology for 3D real-life indoor and outdoor fields. First, an overview of the survey area is drawn, feature points and feature edges are selected, and control points, machines and targets are arranged according to the features of the survey area. According to the control points and targets, a single data source is used for collection in combination with the feature layout. The data generated by the single data source is preprocessed and uniformly registered, that is, the data obtained from different viewpoints are unified into a coordinate system. The surface coverage data of the entire indoor and outdoor real scene is obtained through registration, and then the 3D model is modeled.

[0005] The 3D real-scene modeling technology under a single data source has its limitations and the defect of being unable to fully express the detailed features of the surface of an object. For example, due to the limitations of the instrument itself, 3D laser scanning is more suitable for scenes with a wide field of view to avoid occlusion, and cannot cover all scenes; and when using a plane CAD drawing to perform 3D modeling in a modeling software, it is necessary to initially lay out control points, perform field measurements of fragments, and map making in the office, which requires more manual intervention. This leads to certain limitations in scene coverage, accuracy, modeling details, and modeling integrity in the 3D modeling technology under a single data source. Summary of the invention

[0006] In view of the above technical problems, the present invention proposes a three-dimensional scene digital fusion method for the characteristics of the indoor and outdoor environments of intelligent experiments. The method uses the data of the three-dimensional real indoor and outdoor scenes collected by various methods, and fuses them according to the characteristics of different data sources and the selected algorithm rules to obtain more complete, more accurate and more effective information and spatial structure models.

[0007] The method comprises:

[0008] Step S1, multi-source data collection;

[0009] Collect multi-source data in multiple domains inside and outside the 3D real scene, and generate 3D laser point cloud data, plane CAD drawings, BIM data, depth image data and holographic image data for the same measurement area;

[0010] Step S2, multi-source data preprocessing;

[0011] Perform point cloud denoising, point cloud matching, and point cloud thinning processing on 3D laser point cloud data; perform layer cleaning, coordinate conversion, and element simplification processing on plane CAD drawings; perform semantic mapping, semantic filtering, and individual element extraction processing on BIM data; perform image matching, image stitching, and 2D image generation processing on depth image data and holographic image data respectively;

[0012] Step S3, multi-source data fusion modeling;

[0013] The preprocessed three-dimensional laser point cloud data, plane CAD drawings, and BIM data are normalized and fused to form a multi-source data fusion model, and the texture information of the preprocessed depth image data and holographic image data is mapped to the multi-source data fusion model to obtain a three-dimensional real-scene physical space model of the indoor and outdoor fields.

[0014] In the step S2, point cloud denoising, point cloud matching, and point cloud thinning are performed on the three-dimensional laser point cloud data; specifically, the steps include:

[0015] Remove noise data and outliers due to occlusion scanning through point cloud denoising. Point cloud denoising methods include bilateral filtering, Gaussian filtering, binning denoising, KD-Tree, straight-through filtering, and random sampling consistency filtering.

[0016] Through point cloud registration processing, point clouds in different coordinate systems are converted to the same coordinate system to obtain a complete point cloud model; wherein: first, the feature points of the target object are selected, including geometric feature points and layout feature points. After the initial estimation of the rigid body transformation, the rotation and translation matrices of two adjacent point clouds are solved through the feature points with the same name, thereby completing the point cloud data splicing;

[0017] A voxel grid filter is used to create a voxel grid in the three-dimensional space where the point cloud is located, and the input point cloud data is divided into multiple tiny three-dimensional voxel grids. For each voxel in the grid, the centroid of all points in the voxel is used to replace the point in the voxel. The centroid points in all voxels constitute the filtered point cloud. The degree of downsampling is controlled by controlling the size of the voxel. The larger the voxel, the sparser the filtered point cloud.

[0018] In the step S2, semantic mapping, semantic filtering, and individual element extraction are performed on the BIM data; specifically, the steps include:

[0019] According to the semantic information in the IFC model of BIM data standard format, a semantic mapping relationship is established between the spatial entity model and the semantic information is transmitted;

[0020] Filter out unnecessary entities in the IFC model based on semantic mapping relationships, perform coordinate system conversion based on geometric differences, and convert IFC construction entity geometry to spatial data model geometry representation;

[0021] By reading and filtering the semantic information in the IFC model, the individual elements of the spatial data model and their attribute information are extracted.

[0022] In the step S2, image matching, image stitching, and two-dimensional image generation processing are performed on the depth image data and the holographic image data; wherein:

[0023] Perform image data preprocessing and feature extraction on the depth image data and holographic image data, and match the extracted feature points after removing the mismatched points;

[0024] The positions of the overlapping parts of two adjacent images are obtained, the mapping relationship of the registration functions of the two images is determined, the overlapping parts of the images are smoothed, and then the images are stitched.

[0025] In the step S3, the pre-processed three-dimensional laser point cloud data, plane CAD drawing, and BIM data are normalized; specifically, the process includes:

[0026] Perform coordinate conversion on pre-processed 3D laser point cloud data, 2D CAD drawings, and BIM data to unify them into the same coordinate reference;

[0027] Among them, based on the characteristic target of the scanning system, the normal vector value and geometric center coordinates of the spherical characteristic target are used as the common control point to solve the coordinate conversion parameters of two adjacent point cloud data. The conversion process is realized by using the Bursa seven-parameter model of metrology.

[0028] Among them, a three-dimensional model with actual geographical location is established by converting the geodetic coordinate control points in the known CGCS2000 coordinate system on the ground. After setting the coordinates of the common points, the conversion parameters are obtained based on the common points to complete the coordinate conversion.

[0029] In the step S3, data fusion processing is performed on the normalized three-dimensional laser point cloud data, the plane CAD drawing, and the BIM data; specifically, the processing includes:

[0030] For the 3D laser point cloud data, the inclined image is densely matched to obtain a 3D feature point set, and the point set is subjected to gross error elimination and simplification processing; the iterative closest point algorithm is used to align with the 3D laser point cloud and output the point cloud data;

[0031] Given two point cloud models P and Q to be registered, point P in point cloud P i Find the point Q with the shortest Euclidean distance from the point cloud Q i , with P i and Q i As the corresponding point pair, the transformation matrix is ​​obtained. Through continuous iterative calculation and minimization of the error function, the optimal transformation matrix is ​​obtained to make the two point clouds coincide.

[0032] Rotate and translate by matching points, using the least squares method as a criterion until the distance between points reaches a preset threshold;

[0033] Further integrate CAD, BIM data and point cloud data, extract basic structures from CAD drawings using machine learning, extract monomer elements and corresponding semantic information from BIM and integrate element point clouds, and perform orthographic projection on the integrated point clouds to obtain data organization in two dimensions;

[0034] The derivative-free optimization algorithm is used to solve the four-degree-of-freedom transformation problem, that is, the translation and scaling of the x and y coordinates, to obtain no less than two common points for the fusion of CAD and the projected two-dimensional point cloud, and to perform four-parameter coordinate transformation.

[0035] In the step S3, mapping the pre-processed depth image data and texture information of the holographic image data to the multi-source data fusion model specifically includes:

[0036] The texture information of the depth image and the panoramic image is mapped to generate a three-dimensional real scene interior and exterior physical space model with color texture information, that is, the texture image of the object surface corresponding to the target physical three-dimensional model is obtained by using the depth image and the panoramic image;

[0037] Among them, for the parts blocked by the modeled object, the texture information captured from different sides of the object is used to extract and map the texture information respectively, and then the extracted and mapped texture information is fused and registered with the model to generate a complete model.

[0038] In summary, the technical solution of the present invention is based on actual needs. In view of the current problems of single source of data for indoor and outdoor three-dimensional real scene modeling, incomplete scene coverage, low mapping accuracy, fuzzy modeling details, and high labor costs, a three-dimensional scene digital fusion method for the characteristics of indoor and outdoor environments of intelligent experiments is proposed. The method utilizes the data of indoor and outdoor three-dimensional real scenes collected by various methods, and fuses them according to the characteristics of different data sources and selected algorithm rules, extracts the respective advantages from each data source, and complements them to obtain more complete, more accurate, and more effective information and spatial structure models, so as to achieve a comprehensive description of the target or scene, and ensure high reliability, target detection and recognition accuracy, and change detection and information update capabilities. The method realizes the integrated integration of multi-source data in geometry and semantics, and constructs a textured, high-fidelity three-dimensional solid spatial structure model. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 It is a flow chart of the prior art;

[0041] Figure 2 This is a technical architecture diagram of the present invention;

[0042] Figure 3 Flow chart for BIM data preprocessing;

[0043] Figure 4 is a flow chart of Example 1;

[0044] Figure 5 This is a flow chart of Example 2. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0046] The overall technical architecture of the present invention is shown in the attached Figure 2 As shown in the figure, it is divided into three parts: multi-source data acquisition, multi-source data preprocessing and multi-source data fusion modeling.

[0047] 1. Multi-source data collection

[0048] The three-dimensional real-scene indoor and outdoor multi-domain comprehensive integration method adopts multi-source data collection to generate three-dimensional laser point cloud, plane CAD drawing, BIM data, depth image data and holographic image data for the same measurement area.

[0049] 3D laser point cloud scanning performs scanning and measurement in a static data acquisition mode, and finally establishes the point cloud of the entire scene through point cloud stitching. The scanned point cloud can also be used for BIM model and CAD drawing generation.

[0050] The specific process of CAD field mapping is to plan the measurement route according to the distribution of control points and terrain conditions in the survey area, use professional surveying and mapping instruments to measure the coordinates of the measurement points, and perform adjustment calculations on the measurement results to analyze whether they meet the accuracy requirements. If not, re-measurement is required. After meeting the accuracy requirements, closed-loop correction is performed on the coordinates of the measurement points, and then the coordinates of the measurement points are used as control points for fragmented measurements and imported into relevant software to draw CAD drawings.

[0051] BIM technology is obtained through oblique photogrammetry, point cloud scanning, Revit drawing, facade engineering quantity statistics, etc.

[0052] Depth image acquisition, i.e., distance image, uses the depth from the image collector to each point in the scene as the pixel value, directly reflecting the geometry of the visible surface of the scene. The grayscale value of each pixel in the depth image can be used to characterize the distance of a certain point in the scene from the camera. The depth image can be converted into point cloud data by coordinate transformation using the camera's internal calibration parameters.

[0053] Panoramic image acquisition has the characteristics of full viewing angle, virtual reality and high immediacy. It can be combined with laser point cloud to form a multi-sensor space target entity mobile measurement system integrating laser scanner and panoramic camera.

[0054] 2. Multi-source data preprocessing

[0055] There are different preprocessing methods for different types of data.

[0056] (1) 3D laser point cloud data preprocessing

[0057] When performing 3D laser scanning point cloud modeling for 3D real scenes, the modeling effect is greatly affected by the quality of the point cloud data, so the point cloud data needs to be preprocessed. Point cloud scanning results in complex environments have noise, many images, and large amounts of data, so point cloud filtering, point cloud registration, point cloud downsampling and other data preprocessing operations are required in sequence.

[0058] Point cloud denoising can effectively remove noise data and outliers obtained by scanning due to occlusion and other problems. There are many methods for point cloud denoising, such as bilateral filtering, Gaussian filtering, binning denoising, KD-Tree, straight-through filtering, random sampling consistency filtering, etc., and the scope of application is also different. In the application, the appropriate filter is selected according to the filtering effect. Point cloud registration is an important link in the data processing process. Its purpose is to convert point clouds in different coordinate systems into the same coordinate system through registration to obtain a complete point cloud model of the target object. During point cloud preprocessing, coarse registration is performed by manually selecting the feature points of the target object. The selection of target feature points mainly includes two categories, one is the geometric feature points of the target object itself, such as building corners and vertices; the other is artificially arranged feature points, such as targets. After the coarse registration provides an initial estimate of the rigid body transformation, the approximate rotation and translation matrix of the two adjacent point clouds is solved through the feature points of the same name, and then the two point clouds and all the laser point cloud data of the stations in the large scene are stitched. Since the point cloud scanning process is carried out entirely on the ground, the laser cannot penetrate obstacles in the target scene and cannot achieve full data coverage. The collection process is generally multi-station collection, and each measuring station has an independent coordinate system, so multi-station cloud data splicing is required.

[0059] The use of 3D laser scanning technology can obtain a large amount of data in a short time, which will cause more system resources to be occupied during processing and storage, and the computing efficiency is low. Due to multiple station measurements and splicing of multiple point cloud data, data redundancy and uneven point cloud density will occur, and point cloud thinning and downsampling operations are required. Voxel filters can reduce the number of points while ensuring that the original geometric structure of the point cloud remains basically unchanged, and speed up the execution of subsequent operations such as registration and reconstruction. The voxel grid filter creates a voxel grid in the three-dimensional space where the point cloud is located, and divides the input point cloud data into tiny three-dimensional voxel grids. For each voxel in the grid, the centroid of all points in the voxel is used to approximately replace the points in the voxel, and finally the centroid points in all voxels form the filtered point cloud. The degree of downsampling can be controlled by controlling the size of the voxel. The larger the voxel, the sparser the filtered point cloud.

[0060] (2) BIM data preprocessing

[0061] The spatial model conversion method based on BIM data mainly includes four steps: first, according to the rich semantic information in the IFC (BIM data standard format) model, a semantic mapping relationship is established between the building spatial entity model to transfer semantic information, and then unnecessary entities in the IFC model are filtered out based on the semantic mapping relationship, and the coordinate system is converted according to the geometric difference, and the IFC construction entity geometry is converted to the spatial data model geometry representation. Finally, by reading and filtering the semantic information in the IFC model, the individual elements of the spatial data model and their attribute information are extracted. The preprocessing process is shown in the attached figure. Figure 3 shown.

[0062] (3) Depth image / holographic image data preprocessing

[0063] Image stitching is one of the most important steps in panorama generation technology. The image stitching process is mainly divided into three stages: image processing stage, feature matching stage and image fusion stage. The detailed description is as follows:

[0064] 1) The image processing stage includes two parts: preprocessing and image feature extraction.

[0065] 2) The main task in the feature matching stage is to eliminate mismatched points and match the extracted feature points.

[0066] 3) The key to image registration and fusion is to accurately find the position of the overlapping parts of two adjacent images, determine the mapping relationship of the registration functions of the two images, and smooth the overlapping parts of the images so that the final image has a better visual effect.

[0067] 3. Multi-source data fusion modeling

[0068] Multi-source data fusion modeling is to use a variety of methods to collect three-dimensional real-life indoor and outdoor data, and fuse them according to the characteristics of different data sources and the selected algorithm rules, extract the respective advantages from each data source, and complement each other to obtain more complete, more accurate and more effective information and spatial structure models, so as to achieve a comprehensive description of the target or scene, and ensure high reliability, target detection and recognition accuracy, change detection and information update capabilities.

[0069] Based on the pre-processing of various data sources, fusion modeling is carried out. First, multi-source heterogeneous data such as ground 3D laser scanning are normalized and unified into the same coordinate system to achieve alignment between different model data. Then, the CAD drawing is used as the base map and the exported BIM is used as the basis for modeling. The point cloud, CAD, BIM and other models after spatial alignment are fused from the element level to the overall level according to a certain algorithm to generate a multi-source data fusion model with rich information. Finally, the texture information of the depth image and panoramic image is mapped to the above model to generate a high-quality 3D target entity space model with higher precision, complete color texture information and rich in color texture information.

[0070] (1) Data normalization

[0071] Data normalization is to complete the unification of the coordinate system. In order to construct the entire scene target entity model, feature extraction and registration strategies are required for multi-source data (point clouds, images, etc.), and the data collected from different sources are uniformly transformed into coordinates and integrated into a unified coordinate reference. The most commonly used registration method is to use the feature targets of the scanning system, use the normal vector value and geometric center coordinates of the spherical feature targets as common control points, and solve the coordinate transformation parameters of two adjacent point cloud data. The accuracy of data normalization directly affects the accuracy of subsequent modeling, so this step is the key to achieving high-precision modeling. Coordinate transformation generally calculates transformation parameters through common points or common targets. The transformation process usually uses the Bursa seven-parameter model in metrology, namely:

[0072]

[0073] In the formula, the three translation parameters [ΔX, ΔY, ΔZ] T , 3 rotation parameters [ε X ,ε Y ,ε Z ] T and a scale parameter m.

[0074] Sometimes, if you need to build a three-dimensional model with an actual geographical location, you also need to convert the geodetic coordinate control points in the known CGCS2000 coordinate system on the ground. After setting the coordinates of the common points, calculate the conversion parameters based on the common points to complete the coordinate conversion.

[0075] (2) Data fusion algorithm

[0076] Data fusion modeling uses 3D laser point cloud data as the main data for large-scale application scenarios, and fuses the two using ICP or its variant algorithms; CAD and BIM data are then used as secondary data to improve the accuracy of the data source.

[0077] When modeling, the CityBIM standard is used as the model specification, but the model reconstructed based on the above data lacks the corresponding appearance texture details. Therefore, the depth image and panoramic image data are mapped to the above model as color texture to obtain the final complete model, so as to achieve the purpose of objectively and truly reflecting the objects in the physical world.

[0078] For 3D laser point clouds, after preprocessing, the data fusion process is divided into two steps: the first step is to obtain a 3D feature point set by densely matching the high-quality oblique images, and then perform processing such as removing gross errors and simplifying the point set; the second step is to select a suitable algorithm to align with the 3D laser point cloud and output the point cloud data. The key to the problem lies in the precise registration of point clouds. The current mainstream algorithm is to use the Iterative Closest Point (ICP) algorithm or its variants. The ICP principle is that if two point cloud models P and P to be registered are known, the point P in the point cloud P i Find distance P from point cloud Q i The point Q with the shortest Euclidean distance i , and P i and Q i The transformation matrix is ​​obtained as the corresponding point pairs, and some point pairs with distant distances are eliminated. Through continuous iterative calculations, the error function is minimized, and finally the optimal transformation matrix is ​​obtained to make the two point clouds coincide. The error function is shown in the following formula.

[0079]

[0080] Where R represents the rotation transformation matrix; T represents the translation transformation matrix.

[0081] The essence of the ICP algorithm is to rotate and translate by matching points, using the least squares method as a measurement standard until the distance between points reaches a preset threshold. The ICP algorithm plays an irreplaceable role in the point cloud registration process. It can directly register through corresponding points without the need for point cloud segmentation, and its registration effect can also meet the required accuracy requirements.

[0082] Since CAD vector drawings and BIM building location coordinates are more accurate, if the point cloud data after the above fusion processing still cannot meet the model requirements, it is possible to consider further fusion processing of CAD, BIM data and point cloud data to improve the accuracy and reliability of model data. First, machine learning methods can be used to extract the necessary infrastructure (such as walls, doors and windows, etc.) from CAD drawings, extract monomer elements and corresponding semantic information from BIM, fuse element point clouds, and then perform orthographic projection on the fused point clouds to obtain data organization in the two-dimensional direction. Finally, a certain algorithm (such as derivative-free optimization algorithm) is used to solve the four-degree-of-freedom transformation problem, that is, the translation and scaling of x and y coordinates. The basis for the fusion of CAD and the two-dimensional point cloud after projection is to find no less than two common points and perform four-parameter coordinate transformation.

[0083] (3) Texture Mapping

[0084] In addition to the geometric, topological, and semantic information of the elements, a complete entity space model should also include appearance information such as texture. Using depth images and panoramic images, the texture image of the object surface corresponding to the target entity 3D model is obtained, and after a series of changes, it is mapped to the generated 3D model to form a more realistic model. For the parts blocked by the modeled object, the texture information captured from different sides of the object is used to extract and map the texture information respectively, and then these parts are modeled and registered to produce a complete model.

[0085] The method proposed in the present invention can solve the pain points of single data source, incomplete scene coverage, low mapping accuracy, fuzzy modeling details, and high labor costs under a single data source. The multi-source data is normalized by a data fusion algorithm, making the data source for three-dimensional real scene construction more sufficient and comprehensive, thereby improving the coverage completeness of the combat scene, improving mapping accuracy and modeling details, and further reducing labor costs, which is of great significance for the construction of three-dimensional real scenes in both indoor and outdoor fields.

[0086] Example 1

[0087] The three-dimensional scene digital fusion technology oriented to the characteristics of the internal and external field environments proposed in the present invention can solve the problem of three-dimensional modeling of the UAV cooperative combat environment.

[0088] The specific technical process of this embodiment is shown in the attached Figure 4As shown in the figure, firstly, establish a survey area portrait, deploy multi-viewpoint and multi-angle sensors, scanners, etc. Extract the feature points and feature edges of the real scene of the drone battlefield. According to the extracted features, collect multi-source data, including the three-dimensional terrain of the battlefield mountain, the real scene of the three-dimensional model of drones and missiles, and generate data from three-dimensional laser point clouds, BIM data and holographic image data sources. Then pre-process the multi-source data, including denoising, registration and sparseness of three-dimensional laser point cloud data, semantic mapping, filtering and feature extraction of BIM data, image processing, feature matching and image fusion of holographic image data. After that, the pre-processed data is normalized, and after continuous iteration of the data fusion algorithm, the multi-source data is finally unified into the same coordinate system. Finally, the processed data is used for modeling to obtain the three-dimensional model of the internal and external fields of the drone collaborative combat, so as to realize the construction of the combat environment.

[0089] Example 2

[0090] Through the three-dimensional scene digital fusion technology oriented to the characteristics of indoor and outdoor environments proposed in the present invention, the construction of a mobile binocular stereoscopic vision environment is realized, providing the mobile robot combat platform with the ability to model the environment's depth information in three dimensions.

[0091] The specific technical process of this embodiment is as follows Figure 5 As shown in the figure, in the field, through the cooperation of mobile robots and infield remote sensing technology operators, multiple sensors are embedded in the field mobile robots, and the data of three-dimensional laser point cloud and depth image data sources are obtained with the help of multiple types of sensors, and transmitted to the field in real time for processing. In the field, the multi-source data is preprocessed, including denoising, registration and thinning of three-dimensional laser point cloud data, image registration, image stitching and two-dimensional image mapping of depth image data. After that, the preprocessed data is normalized, and the data fusion algorithm is used to process it online in real time, and finally the multi-source data is unified into the same coordinate system. Finally, the processed data is used to model the three-dimensional model of the internal and external fields of the UAV cooperative combat, realize the construction of the combat environment, and coordinate the scene with the field mobile robot, which is of great significance for the real-time cooperative combat of the mobile robot.

[0092] It can be seen that different test objects in the indoor and outdoor fields will face cross-domain interactions, that is, interactions between objects in the physical field and objects in the virtual field. The test environment needs to have the ability to design, allocate, and implement interactions between objects in these different fields, combining various real, virtual, and constructed resources distributed in various places to build an integrated test environment. Based on this, it is necessary to establish a three-dimensional spatial model of the indoor and outdoor fields to support a more intuitive and convenient display of target scenes.

[0093] The present invention proposes a three-dimensional scene digital fusion technology. According to the characteristics of different data sources and the rules of multi-source data fusion algorithm, the data of the three-dimensional real scene indoor and outdoor collected by various methods are fused and processed, and the respective advantages are extracted from each data source to complement each other, so as to obtain more complete, more accurate and more effective information and spatial structure model. The present invention can solve the pain points of single data source, incomplete scene coverage, low mapping accuracy, fuzzy modeling details and high labor cost under a single data source, so that the data source for the construction of three-dimensional real scene is more sufficient and comprehensive, thereby improving the coverage completeness of the combat scene, improving the mapping accuracy and modeling details, and further reducing the labor cost, which is of great significance for the construction of three-dimensional real scene in indoor and outdoor fields.

[0094] Please note that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. The above-mentioned embodiments only express several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, without departing from the concept of the present invention, several variations and improvements can be made, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be based on the attached claims.

Claims

1. A three-dimensional scene digital fusion method for the characteristics of indoor and outdoor environments of intelligent experiments, characterized by: The method comprises: Step S1, multi-source data collection; Collect multi-source data in multiple domains inside and outside the 3D real scene, and generate 3D laser point cloud data, plane CAD drawings, BIM data, depth image data and holographic image data for the same measurement area; Step S2, multi-source data preprocessing; Perform point cloud denoising, point cloud matching, and point cloud thinning processing on 3D laser point cloud data; perform layer cleaning, coordinate conversion, and element simplification processing on plane CAD drawings; perform semantic mapping, semantic filtering, and individual element extraction processing on BIM data; perform image matching, image stitching, and 2D image generation processing on depth image data and holographic image data respectively; Step S3, multi-source data fusion modeling; The preprocessed three-dimensional laser point cloud data, plane CAD drawings, and BIM data are normalized and fused to form a multi-source data fusion model, and the texture information of the preprocessed depth image data and holographic image data is mapped to the multi-source data fusion model to obtain a three-dimensional real-scene physical space model of the indoor and outdoor fields.

2. The three-dimensional scene digital fusion method for intelligent test indoor and outdoor environment characteristics according to claim 1 is characterized in that: In the step S2, point cloud denoising, point cloud matching, and point cloud thinning are performed on the three-dimensional laser point cloud data; specifically, the steps include: Remove noise data and outliers due to occlusion scanning through point cloud denoising. Point cloud denoising methods include bilateral filtering, Gaussian filtering, binning denoising, KD-Tree, straight-through filtering, and random sampling consistency filtering. Through point cloud registration processing, point clouds in different coordinate systems are converted to the same coordinate system to obtain a complete point cloud model; wherein: first, the feature points of the target object are selected, including geometric feature points and layout feature points. After the initial estimation of the rigid body transformation, the rotation and translation matrices of two adjacent point clouds are solved through the feature points with the same name, thereby completing the point cloud data splicing; A voxel grid filter is used to create a voxel grid in the three-dimensional space where the point cloud is located, and the input point cloud data is divided into multiple tiny three-dimensional voxel grids. For each voxel in the grid, the centroid of all points in the voxel is used to replace the point in the voxel. The centroid points in all voxels constitute the filtered point cloud. The degree of downsampling is controlled by controlling the size of the voxel. The larger the voxel, the sparser the filtered point cloud.

3. The three-dimensional scene digital fusion method for intelligent test indoor and outdoor environment characteristics according to claim 2 is characterized in that: In the step S2, semantic mapping, semantic filtering, and individual element extraction are performed on the BIM data; Specifically include: According to the semantic information in the IFC model of BIM data standard format, a semantic mapping relationship is established between the spatial entity model and the semantic information is transmitted; Filter out unnecessary entities in the IFC model based on semantic mapping relationships, perform coordinate system conversion based on geometric differences, and convert IFC construction entity geometry to spatial data model geometry representation; By reading and filtering the semantic information in the IFC model, the individual elements of the spatial data model and their attribute information are extracted.

4. The three-dimensional scene digital fusion method for intelligent test indoor and outdoor environment characteristics according to claim 3 is characterized in that: In the step S2, image matching, image stitching, and two-dimensional image generation processing are performed on the depth image data and the holographic image data; wherein: Perform image data preprocessing and feature extraction on the depth image data and holographic image data, and match the extracted feature points after removing the mismatched points; The positions of the overlapping parts of two adjacent images are obtained, the mapping relationship of the registration functions of the two images is determined, the overlapping parts of the images are smoothed, and then the images are stitched.

5. The three-dimensional scene digital fusion method for intelligent test indoor and outdoor environment characteristics according to claim 4 is characterized in that: In the step S3, the pre-processed three-dimensional laser point cloud data, plane CAD drawing, and BIM data are normalized; specifically, the process includes: Perform coordinate conversion on pre-processed 3D laser point cloud data, 2D CAD drawings, and BIM data to unify them into the same coordinate reference; Among them, based on the characteristic target of the scanning system, the normal vector value and geometric center coordinates of the spherical characteristic target are used as the common control point to solve the coordinate conversion parameters of two adjacent point cloud data. The conversion process is realized by using the Bursa seven-parameter model of metrology. Among them, a three-dimensional model with actual geographical location is established by converting the geodetic coordinate control points in the known CGCS2000 coordinate system on the ground. After setting the coordinates of the common points, the conversion parameters are obtained based on the common points to complete the coordinate conversion.

6. The three-dimensional scene digital fusion method for intelligent test indoor and outdoor environment characteristics according to claim 5 is characterized in that: In the step S3, data fusion processing is performed on the normalized three-dimensional laser point cloud data, the plane CAD drawing, and the BIM data; specifically, the processing includes: For the 3D laser point cloud data, the inclined image is densely matched to obtain a 3D feature point set, and the point set is subjected to gross error elimination and simplification processing; the iterative closest point algorithm is used to align with the 3D laser point cloud and output the point cloud data; Given two point cloud models P and Q to be registered, point P in point cloud P i Find the point Q with the shortest Euclidean distance from the point cloud Q i , with P i and Q i As the corresponding point pair, the transformation matrix is ​​obtained. Through continuous iterative calculation and minimization of the error function, the optimal transformation matrix is ​​obtained to make the two point clouds coincide. Rotate and translate by matching points, using the least squares method as a criterion until the distance between points reaches a preset threshold; Further integrate CAD, BIM data and point cloud data, use machine learning to extract the basic structure from the CAD drawings, extract the monomer elements and corresponding semantic information from BIM and integrate the element point cloud, and perform orthographic projection on the integrated point cloud to obtain data organization in the two-dimensional direction; The derivative-free optimization algorithm is used to solve the four-degree-of-freedom transformation problem, that is, the translation and scaling of the x and y coordinates, to obtain no less than two common points for the fusion of CAD and the projected two-dimensional point cloud, and to perform four-parameter coordinate transformation.

7. The three-dimensional scene digital fusion method for intelligent test indoor and outdoor environment characteristics according to claim 6 is characterized in that: In the step S3, mapping the pre-processed depth image data and texture information of the holographic image data to the multi-source data fusion model specifically includes: The texture information of the depth image and the panoramic image is mapped to generate a three-dimensional real scene interior and exterior physical space model with color texture information, that is, the texture image of the object surface corresponding to the target physical three-dimensional model is obtained by using the depth image and the panoramic image; Among them, for the parts blocked by the modeled object, the texture information captured from different sides of the object is used to extract and map the texture information respectively, and then the extracted and mapped texture information is fused and registered with the model to generate a complete model.

Images