A method for generating a measurable bearing map based on multi-modal data

Abstract

The application discloses a kind of measurable azimuth chart generation methods based on multi-modal data.Real scene data orthographic image is two-dimensional expression general intuitive data.The application includes collecting the multi-modal data of survey area, azimuth coordinate conversion and constructs azimuth net, image visible information analysis and visible information integration;In azimuth coordinate system, the object azimuth information in the range of interest is orthographic projection, to azimuth coordinate system xy plane, obtain range information, and extract the resolution and image size of azimuth chart in xy plane, assign to geographic coordinate conversion parameter;Coordinate depth map is generated, azimuth chart texture mapping, azimuth chart edge joint processing and uniform color, header information update and band merging and space measurement.The application proposes the azimuth chart generation method of retaining original image resolving power, avoids the problems of texture distortion and resolution loss generated by other indirect results.

Description

Technical Field

[0001] This invention belongs to the field of surveying and mapping technology, specifically relating to a method for generating measurable azimuth maps based on multimodal data. Background Technology

[0002] Real-world data is currently the most sought-after form of 3D information representation, supporting the technological demands of various industries for its application. Real-world data is primarily represented in 3D models, orthophotos (DOM), terrain data (DEM), and surface models (DSM). Among these, orthophotos are a universal and intuitive form of 2D representation, characterized by simple data type, strong real-world perception, high reliability, and low distortion compared to other methods. Therefore, it has long served as the data foundation platform for collaborative operations in the field of intelligent surveying. However, in recent years, with the expansion of full-scene real-world applications, traditional quasi-orthophoto data based on the horizontal plane can no longer meet the needs of large-scale, high-elevation, and non-horizontal real-world scenarios. This has led to the evolution of various photographic generalizations, such as oblique photography, close-up photography, and superior-view photography, gradually focusing on breakthroughs in general photographic theory. These advancements have solved the problem of solving for the rigid high-precision orientation of the data space, resulting in a wide range of data sources to choose from in the production of real-world products. This has led to diverse needs in various fields of exploration, such as landslide identification, dangerous rock identification, geological disaster diagnosis, and crack detection. On the one hand, it requires detailed azimuth maps to help obtain a lot of field perception data. On the other hand, it also requires multi-directional low-dimensional image slices as input for intelligent processing to predict high-dimensional expected values, no longer limited by the limitations of traditional planar images.

[0003] To address these issues, the invention patent "Method for Generating Orthophotos Based on Panoramic Images" [Publication No.: CN112041892A] proposes generating high-precision orthophotos based on panoramic images and depth maps to obtain more detailed orthophoto data than traditional aerial photography. However, it lacks general applicability, is limited by the horizontal projection reference plane, and does not have spatial measurement capabilities. The invention patent "A Method for Generating Orthographic Images Based on Real-Scene 3D Models" [Publication No.: CN114627237A] achieves a projection image close to parallel to the ground by resampling based on a real-scene 3D model. It also uses additional image bands to store sampled values ​​from depth images. However, this method has two problems: firstly, the texture data obtained from sampling based on the real-scene 3D model suffers from severe texture loss, such as streaking and blurring; secondly, the generated orthophotos are based on the model's center point, making them unsuitable for areas with inconsistent spatial orientation. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, this invention provides a method for generating measurable azimuth maps based on multimodal data. It can combine appropriate multi-source data according to the engineering requirements to generate a two-dimensional projection azimuth map with the best orientation, while retaining the spatial reversible measurement function and realizing the association of real-scene three-dimensional data.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for generating measurable azimuth maps based on multimodal data, specifically including the following steps:

[0007] Step 1: Collect multimodal data for the test area;

[0008] Step 2: Azimuth coordinate transformation;

[0009] Step 3: Construct the orientation network;

[0010] Step 4: Image Visual Information Analysis;

[0011] Step 5: Visual information integration;

[0012] Step 6: In the azimuth coordinate system, perform orthographic projection on the object space information within the area of ​​interest and project it onto the xy plane of the azimuth coordinate system to obtain its range information. Extract the resolution and image size of the azimuth map in the xy plane and assign them to the geographic coordinate transformation parameters {A, B, C, D, E, F}.

[0013] Step 7: Generate a coordinate depth map. Perform depth detection on the azimuth network of the ground surface in the azimuth coordinate system, and record the z-coordinate value of the depth value in the azimuth coordinate system. The range of the depth map should be consistent with the range of the azimuth map.

[0014] Step 8: Orientation map texture mapping;

[0015] Step Nine: Edge finishing and color balancing of the orientation map;

[0016] Step 10: Header information update and band merging;

[0017] Step 11: Spatial Measurement;

[0018] Step 12: Output and verification of results.

[0019] Furthermore, in step one, the multimodal data includes aerial triangulation data, point cloud data, and model data.

[0020] Furthermore, step two specifically includes:

[0021] 2.1 Determination of Azimuth Coordinate System Information

[0022] The azimuth coordinate system information is determined through user specification and automatic constraint solving; in the automatic constraint solving process, the object surface normal vector is first used as the basis for determining the azimuth coordinate system information. Optimize and fit the best direction vector It is the Z-axis of the azimuth coordinate system; then, with the normal vector as... The plane along the direction vector Move the object beyond its surface area without being obscured by other features. The origin of the azimuth coordinate system is the centroid O' of the surface information projected onto the plane. Use the centroid O' and... The orientation and position of the xy plane of the azimuth coordinate system are defined together. The orientation vectors in the remaining planes are determined by the orientation of the object scene in the xy plane of the azimuth coordinate system, and the direction with the maximum distribution characteristic is selected as the horizontal axis X direction. This fixes the unit vector expression of the three coordinate axes (O`-X`Y`Z`) of the azimuth coordinate system in the object coordinate system.

[0023] 2.2 Solving and representing spatial transformation information between the azimuth coordinate system and the object coordinate system

[0024] Based on the origin of the azimuth coordinate system (O`-X`Y`Z`) and the unit vectors of the coordinate axes in the object coordinate system The spatial transformation parameter [R|t] from the object coordinate system to the azimuth coordinate system was established.

[0025] Given that the origin of the azimuth coordinate system has coordinates (X0, Y0, Z0) in the object coordinate system, the unit vectors of the three axes of the azimuth coordinate system in the object coordinate system are expressed as follows: The transformation parameter [R|t] is then:

[0026]

[0027]

[0028] Furthermore, step three specifically includes:

[0029] 3.1 When using point cloud information as object surface sampling information, select the object point cloud information within the projection range of interest, and construct a 2.5-dimensional object surface model with the Z-axis direction and the orientation map projection direction to ensure good geometric expression accuracy in the xy plane. The constructed 2.5-dimensional model is the orientation network data.

[0030] 3.2 When using model data to express information about the object surface, the model data is truncated according to the projection range of interest and used as the azimuth net data for the calculation.

[0031] Furthermore, step four specifically includes:

[0032] 4.1 The set of visual images corresponding to the picking orientation network {I j Remove images that are obscured by information from other objects;

[0033] 4.2 Based on the azimuth mesh surface element normal vector and occlusion detection, obtain the angle value between the visible beam and its normal vector, and retain the images with the k smallest angle values ​​as the candidate visible image set of its surface element.

[0034] Furthermore, step five specifically includes:

[0035] By applying the visual image consistency constraint of the neighboring surface elements, each surface element m is redefined. i Assign values ​​to best visual image The means of visual information integration is to establish its discrete energy function:

[0036]

[0037] in Used to describe surface element m i Select Image I j The cost, q∈N(p) is the cost of surface element m. p The neighborhood index value, Used to express the cost of choosing different image indices for adjacent facets.

[0038] Furthermore, step eight specifically includes:

[0039] Using high-precision aerial triangulation data, surface elements, and optimal visual imagery, the best texture information is mapped onto the azimuth map. The corresponding points in the object space coordinate system are (X, Y, Z), the corresponding points in the azimuth coordinate system are (x, y, z), and the corresponding points in the azimuth map pixel coordinate system are (u, v). The spatial mapping relationship is as follows:

[0040]

[0041]

[0042] Surface m i Best texture images The projection matrix is Then its original pixel values ​​are mapped to the azimuth map I` i The process of calculating the pixel value is as follows:

[0043]

[0044] Furthermore, step nine specifically includes:

[0045] 9.1 First, feature pixels are detected in the overlapping area of ​​the edge to perform corresponding point matching. Then, the geometric correction of each corresponding point interval in the edge is interpolated by the coordinate difference of the corresponding points in the edge. This generates the geometric correction of the closed boundary of the surface element. The geometric correction of the interior is solved by polynomial fitting of the boundary geometric correction condition constraint.

[0046] 9.2 Using the color difference of the overlapping area as an observation, the zero-order boundary information is used to construct a polynomial fitting solution to solve the color correction amount inside the surface element, thus completing the color uniform processing between different adjacent surface elements.

[0047] Furthermore, step eleven specifically includes:

[0048] Using the generated azimuth map, depth map, geographic coordinate transformation parameters {A, B, C, D, E, F}, and spatial transformation parameter [R|t], the image coordinate system of the azimuth map is transformed to the xy plane of the azimuth coordinate system, and then the inverse operation is performed to the object coordinate system to obtain the real spatial coordinates of the scene. Using the obtained geographic coordinate transformation parameters {A, B, C, D, E, F}, the planar coordinates and depth values ​​in the xy plane of the azimuth coordinate system are obtained. Then, the coordinate information in the original object coordinate system is solved by reversing the process of formula 5.

[0049] The beneficial effects of this invention are:

[0050] 1) This invention proposes a method for generating azimuth maps that preserves the resolution of the original images, avoiding the problems of texture distortion and resolution loss caused by other indirect results, truly reflecting the appearance characteristics of geographic entities, and accurately capturing the surface morphology of ground features.

[0051] 2) This invention proposes a general orientation map control mechanism, which can select local scenes of interest in the geographic reality according to application needs, and generate separate maps in a specified spatial plane to highlight important observation information. It also provides efficient and fast measurable orientation map reality data for diverse applications, reducing the cost of reality collaborative services.

[0052] 3) This invention patent is designed for wide-area engineering needs, takes into account the advantages of multimodal data, and proposes a complete set of orientation map generation theories and production processes. It solves the problems of geometric and texture distortion, and is a supplement to the expression of real scene data. It can provide detailed records for survey results and provide convenient supporting data for intelligent survey technology. Attached Figure Description

[0053] Figure 1 This is a flowchart illustrating the process of generating a measurable azimuth map based on multimodal data according to the present invention.

[0054] Figure 2 This is a schematic diagram illustrating the principle of a directional map.

[0055] Figure 3 This is a crop of the azimuth map field of view.

[0056] Figure 4 Geometric difference observations for different neighborhood elements are extracted;

[0057] Figure 5 A schematic diagram of polynomial fitting solution for closed boundary constraints;

[0058] Figure 6 The original close-fit model;

[0059] Figure 7 It is a high-precision measurable azimuth map. Detailed Implementation

[0060] The present invention will now be described in detail with reference to specific embodiments.

[0061] This invention addresses the demand for high-precision, spatially measurable 2D imagery in modern surveying and mapping fields. It proposes a method for generating measurable azimuth maps based on multimodal data. This method can combine appropriate multi-source data according to the engineering requirements to generate a 2D projected azimuth map with optimal orientation, while retaining spatial reversible measurement capabilities and enabling the correlation of real-world 3D data. This invention retains the high resolution of the original image, accurately reflecting the information of geographic entities and facilitating the capture of imperfections in surface structural details. Furthermore, the projection direction and position of the azimuth map can be controlled by parameters, allowing for more general operation and facilitating focused analysis of local scenes during application. Secondly, the azimuth map generation process considers the scene range, allowing users to select surface real-world information within the area of ​​interest to generate high-precision 2D image snapshots. In addition, the proposed method can achieve real-time rendering technology based on graphics cards, facilitating the sharing of multi-professional collaborative application services at low cost.

[0062] like Figure 1 As shown, the present invention specifically includes the following steps:

[0063] Step 1: Collect multimodal data of the survey area, including aerial triangulation data, point cloud data, and model data;

[0064] Collect multimodal data of the survey area, including aerial triangulation data (including original images, attitude data, and sensor parameters), point cloud data, and model data (Mesh, BIM). Either point cloud data or model data can be selected to provide object surface information for the orientation map; aerial triangulation data provides high-resolution texture mapping, while point cloud data and model data provide geographic reality object surface data.

[0065] Step 2: Azimuth coordinate transformation;

[0066] Because traditional geographic scenes are large in scope, have a wide range of geographic coordinate values, and the orthophotos of local scene information cannot accurately describe and reflect complete and real information, for areas that require detailed mapping, azimuth coordinate transformation is required so that the azimuth mapping direction is approximately collimated and perpendicular to the surface of the object area.

[0067] Step two specifically includes:

[0068] 2.1 Determination of Azimuth Coordinate System Information

[0069] The orientation coordinate system information is determined based on the salient direction and display distance of the object's surface information, through user specification and automatic constraint solving. In the automatic constraint solving process, the object's surface normal vector is first used as the basis for the determination. Optimize and fit the best direction vector It is the Z-axis of the azimuth coordinate system; then, with the normal vector as... The plane along the direction vector Move the object beyond its surface area without being obscured by other features. The origin of the azimuth coordinate system is the centroid O' of the surface information projected onto the plane. Use the centroid O' and... The orientation and position of the xy plane of the azimuth coordinate system are defined together. The orientation vectors in the remaining planes are determined by the orientation of the object scene in the xy plane of the azimuth coordinate system, and the direction with the maximum distribution characteristic is selected as the horizontal axis X direction. This fixes the unit vector expression of the three coordinate axes (O`-X`Y`Z`) of the azimuth coordinate system in the object coordinate system.

[0070] 2.2 Solving and representing spatial transformation information between the azimuth coordinate system and the object coordinate system

[0071] Based on the origin of the azimuth coordinate system (O`-X`Y`Z`) and the unit vectors of the coordinate axes in the object coordinate system The spatial transformation parameter [R|t] from the object coordinate system to the azimuth coordinate system was established.

[0072] like Figure 2 As shown, the coordinates of the origin of the azimuth coordinate system in the object coordinate system are (X0, Y0, Z0). The unit vectors of the three axes of the azimuth coordinate system in the object coordinate system are expressed as follows: The transformation parameter [R|t] is then:

[0073]

[0074]

[0075] It contains the rotation matrix R 3×3 Translation matrix t 3×1Using a rotation matrix instead of traditional rotation angle elements avoids problems of unclear or ambiguous definitions.

[0076] Step 3: Construct a azimuth network

[0077] An azimuth mesh is a surface mesh representation of the object of interest (IOI) in an azimuth coordinate system, with orientation, and it is only constructed within the IIO projection range. The IIOI is constructed by selecting the IIOI within the range and considering azimuth constraints, and is used for subsequent mapping of the surface information azimuth map. Simultaneously, azimuth coordinate transformation information is combined to obtain the range information in the azimuth coordinate system.

[0078] Step three specifically includes:

[0079] 3.1 When using point cloud information as object surface sampling information, select the object point cloud information within the projection range of interest, and construct a 2.5-dimensional object surface model with the Z-axis direction and the orientation map projection direction to ensure good geometric expression accuracy in the xy plane. The constructed 2.5-dimensional model is the orientation network data.

[0080] 3.2 When using model data to express information about the object surface, the model data is truncated according to the projection range of interest and used as the azimuth net data for the calculation.

[0081] like Figure 3 As shown, the area of ​​interest is constructed using the near and far planes of the azimuth coordinate system. The object space information between the near and far planes represents the real scene requiring azimuth projection. Then, based on different representations of the geographic real scene object surface data, azimuth network data in the azimuth coordinate system is constructed. For example, if it is point cloud data, the azimuth network data is 2.5D data oriented towards the orthophoto projection direction (the positive z-axis direction of the azimuth coordinate system); if it is grid model data, the required object surface data can be directly extracted as azimuth network data.

[0082] Step 4: Image Visual Information Analysis

[0083] By constructing the orientation mesh and image data and image pose data, the visual information of the surface elements in each orientation mesh is calculated. The visual information includes the image index, the projection coordinates in the image, etc.; that is, the surface element data in each orientation mesh, and the visual texture information is found in the original image.

[0084] Step four specifically includes:

[0085] 4.1 The set of visual images corresponding to the picking orientation network {I j Remove images that are obscured by information from other objects;

[0086] 4.2 Based on the azimuth mesh surface element normal vector and occlusion detection, obtain the angle value between the visible beam and its normal vector, and retain the images with the k smallest angle values ​​as the candidate visible image set of its surface element.

[0087] First, select the set of visible images within the space of interest {I}. j}, where j is the image index value, represented using dummy notation; then for each azimuth grid cell m i Assign value to visual image set And for the set of visual images The image beam is sorted based on the angle between its normal vector and the surface element. Sort by size in ascending order, retaining the top K candidate images. Let i be the index of the face element, represented using dummy notation. The indices i and j of both the face element and the image can be natural numbers in the range [0, ..., the number of elements in the corresponding set).

[0088] Step 5: Visual Information Integration

[0089] The visual information of each facet in the orientation grid is adjusted and optimized to achieve consistency of the best candidate texture source image of local adjacent facets as much as possible, so as to ensure that the texture and geometric distortion in the subsequent orientation map is minimized.

[0090] Step five is as follows:

[0091] By applying the visual image consistency constraint of the neighboring surface elements, each surface element m is redefined. i Assign values ​​to best visual image The means of visual information integration is to establish its discrete energy function:

[0092]

[0093] in Used to describe surface element m i Select Image I j The cost, q∈N(p) is the cost of surface element m. p The neighborhood index value, Used to express the cost of choosing different image indices for adjacent facets.

[0094] To maximize the selection of adjacent facets from the same texture data source, ensuring minimal human intervention, the optimal texture image data for each facet in the orientation mesh is ultimately determined.

[0095] Step 6: In the azimuth coordinate system, perform orthographic projection on the object space information within the area of ​​interest and project it onto the xy plane of the azimuth coordinate system to obtain its range information. Extract the resolution and image size of the azimuth map in the xy plane and assign them to the geographic coordinate transformation parameters {A, B, C, D, E, F}.

[0096] The orthophoto plane range of the azimuth map is determined within the xy plane of the azimuth coordinate system. The maximum and minimum values ​​in both the horizontal and vertical directions are obtained, and the planar resolution of the azimuth map is set to be no less than the ground resolution of the original image. This resolution is used to subsequently fill in the geographic header information of the azimuth map, i.e., the geographic coordinate transformation parameters {A, B, C, D, E, F}. This determines the transformation parameters from the pixel coordinate system of the azimuth map to the xy plane of the azimuth coordinate system.

[0097] Step 7: Generate a coordinate depth map. Perform depth detection on the azimuth network of the ground surface in the azimuth coordinate system. Record the z-coordinate value of the depth value in the azimuth coordinate system. The range of the depth map is consistent with the range of the azimuth map. By combining the transformation parameters of the azimuth coordinate system and the object coordinate system, the reverse spatial measurement function of the azimuth map can be realized.

[0098] Depth probing is performed on all object points p within the region of interest, and each pixel is saved to a floating-point image of the same size as the orientation map, such as... Figure 2 As shown; a tree-based spatial retrieval and spatial collision method was used in the depth detection process. A ray was constructed from each pixel in the orientation map, starting from the pixel and moving in the opposite direction of the z-axis of the orientation coordinate system. The z-value closest to the xy-plane of the orientation coordinate system was selected as the depth value.

[0099] Step 8: Orientation Map Texture Mapping

[0100] By utilizing the results of visual information integration, spatial digital differential correction is performed on the orthophoto texture of the fill element in the azimuth map in the optimal texture image data.

[0101] Step eight is as follows:

[0102] Using high-precision aerial triangulation data, surface elements, and optimal visual imagery, the best texture information is mapped onto the azimuth map. The corresponding points in the object space coordinate system are (X, Y, Z), the corresponding points in the azimuth coordinate system are (x, y, z), and the corresponding points in the azimuth map pixel coordinate system are (u, v). The spatial mapping relationship is as follows:

[0103]

[0104]

[0105] Surface m i Best texture images The projection matrix is Then its original pixel values ​​are mapped to the azimuth map I` i The process of calculating the pixel value is as follows:

[0106]

[0107] Step Nine: Edge Joining and Color Mixing of the Orientation Map

[0108] Since the orientation map texture information is selected from the original image, the mapping process depends on the accuracy of the object surface. Therefore, there are obvious geometric misalignments and texture color differences at the seams of different texture sources. Thus, additional seam processing and color balancing are required. A correction energy function is constructed based on the geometric and color differences at the seams of adjacent surface elements according to their different texture information.

[0109] Step nine specifically includes:

[0110] 9.1 In the edge processing of the orientation map, for the geometric difference of the edge, firstly, the feature pixels in the overlapping area of ​​the edge are detected and the corresponding points are matched. Then, the geometric correction amount of each corresponding point interval in the edge is interpolated by the coordinate difference of the corresponding points in the edge, thereby generating the geometric correction amount of the closed boundary of the surface element. The geometric correction amount inside is solved by polynomial fitting of the boundary geometric correction condition constraint.

[0111] 9.2 In the edge uniform color processing of the orientation map, the geometric error has been eliminated through 9.1 for color difference correction. The color difference of the overlapping area is used as the observation. Similarly, the zero-order boundary information is used to construct a polynomial fitting to solve the color correction amount inside the surface cell, and the uniform color processing between different adjacent surface cells is completed.

[0112] like Figure 4 As shown, adjacent face elements m1 and m2 have potential pairs of corresponding points {u} in the overlapping region. i →v i}, also using the dummy notation method, adjacent pairs of points with the same name have a region of lines with the same name {u i,i+1 →v i,i+1 If geometric distortion exists, corresponding point pairs will have coordinate differences in the orientation map. The coordinate differences of all corresponding point pairs are linearly interpolated to the overlapping area between each surface element according to the linear direction. Let the corresponding point pairs {u} i →v i After linear matching, the coordinate difference with the mean of the corresponding coordinates is ε. The coordinate difference at the boundary of each surface element can then be calculated. The correction for the overlapping region is ε. Taking the geometric distortion ε of the corresponding points as a known observation, each surface element has an observation overlap boundary with a closed Ω region. and its correction amount like Figure 5 As shown. Therefore, the geometric correction f(u) of the interior m(u) of the surface element becomes a polynomial fit with known boundary conditions:

[0113]

[0114] When geometric correction is performed on the surface texture band, the corresponding depth map band is also geometrically corrected simultaneously.

[0115] The texture homogenization process between neighboring pixels is similar to geometric correction, except that the observation becomes the pixel difference in the overlapping area. After correction, the true value of the overlapping area is the mean value of the corresponding pixel, and the boundary constraint is the difference between the pixel value of the corresponding point and the mean value.

[0116] Step 10: Header information update and band merging

[0117] To ensure spatially reversible measurements, spatial coordinate transformation information [R|t] and geographic transformation parameters {A, B, C, D, E, F} in azimuth coordinates need to be written into the azimuth map header information. At the same time, the azimuth map and depth map bands are combined into a single image file.

[0118] Step 11: Spatial Measurement

[0119] Using the generated azimuth map, depth map, geographic coordinate transformation parameters {A, B, C, D, E, F}, and spatial transformation parameter [R|t], the image coordinate system of the azimuth map is transformed to the xy plane of the azimuth coordinate system, and then the inverse operation is performed to the object coordinate system to obtain the real spatial coordinates of the scene. Using the obtained geographic coordinate transformation parameters {A, B, C, D, E, F}, the planar coordinates and depth values ​​in the xy plane of the azimuth coordinate system are obtained. Then, the coordinate information in the original object coordinate system is solved by reversing the process of formula 5.

[0120] Step 12: Output and verification of results.

[0121] Application Example: In high-precision building BIM information acquisition applications, the method of this patented invention is used to generate orientation maps from a 5mm ground resolution 3D model obtained through close-up photography. This measurable orientation map processing significantly reduces the computational burden on 3D model data during data transfer, while improving its measurability, spatial extraction, and spatial analysis capabilities. It also enables the acquisition of building details and the identification of defects, thus demonstrating and highlighting the advantages of this patented invention. Figure 6 , 7 As shown, the original 3D patch model presents significant obstacles to information exchange and processing due to the massive amounts of data and irrelevant data in a large number of non-interest areas and different exploration operations. By selecting and generating location maps, lightweight geospatial reconstruction can be achieved, highlighting key areas of interest, and it can be compatible with cross-domain data processing platforms.

[0122] The content of this invention is not limited to the embodiments listed. Any equivalent modifications made by those skilled in the art to the technical solutions of this invention by reading this specification are covered by the claims of this invention.

Claims

1. A method for generating measurable azimuth maps based on multimodal data, characterized in that: Specifically, the steps include the following: Step 1: Collect multimodal data for the test area; Step 2: Azimuth coordinate transformation; Step 3: Construct the orientation network; Step 4: Image Visual Information Analysis; Step 5: Visual information integration; Step Six: In the azimuth coordinate system, perform orthographic projection on the object space information within the area of ​​interest and project it onto the azimuth coordinate system. Within the plane, obtain its range information and extract the azimuth map. The in-plane resolution and image size are assigned to the geographic coordinate transformation parameters {A, B, C, D, E, F}; Step 7: Generate a coordinate depth map. Perform depth detection on the azimuth network of the ground surface in the azimuth coordinate system, and record the z-coordinate value of the depth value in the azimuth coordinate system. The range of the depth map should be consistent with the range of the azimuth map. Step 8: Orientation map texture mapping; Step Nine: Edge finishing and color balancing of the orientation map; Step 10: Header information update and band merging; Step 11: Spatial Measurement; Step 12: Output and verification of results; Step two specifically includes: 2.1 Determination of Azimuth Coordinate System Information The azimuth coordinate system information is determined through user specification and automatic constraint solving; in the automatic constraint solving process, the object surface normal vector is first used as the basis for determining the azimuth coordinate system information. Optimize and fit the best direction vector It is the Z-axis of the azimuth coordinate system; then, with the normal vector as... The plane along the direction vector The object can be moved beyond the surface of the object of interest without being obscured by other terrain features. The origin of the azimuth coordinate system is the centroid of the projection of the surface information of the object of interest onto the plane. Use center of gravity and They jointly defined the azimuth coordinate system. The orientation and position of the plane, and the remaining in-plane orientation vectors through the object-side scene in the orientation coordinates. With the orientation of the plane determined, its direction of maximum distribution characteristics is selected as the horizontal axis (X-direction); thus, the azimuth coordinate system is fixed. Unit vector representation of the three coordinate axes in the object coordinate system ; 2.2 Solving and representing spatial transformation information between the azimuth coordinate system and the object coordinate system According to the azimuth coordinate system The origin and coordinate axes in the object coordinate system are unit vectors The spatial transformation parameter [R|t] from the object coordinate system to the azimuth coordinate system was established; Given that the coordinates of the origin of the azimuth coordinate system in the object coordinate system are (X0, Y0, Z0), the unit vectors of the three axes of the azimuth coordinate system in the object coordinate system are expressed as follows: Then the transformation parameter [R|t] is: (Official 1) (Official 2); Step four specifically includes: 4.1 The set of visual images corresponding to the picking orientation network Remove images that are obscured by information from other objects; 4.2 Based on the azimuth mesh element normal vector and occlusion detection, obtain the angle between the visible beam and its normal vector, and retain the maximum number of forward beams. The image with the smallest intersection angle value is the set of candidate visible images for its surface elements; Step five specifically involves: By applying the visual image consistency constraint of the neighboring surface elements, each surface element m is redefined. i Assigning the best visual image The means of visual information integration is to establish its discrete energy function: (Official 3) in Used to describe face elements Select Image The cost, For face element The neighborhood index value, Used to express the cost of selecting different image indices for adjacent face cells; Step eight specifically involves: Using high-precision aerial triangulation data, surface elements, and optimal visual imagery, the best texture information is mapped onto the azimuth map; where the corresponding points in the object space coordinate system are (X, Y, Z), and the corresponding points in the azimuth coordinate system are... In the orientation map pixel coordinate system, the corresponding points are Then its spatial mapping relationship is: Mianyuan Best texture images The projection matrix is Then its original pixel values ​​are mapped to the azimuth map. The process of calculating the pixel value is as follows: (Official 6).

2. The method for generating a measurable azimuth map based on multimodal data according to claim 1, characterized in that: In step one, the multimodal data includes aerial triangulation data, point cloud data, and model data.

3. The method for generating a measurable azimuth map based on multimodal data according to claim 2, characterized in that: Step three specifically includes: 3.1 When using point cloud information as object surface sampling information, select the object point cloud information within the projection range of interest, and construct a 2.5-dimensional object surface model with the Z-axis direction and the orientation map projection direction to ensure good geometric expression accuracy in the xy plane. The constructed 2.5-dimensional model is the orientation network data. 3.2 When using model data to express information about the object surface, the model data is truncated according to the projection range of interest and used as the azimuth net data for the calculation.

4. The method for generating a measurable azimuth map based on multimodal data according to claim 3, characterized in that: Step nine specifically includes: 9.1 First, feature pixels are detected in the overlapping area of ​​the edge to perform corresponding point matching. Then, the geometric correction of each corresponding point interval in the edge is interpolated by the coordinate difference of the corresponding points in the edge. This generates the geometric correction of the closed boundary of the surface element. The geometric correction of the interior is solved by polynomial fitting of the boundary geometric correction condition constraint. 9.2 Using the color difference of the overlapping area as an observation, the zero-order boundary information is used to construct a polynomial fitting solution to solve the color correction amount inside the surface element, thus completing the color uniform processing between different adjacent surface elements.

5. The method for generating a measurable azimuth map based on multimodal data according to claim 4, characterized in that: Step eleven specifically involves: Using the generated azimuth map, depth map, geographic coordinate transformation parameters {A, B, C, D, E, F}, and spatial transformation parameter [R|t], the image coordinate system of the azimuth map is transformed to the xy plane of the azimuth coordinate system, and then the inverse operation is performed to the object coordinate system to obtain the real spatial coordinates of the scene. Using the obtained geographic coordinate transformation parameters {A, B, C, D, E, F}, the planar coordinates and depth values ​​in the xy plane of the azimuth coordinate system are obtained. Then, the coordinate information in the original object coordinate system is solved by reversing the process of formula 5.

Images