A road scene spatial rapid survey system combining RTK and image reconstruction

Abstract

The present invention discloses a road scene space rapid survey system combining RTK and image reconstruction, relates to the technical field of road scene space survey, and is used to solve the problem of poor road mapping splicing effect; the present invention collects RTK tracks in road inspections and synchronizes them with image frames, extracts continuous control points after removing abnormal points to construct a road topology skeleton, divides the space into multiple mapping units based on control points and direction vectors, sets redundant overlapping areas, generates a mapping task package with structural continuity, extracts road structure areas in the image through semantic segmentation and edge detection, constructs a structure mask and projects it into three-dimensional space, guides point cloud density enhancement and reconstruction optimization, thereby further identifying block boundary structure anchor points, completing local rigid body registration, and constructing a global error minimization model to achieve posture consistency and spatial continuous splicing of all blocks, thereby improving the survey efficiency of road scene space.

Description

Technical Field

[0001] The present invention relates to the technical field of road scene space survey, and more particularly to a road scene space rapid survey system combining RTK and image reconstruction. Background Art

[0002] Amid the rapid development of smart transportation, autonomous driving, high-precision mapping, and digital urban management, the acquisition and representation of spatial information about road scenes has become a fundamental and critical component. As the most important structural network in urban space, roads' spatial topology, surface morphology, and element distribution directly impact path planning, traffic organization, and the overall performance of urban perception systems.

[0003] Deficiencies in existing technologies: Spatial mapping and surveying of road scenes mainly rely on a single sensor (such as GNSS or visual SLAM) for path positioning and three-dimensional reconstruction. Positioning methods based on images or SLAM are severely affected by changes in lighting, texture, and perspective, and lack global consistency, resulting in spatial drift and splicing misalignment in the reconstruction results. Existing reconstruction methods mostly use uniform sampling and blind point cloud generation strategies, and fail to identify and optimize structural elements such as lane lines and road edges, resulting in fuzzy road geometry expression and incomplete semantic structure. The overall mapping process often lacks a clear spatial organizational structure, lacks control point drive and main axis constraint mechanisms, and is difficult to achieve modular, efficient block division and parallel mapping, resulting in low spatial splicing efficiency in road scenes. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, the following solution is proposed to solve the problem of poor road mapping splicing effect in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A road scene space rapid survey system combining RTK and image reconstruction, including a road trajectory acquisition module, a road mapping and analysis module, a structure enhancement module, and a road survey module, wherein each module is connected via signals;

[0007] The road trajectory acquisition module is used to collect road inspection trajectories and synchronize image frames with trajectory points, eliminate abnormal jump points, and extract continuous control points. It then establishes a road topology skeleton based on the control points and direction vectors.

[0008] The road mapping analysis module is used to divide the road space into continuous mapping units based on control points and direction information, set the spatial range and main axis direction and region attribution mapping for the mapping units, and construct independent mapping task packages to map overlapping areas;

[0009] The structure enhancement module is used to extract the structural semantic areas and edge information in the image, construct a structure mask and map it to the three-dimensional space, and mark it as a structure enhancement point;

[0010] The road survey module is used to identify areas where tile boundary structures overlap and establish anchor points, complete local rigid body registration, build a global error minimization model, uniformly optimize the pose and position of all tiles, and conduct overall road space surveys.

[0011] In a preferred embodiment, for collecting multi-source data in the network and performing graph structured modeling, the specific steps are as follows: for collecting road inspection trajectories and synchronizing image frames and trajectory points, the specific steps are as follows:

[0012] The three-dimensional spatial positioning data collected in the road scene is used as RTK trajectory data. The RTK trajectory data corresponds to the real-time spatial position of each frame of image acquisition in the road scene. The acquisition object is the installation location of the image acquisition terminal, including vehicle-mounted cameras, mobile work platforms, handheld devices, and trajectory scanning devices. The acquisition path is all spatial points passed along the road topology path during the road inspection process.

[0013] Coverage areas include motor vehicle lanes, sidewalks, bicycle lanes, curbs, lane line intersections; turning corners, intersections, ramps, intersections, underpasses, culverts, and overpasses;

[0014] Perform RTK track point collection as a track point sequence. The RTK track point includes the 3D positioning coordinates corresponding to the collection position and the corresponding timestamp;

[0015] The collected image frame data includes the image frame and the image frame collection time;

[0016] For each image frame, find the RTK track point closest to the timestamp and perform time binding between the image and the track point;

[0017] Set the time matching tolerance threshold. If the absolute value of the difference between the image frame acquisition time and the timestamp of the closest RTK track point is less than or equal to the matching tolerance threshold, the frame image is bound to an RTK track point and the pairing data is retained.

[0018] In a preferred embodiment, abnormal jump points are removed, and continuous control points are extracted. A road topology skeleton is established based on the control points and direction vectors. The specific steps include:

[0019] Perform RTK trajectory smoothing and pseudo-jump point removal, and convert three consecutive trajectory points into , the direction vector is calculated as ,in, It is from the point Pointing Point The direction vector, It is from the point Pointing Point direction vector;

[0020] Tectonic direction change angle: ,in, represents the Euclidean norm of a vector; represents the turning angle of the trajectory at point i;

[0021] Calculate the spatial distance between adjacent trajectory points : ,like , there is a violent space jump, is the distance threshold;

[0022] The pseudo jump point determination rules are set as follows: If the trajectory point satisfies both: and , it is determined to be an abnormal jump point and recorded as a pseudo jump point; is the set corner threshold;

[0023] If the pseudo jump point is only an isolated jump, it is directly eliminated. If the trajectory points before and after the jump point are continuous, linear interpolation or spline reconstruction is used to fill the gap;

[0024] Summarize the interpolation points to form the final set of control points: ,in, is a three-dimensional Euclidean real space, is the kth control point; n is the total number of extracted control points;

[0025] Each control point from the trajectory point set or interpolation point is used as a space partition anchor point and skeleton graph node.

[0026] In a preferred embodiment, each control point from the trajectory point set or interpolation point is used as a space partition anchor point and a skeleton graph node. The specific steps are as follows:

[0027] Construct a direction vector for any pair of consecutive k-th and k+1-th control points , C is the control point set, and the unit direction vector is defined as: ,in, , represents the main axis direction of the k-th space segment;

[0028] Construct a set of posture vectors and a set of direction vectors: , each direction vector and corresponding control points Bind to form a posture reference pair: , the pose information is used as the orientation reference for the partition, and as the main vector for initializing the camera pose estimation for image reconstruction, and as the pose constraint input during the tile stitching process;

[0029] Construct the road skeleton graph structure and define the road skeleton graph as: : ;

[0030] A set of directed edges: ;

[0031] Each edge records the following attribute tuple: ,in, is the side length, which represents the spatial distance between the control point pairs; is the direction angle, which represents the turning angle between road segments.

[0032] In a preferred embodiment, the method for dividing the road space into continuous mapping units based on control points and direction information, setting the spatial range and main axis direction and region attribution mapping for the mapping units, and constructing independent mapping task packages for mapping overlapping areas includes the following steps:

[0033] Construct a spatial principal axis reference set based on the output control point sequence and the corresponding direction vector set. Each pair of adjacent control points is considered as a segment interval, and the direction vector represents the basic segment principal axis direction.

[0034] After obtaining the main axis reference of each segment, for the kth segment, the spatial area center of the corresponding local mapping unit is recorded as: ;

[0035] The boundary region of the mapping unit is constructed as follows: , where Box represents the spatial enclosing area defined by center and direction parameters; is the main direction vector; its length is L; its vertical direction is , the width W is set by the environmental constraints; the height direction is set to H;

[0036] After the structure is constructed, each mapping unit area serves as a spatial carrier for independent image reconstruction, and the image data, point cloud data, and texture mapping processing are all confined to the mapping unit.

[0037] The overlapping length of the mapping unit is , do equidistant expansion on the original mapping unit length L, and determine the effective area of the mapping unit as: ;

[0038] During the image frame attribution process, if the center point of the image frame falls into the valid area of the adjacent mapping unit, the image frame will be assigned to two mapping units for redundant mapping;

[0039] After completing the division of spatial mapping units, the spatial parameters, image data, direction constraints and reconstruction configuration of each mapping unit are structured and packaged to form an independent mapping task package;

[0040] Construct a task unit for each mapping unit: ,in, is the set of image frames belonging to the mapping unit; is the set of RTK trajectory points corresponding to the image frame; is the main direction vector; is the center position of the mapping unit.

[0041] In a preferred embodiment, for extracting structural semantic regions and edge information in an image, constructing a structure mask and mapping it to a three-dimensional space, and marking it as a structure enhancement point, the specific steps are as follows:

[0042] Extracting road semantic and geometric structural regions from image frames assigned to local mapping units, and using a combined semantic segmentation and structural edge extraction mechanism to identify typical road structure regions in each image frame.

[0043] Perform pixel-level structural classification on the image to obtain a semantic label map;

[0044] At the same time, the Canny operator is used to perform gradient edge detection to obtain the image edge structure map;

[0045] Combine the semantic map with the edge map to construct a multi-structure joint mask;

[0046] After image feature extraction, the camera pose and spatial orientation information obtained by RTK registration are used to perform three-dimensional projection of the high-confidence structure area in each image to obtain a set of structural guide points, which are used as structural enhancement points for structural enhancement.

[0047] In a preferred embodiment, after image feature extraction, the camera pose and spatial orientation information obtained by RTK registration are used to perform three-dimensional projection of the high-confidence structure area in each image to obtain a set of structural guide points, which are used as structural enhancement points for structural enhancement. The specific steps are as follows:

[0048] For each image frame, the pixel-level mask is projected into a set of three-dimensional space points using the camera's internal and external parameters.

[0049] Convert the two-dimensional image coordinates into normalized camera coordinate system vectors, and map them into three-dimensional space projection directions through camera extrinsics;

[0050] Combine the RTK trajectory points corresponding to the image frame to generate the line of sight space points, and record the 3D points after the projection of all significant structures as a 3D point set;

[0051] In the point cloud generation stage, an adaptive sampling mechanism is used for the significant structure area to enhance the spatial position of the structure point area. Use the reconstruction pipeline as a sampling reference constraint to construct a spatial density adjustment factor : ,in, Indicates the candidate position for reconstruction of the current point cloud, Enhanced impact range threshold for structures; is the response slope control factor, q represents the spatial position coordinates of the structural reinforcement point area;

[0052] After the initial point cloud generation, the structural region undergoes geometric optimization and texture refinement. For each point cloud within a mapping unit, a local neighborhood subset close to the structural guide point is extracted, and geometric boundary repair and texture mapping optimization operations are performed. Based on a local optimization strategy for reprojection error, the optimization goal is to minimize the multi-view reprojection error of the structural region.

[0053] In a preferred embodiment, the steps for identifying the overlapping areas of the tile boundary structures and establishing anchor points to complete local rigid body registration are as follows:

[0054] Between each pair of adjacent tiles, the structural salient points of the boundary area are extracted, including lane corners, roadside boundaries, sudden corners or traffic sign outlines, to form splicing anchor points, and the boundary structural feature point sets of the kth and k+1th tiles are combined. Respectively expressed as: ,in, 、 Respectively represent the three-dimensional coordinate vectors of the i-th and j-th structural feature points in the k-th block;

[0055] Determine whether there are significant structural coincidence point pairs by nearest neighbor matching: ;

[0056] And form a candidate matching set: ,in, is the set of candidate matching structure point pairs between blocks k and k+1;

[0057] If satisfied , then the two blocks have good spatial connectivity at the structural boundary, and further registration is performed, where is the minimum matching point pair number threshold;

[0058] Initial rigid body registration is performed using structural coincident point pairs to unify adjacent tiles into a continuous road axis coordinate system;

[0059] In obtaining the set of structural coincidence points Finally, the rigid body transformation is constructed to determine the rigid body space transformation matrix of tile k+1 relative to tile k: ,in, represents the rigid body transformation group in three-dimensional space, is the rotation matrix in three-dimensional space, which indicates the change of attitude direction; t is the translation vector, which indicates the displacement of the coordinate origin;

[0060] Minimize the coincidence error function: , get the transformation relationship from block k+1 to block k coordinate system;

[0061] Transform all point clouds in the k+1th tile into: ,in, is the original 3D coordinate of the j-th point cloud point in tile k+1; It means that the point in the k+1 block is mapped to the new coordinates in the block k coordinate system after rigid body transformation.

[0062] In a preferred embodiment, a global error minimization model is constructed to uniformly optimize the poses and positions of all image blocks and perform an overall road space survey. The specific steps are as follows:

[0063] After completing the local rough alignment, a global alignment optimization mechanism is used to make all tiles have a consistent coordinate system in the global road space;

[0064] Transform the coordinates of each of the N tiles contained into: ,in, is the global pose of the kth tile; Describe the orientation / posture change of tile k in space; Represents the position offset of tile k relative to the global coordinate system;

[0065] Minimize the structural point coincidence error between all tiles: ,in, 、 Represent the coordinates of the structural points of tiles i and j respectively; 、 Represent the global pose transformation of tiles i and j respectively; SY is the index set of all tile pairs with overlapping structural points;

[0066] The structural point coincidence error between tiles is minimized, the optimal transformation of each tile is solved, and the optimal transformation result is used as the input for road scene modeling to conduct overall road space survey.

[0067] The technical effects and advantages of the road scene space rapid survey system combining RTK and image reconstruction of the present invention are as follows:

[0068] This method collects high-precision RTK trajectory data during road inspections and synchronizes it with image frames in time to establish a one-to-one correspondence between images and spatial positions. To address the multipath effects and pseudo-jump point problems common on urban roads, a continuity detection and anomaly rejection mechanism is designed to extract a physically reasonable and spatially stable control point sequence. Based on the control points and direction vectors, a road topology skeleton structure is constructed. Based on this, the entire road space is divided into multiple local mapping units (Tiles). Each unit has a main axis direction and a boundary range. Redundant areas are constructed by setting spatial overlap bands to ensure structural connectivity between tiles. The system further extracts semantic structural information such as lane lines, curbs, and signs from the image, integrates edge detection to construct a structural mask, and maps it to three-dimensional space to guide density distribution adjustment in point cloud reconstruction.

[0069] Finally, local pose registration of tile boundaries is achieved through structural point matching, a global error minimization model is constructed, and the pose and position of all tiles are uniformly optimized to achieve continuous splicing and structural consistency of the road scene spatial model, thereby improving the mapping efficiency in road spatial scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 This is a structural diagram of a road scene spatial rapid survey system combining RTK and image reconstruction according to the present invention. DETAILED DESCRIPTION

[0071] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0072] In order to achieve the above objectives, Figure 1 The present invention provides a schematic structural diagram of a road scene spatial rapid survey system combining RTK and image reconstruction. The system specifically includes a road trajectory acquisition module, a road mapping and analysis module, a structure enhancement module, and a road survey module. The modules are connected by signals.

[0073] The road trajectory acquisition module is used to collect road inspection trajectories and synchronize image frames with trajectory points, eliminate abnormal jump points, and extract continuous control points. It then establishes a road topology skeleton based on the control points and direction vectors.

[0074] The road mapping analysis module is used to divide the road space into continuous mapping units based on control points and direction information, set the spatial range and main axis direction and region attribution mapping for the mapping units, and construct independent mapping task packages to map overlapping areas;

[0075] The structure enhancement module is used to extract the structural semantic areas and edge information in the image, construct a structure mask and map it to the three-dimensional space, and mark it as a structure enhancement point;

[0076] The road survey module is used to identify areas where tile boundary structures overlap and establish anchor points, complete local rigid body registration, build a global error minimization model, uniformly optimize the pose and position of all tiles, and conduct overall road space surveys.

[0077] In multi-source perception data collection for road space, RTK-GNSS (Real-Time Kinematic High-Precision Satellite Positioning System) typically independently collects position information at a high frequency, while image acquisition devices (such as cameras or multi-view cameras) capture image sequences based on frame rates. For example, RTK provides high-frequency spatial positioning data (e.g., above 10 Hz), while image acquisition frame rates are relatively low (e.g., 5 Hz to 15 Hz). Due to differences in time bases and data update cycles between the two types of devices, without a unified time reference or synchronization strategy, temporal drift or spatial alignment errors can easily occur between image frames and trajectory points, affecting the accuracy of subsequent image reconstruction and spatial modeling.

[0078] Step 1: Construct a priori road topology for the RTK trajectory. Based on the high-precision positioning characteristics of RTK, a road topology priori model with spatial continuity, attitude reference, and structural expressiveness is constructed. The specific steps are as follows:

[0079] Perform RTK trajectory data acquisition and time synchronization. During the acquisition phase, a time synchronization mechanism is established between image frames and RTK trajectory points. Each image frame is bound to a spatial trajectory point based on the principle of minimum time difference.

[0080] The three-dimensional spatial positioning data collected in the road scene is used as RTK trajectory data. The RTK trajectory data corresponds to the real-time spatial position of each frame of image acquisition in the road scene. The acquisition object is the installation location of the image acquisition terminal (such as a vehicle-mounted camera, mobile work platform, handheld device, trajectory scanning device, etc.). The acquisition path is all spatial points passed along the road topology path during the road inspection process;

[0081] The coverage areas include but are not limited to: motor vehicle lanes, sidewalks, bicycle lanes; the intersection of curbs and lane lines; turning corners, intersections, ramps, and intersections; under overpasses, culverts, overpasses, and other areas with strong spatial constraints.

[0082] Perform RTK track point collection as a track point sequence, set as: ,in, , Represents the three-dimensional positioning coordinates at the i-th moment, located in the geocentric coordinate system or local projection coordinate system, Indicates the acquisition timestamp of the trajectory point, recorded in a unified system reference time (such as UTC time or system relative time);

[0083] Image frame data is represented as: ,in, is the image frame; is the image frame acquisition time;

[0084] For each image frame, find the RTK track point closest to the timestamp, which is recorded as: , the pairing result is , used to represent the temporal binding of images and spatial points;

[0085] Set the time matching tolerance threshold , if the match is satisfied: , then the data pair is retained; otherwise the image frame is discarded, usually The value range is 0.02 to 0.05 seconds, which can be determined according to the RTK frequency and camera synchronization capability; Representation and image frame The timestamp of the closest RTK track point.

[0086] Although RTK systems offer high-precision positioning capabilities, they are often affected by high-rise buildings, electromagnetic interference, or multipath effects in non-ideal environments such as urban roads, resulting in sudden pseudo-jumps. These jumps are often accompanied by sudden changes in direction or a sharp increase in distance, severely disrupting trajectory continuity and structural expression. If left untreated, they can directly lead to subsequent module division offsets, direction estimation failures, and even image reconstruction crashes. Therefore, continuity detection and outlier removal are performed on the collected trajectory sequence to restore the physical plausibility and spatial consistency of the trajectory.

[0087] To smooth the RTK trajectory and remove false jump points, the specific steps are as follows:

[0088] Suppose any three consecutive trajectory points are , calculate the direction vector: ,in, It is from the point Pointing Point The direction vector, It is from the point Pointing Point The direction vector of

[0089] Structural direction change angle: ,in, represents the vector dot product, represents the Euclidean norm of a vector; represents the turning angle of the trajectory at point i, in radians;

[0090] Calculate the spatial distance between adjacent trajectory points: ,like , it means there is a violent space jump, is the distance threshold;

[0091] The pseudo jump point determination rules are set as follows: If a point satisfies both: and , it is determined to be an abnormal jump point and recorded as a pseudo jump point; is the set corner threshold;

[0092] If the point has only isolated jumps, it is directly eliminated. If the trajectory points before and after the jump point are continuous, linear interpolation or spline reconstruction is used to fill the gap.

[0093] Summarize the interpolation points to form the final set of control points: ,in, It is a three-dimensional Euclidean real space, that is, a three-dimensional vector space composed of three real coordinates, such as a set of triplets (x, y, z); n is the total number of control points extracted, which changes dynamically depending on the complexity of the path. From the original trajectory point set or interpolated points, the order is kept consistent with the trajectory path, as the space partition anchor points and skeleton graph nodes.

[0094] In image reconstruction-based mapping, the camera pose (i.e., viewing direction) directly affects the accuracy of image registration, the stability of depth estimation, and the direction of point cloud projection. Ignoring the initialization of the camera orientation will lead to divergence in the orientation of local mapping units and discontinuous reconstruction results. Therefore, it is necessary to estimate the direction vector based on the spatial relationship between control point pairs to initialize the image pose and spatial module orientation. The specific steps for control point pose estimation and viewpoint constraint establishment are as follows:

[0095] Construct a direction vector for any pair of consecutive k-th and k+1-th control points , define the unit direction vector as: ,in, , represents the main axis direction of the k-th spatial segment, which is used to indicate the main observation direction of the camera in the area or the road extension direction;

[0096] Construct a set of attitude vectors and a set of directions: , each direction vector and corresponding control points Bind to form a posture reference pair: ,This pose information will be used as the orientation reference for the partition, as the main vector for initializing the camera pose estimation for image reconstruction, and as the pose constraint input in the tile stitching process.

[0097] In a multi-block, multi-path road scene, a global data structure that can express the road structure topology is constructed, that is, a road skeleton graph structure is constructed to define the graph. : ;

[0098] A set of directed edges: ;

[0099] Each edge records the following attribute tuple: ,in, is the side length, which represents the spatial distance between the control point pairs; is the direction angle, which indicates the turning angle between road segments;

[0100] A road skeleton graph is constructed for connection planning, splicing path reconstruction and direction navigation between spatial segments, and is used for error constraint propagation in the global alignment stage as a topological index structure for the road scene.

[0101] Step 2: Based on the mapping module division of local reconstruction units, the road space is divided into controllable and splicable modular units according to the control point sequence and road skeleton structure constructed in Step 1. By combining the spatial distribution pattern, trajectory direction vector, and perspective consistency principle, the overall road space reconstruction task is decomposed into several structurally independent but spatially continuous mapping units. The specific steps are as follows:

[0102] Establish the spatial division benchmark and reconstruction axial reference as follows:

[0103] Based on the output control point sequence and the corresponding direction vector set, the division center and main direction of each local mapping unit are determined as the basis for the subsequent construction of spatial mapping units;

[0104] Let the kth control point be , whose direction vector is , then the spatial principal axis reference set can be constructed: , where n is the total number of control points, and each pair of adjacent control points It is considered as a basic segment interval, representing a road logical unit. The direction vector indicates the main axis direction of the segment, which is used to guide the orientation layout of subsequent building blocks in space.

[0105] After obtaining the principal axis reference of each segment, a local mapping unit area needs to be constructed with the control point pair as the center. Each mapping unit area should have sufficient image data covering the segment and good spatial closure to facilitate subsequent point cloud generation and texture mapping.

[0106] Therefore, for the kth segment, the spatial region center of the corresponding local mapping unit is recorded as: , and set its spatial bounding box parameters as the long axis direction as the direction vector, the length is set to L (covering distance); the vertical direction , the width W is set by environmental constraints, usually consistent with the horizontal width of the road; the height direction is set to H, estimated according to the vertical range of the image viewing angle;

[0107] The boundary region of the mapping unit is constructed as follows: , where Box represents the spatial enclosing area defined by the center and direction parameters. After the structure is constructed, each mapping unit area will become a spatial carrier for independent image reconstruction, and the processing of image data, point cloud data, and texture mapping are all confined to the mapping unit.

[0108] In the actual reconstruction process, if the mapping units are completely separated, texture discontinuity, structural fracture, or splicing errors are very likely to occur at the reconstruction seams. Therefore, a reconstruction redundancy band design mechanism is introduced to reserve a certain spatial overlap area between adjacent mapping units for geometric alignment and feature fusion.

[0109] Assume that the overlapping length of the mapping unit is , then make an equidistant extension on the original mapping unit length L, and define the effective area of the mapping unit as: , in the process of image frame attribution, if the center point of the image frame falls into , then the frame will belong to two mapping units for redundant mapping and be used for feature fusion processing between the two mapping units;

[0110] Adjacent segments are overlapped to a certain extent in the length direction. The images in the overlapping area will participate in the reconstruction process of two mapping units at the same time. The adjacent mapping units will have a common structure in the reconstruction results, which can be used for boundary matching, posture calibration and error fusion in the subsequent stitching stage.

[0111] After completing the division of spatial mapping units, the system needs to structurally encapsulate the spatial parameters, image data, directional constraints, and reconstruction configuration of each mapping unit to form an independent mapping task package. For each mapping unit, the task unit is constructed: ,in, is the set of image frames belonging to the mapping unit; is the set of RTK trajectory points corresponding to the image frame; is the main direction vector; The center position of the mapping unit, used for path indexing and local coordinate system initialization.

[0112] Starting from the control point drive, a mapping unit division method with directional constraints, structural continuity and redundant robustness is constructed. Each segment has a clear spatial boundary, controllable calculation range and precise image attribution logic, thereby improving the spatial organization ability and structural integrity of the overall system performance. Figure 1 Consistency.

[0113] Step 3: Perform structural feature-guided enhanced reconstruction of road elements. This involves identifying and reconstructing structurally constrained and semantically significant areas of the road scene (such as lane lines, edges, curbs, and signage). By extracting and mapping image structural features, combined with directional consistency constraints and spatial boundary geometry guidance strategies, the reconstruction quality of key areas is enhanced. The specific steps are as follows:

[0114] Extract the regional information with road semantics and geometric structure from the image frames assigned to the local mapping unit as the enhancement object for subsequent spatial reconstruction. Use the semantic segmentation and structural edge joint extraction mechanism to identify the typical road structure area in each image frame. , where m = 1, 2, ..., M, is used to represent the image frame number in the local mapping unit;

[0115] Introducing a pre-trained road scene structure semantic segmentation model , perform pixel-level structural classification on the image and obtain the semantic label map: ;

[0116] At the same time, the Canny operator is used for gradient edge detection to obtain the image edge structure diagram: ,in, It is the Gaussian kernel scale parameter for edge detection, which is used to adjust the smoothness of edge response;

[0117] Combine the semantic map with the edge map to construct a multi-structure joint mask: ;in, For structural label areas (such as lane lines, road edges, obstacles), It is the edge response area.

[0118] This mask is used to indicate the areas in the image that are most valuable for expressing structures in the mapping process.

[0119] For example, suppose the system captures a frame of image, which is taken on a main road in the city. The picture contains multiple white lane lines, a yellow solid line, curbs and sidewalk boundaries. The image has typical road linear elements with obvious color and structure contrast. The system calls the structured semantic segmentation model to perform pixel-by-pixel label inference on the image. The model identifies areas such as lane lines, solid lines, sidewalk edges, arrow signs, etc. in the image, and generates corresponding semantic label maps. At the same time, the Canny edge detection algorithm is used to extract grayscale gradient change areas from the image to enhance the structural contour recognition of texture fuzzy areas. The above two are logically intersected, and the areas that are significant in both semantics and edges are retained to generate structural mask areas, whose corresponding images contain linear structures such as lane lines and road boundaries.

[0120] After image feature extraction, the salient areas of these two-dimensional images need to be accurately mapped into three-dimensional space. Using the camera pose and spatial orientation information obtained from RTK registration, the high-confidence structural areas in each image are projected into three dimensions to obtain a set of structural guide points. These projections fall on specific coordinate points in space, representing the corresponding positions of the structurally significant areas in the image. The specific steps are as follows:

[0121] For each image frame, the pixel-level mask is projected into a set of three-dimensional space points using the camera's internal and external parameters.

[0122] Convert the two-dimensional image coordinates into normalized camera coordinate vectors, and then map them into three-dimensional space projection directions through camera extrinsics;

[0123] Combined with the RTK trajectory points corresponding to the frame, line-of-sight space points can be generated, and the three-dimensional points after projection of all significant structures are recorded as a three-dimensional point set. Its spatial distribution is used to indicate the areas where the sampling density needs to be enhanced in the point cloud reconstruction stage.

[0124] In the point cloud generation stage, an adaptive sampling mechanism is introduced for the significant structure area to improve the geometric restoration accuracy. The dense point cloud generation algorithm is set as the input of the image pair and its corresponding pose information, feature matching points, etc., and the spatial position of the structure enhancement point area is converted into the image pair. Introducing the reconstruction pipeline as a sampling reference constraint to construct a spatial density adjustment factor For structural enhancement: ,in, Indicates the candidate position for reconstruction of the current point cloud, Enhanced impact range threshold for structures; is the response slope control factor, which is used to control the density change speed, and q represents the spatial position coordinates of the structural reinforcement point area;

[0125] The closer the value of the structural space density adjustment factor is to 1, the closer the area is to the structural feature point, and the reconstruction algorithm needs to increase the sampling density.

[0126] After the initial point cloud generation, geometric optimization and texture refinement are performed on the structural area to enhance the edge clarity and surface consistency of the reconstruction results. For the point cloud within each mapping unit, a local neighborhood subset close to the structural guide point is extracted, and geometric boundary repair and texture mapping optimization operations are performed. A local optimization strategy based on reprojection error is adopted. The optimization goal is to minimize the multi-view reprojection error in the structural area, making the point cloud reconstruction more image-consistent in semantically important areas.

[0127] Because each tile may be generated from a different path segment, its image acquisition is subject to directional changes, RTK error perturbations, and local view occlusions. This results in the reconstruction results of different tiles being in different coordinate systems, making it impossible to directly splice them into a continuous road space model. For example, in a road scene, each local module (mapping unit) often corresponds to a linear driving path, and its boundary is usually located at the intersection of the previous and next image reconstruction areas. However, due to factors such as image quality, attitude error, and RTK positioning jitter, adjacent tiles are not naturally seamless in geometry.

[0128] Therefore, tile stitching and global alignment guided by structural overlap are required to ensure that the three-dimensional structure of the entire road scene has geometric continuity, semantic consistency and coordinate uniformity. It is necessary to find spatial structural overlap areas between tile boundaries as stitching anchor points. Spatial structure not only refers to the overlap of geometric point clouds, but also refers to the spatial consistency clues formed by structural features (such as lane line edges, sidewalk corners, and road signs).

[0129] Step 4: Perform local tile stitching and global alignment of the road scene space. The specific steps are as follows:

[0130] Between each pair of adjacent tiles, the structural salient points of their boundary areas are first extracted, including lane corners, curb edges, sudden corners or traffic sign outlines, etc., to form splicing anchor points, that is, the set of boundary structural feature points of the kth and k+1th tiles. They are: ,in, 、 Respectively represent the three-dimensional coordinate vectors of the i-th and j-th structural feature points in the k-th block;

[0131] Determine whether there are significant structural coincidence point pairs by nearest neighbor matching: , forming a candidate matching set: ,in, is the set of candidate matching structure point pairs between blocks k and k+1;

[0132] If satisfied: , then it is considered that the two blocks have good spatial connectivity at the structural boundary and can be further aligned. is the minimum matching point pair number threshold, which is used to determine whether there is a valid structural overlap;

[0133] Since each tile is constructed based on local control points, its reconstructed coordinate system is a relative coordinate system. Due to factors such as direction estimation fluctuations, RTK attitude disturbances, and inconsistent image acquisition, different tiles may have coordinate offsets or attitude rotation errors in the global space. Therefore, initial rigid body registration (attitude alignment) is performed using structural coincidence point pairs to unify adjacent tiles into a continuous road principal axis coordinate system.

[0134] Input the structure point pair set, that is, the matching point set obtained using the previous substep:

[0135] In obtaining the set of structural coincidence points Finally, the rigid body transformation is constructed to determine the rigid body space transformation matrix of tile k+1 relative to tile k: ,in, represents the rigid body transformation group in three-dimensional space (including rotation and translation), is the rotation matrix in three-dimensional space, which indicates the change of attitude direction; t is the translation vector, which indicates the displacement of the coordinate origin;

[0136] By minimizing the coincidence error function: , we get the transformation relationship from tile k+1 to tile k coordinate system. Then we transform all point clouds in tile k+1 into: ,in, is the original 3D coordinate of the j-th point cloud point in tile k+1; It means that the points in the k+1 block are mapped to the new coordinates in the block k coordinate system after rigid body transformation, thereby achieving coordinate consistency;

[0137] After completing the local coarse alignment, to eliminate the accumulated errors at the tile level (especially in long paths or loop closure scenarios), a global alignment optimization mechanism is used to ensure that all tiles have a consistent coordinate system in the global road space.

[0138] Transform the coordinates of each of the N tiles contained in the system into: ,in, is the global pose (attitude and position) of the kth tile, which describes how the tile is placed in three-dimensional space; Describe the orientation / posture change of tile k in space; Represents the position offset of tile k relative to the global coordinate system, that is, the displacement of the tile coordinate origin relative to the global origin;

[0139] Construct a global optimization problem to minimize the structural point coincidence error between all tiles: ,in, 、 Represent the coordinates of the structural points of tiles i and j respectively; 、 Represent the global pose transformation of tiles i and j respectively; SY is the index set of all tile pairs with overlapping structural points;

[0140] By minimizing the coincidence error of the structural points between the above-mentioned tiles, the system automatically solves the optimal transformation for each tile, making all coincident structural points as consistent as possible in the global coordinate system (with minimal coincidence error), thereby achieving posture consistency, high-precision stitching, and path continuity. This solves the problem of local reconstruction error accumulation, establishes a multi-source fusion alignment framework with RTK trajectories as the initial benchmark and image structural points as the constraint core, uses the optimal transformation results as the input for road scene modeling, and conducts an overall road space survey.

[0141] For example, a section of a winding road in a suburban city is undergoing high-precision 3D modeling for road expansion design and municipal maintenance. The survey team deployed a road survey system, using an onboard RTK-GNSS system to record the vehicle's position and posture in real time. Simultaneously, multiple high-definition wide-angle cameras were installed to capture images of both sides of the road. The system automatically divided the entire road section into multiple image tiles (tiles 1, 2, 3, and so on), each covering approximately 10 meters.

[0142] The boundary between tiles 1 and 2 represents the same curve. The guardrails, curbs, and other structures captured by the camera should overlap. However, due to vehicle vibration, tilt, and image reconstruction errors, the guardrail point clouds in tiles 1 and 2 are misaligned by tens of centimeters, and their orientations are slightly offset (e.g., one side is slightly tilted upwards, the other slightly downwards). Without alignment, the entire road skeleton becomes disconnected, resulting in spatial discontinuities and increased measurement errors.

[0143] In the boundary area between tile 1 and tile 2, the system automatically identifies structural points such as the top of the guardrail and the intersection of the white line on the ground, and establishes a point pair set;

[0144] The coordinate transformation matrix of all tiles is included in the optimization variables, with the goal of minimizing the structural point error between all tile pairs;

[0145] After optimization and solution, the result is that block 1 is fine-tuned and translated to the left, and block 2 is rotated about 0.5 degrees downward, so that the boundary guardrail structure points are aligned in three-dimensional space. The boundaries of all block paths are continuous, and adjacent structures are naturally integrated. The constructed three-dimensional road model path is continuous and seamless, with a natural posture, and the error converges to the centimeter level.

[0146] The relevant threshold information in this embodiment is pre-set by professionals and will not be explained in detail here. Some parameter English letters in the embodiments have the same situation, but different meanings are explained when used, which will not be explained one by one here.

[0147] This method collects high-precision RTK trajectory data during road inspections and synchronizes it with image frames in time to establish a one-to-one correspondence between images and spatial positions. To address the multipath effects and pseudo-jump point problems common on urban roads, a continuity detection and anomaly rejection mechanism is designed to extract a physically reasonable and spatially stable control point sequence. Based on the control points and direction vectors, a road topology skeleton structure is constructed. Based on this, the entire road space is divided into multiple local mapping units (Tiles). Each unit has a main axis direction and a boundary range. Redundant areas are constructed by setting spatial overlap bands to ensure structural connectivity between tiles. The system further extracts semantic structural information such as lane lines, curbs, and signs from the image, integrates edge detection to construct a structural mask, and maps it to three-dimensional space to guide density distribution adjustment in point cloud reconstruction.

[0148] Finally, local pose registration of tile boundaries is achieved through structural point matching, a global error minimization model is constructed, and the pose and position of all tiles are uniformly optimized to achieve continuous splicing and structural consistency of the road scene spatial model, thereby improving the mapping efficiency in road spatial scenes.

[0149] The above formulas are all dimensionless and numerical calculations. The formula is a formula that is closest to the actual situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0150] The above embodiments may be implemented in whole or in part through software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product.

[0151] Those skilled in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0152] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0153] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0154] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A road scene spatial rapid survey system combining RTK and image reconstruction, characterized by: It includes a road trajectory acquisition module, a road mapping and analysis module, a structure enhancement module, and a road survey module, and each module is connected by signals; The road trajectory acquisition module is used to collect road inspection trajectories and synchronize image frames with trajectory points, eliminate abnormal jump points, and extract continuous control points. It then establishes a road topology skeleton based on the control points and direction vectors. The road mapping analysis module is used to divide the road space into continuous mapping units based on control points and direction information, set the spatial range and main axis direction and region attribution mapping for the mapping units, and construct independent mapping task packages to map overlapping areas; The structure enhancement module is used to extract the structural semantic areas and edge information in the image, construct a structure mask and map it to the three-dimensional space, and mark it as a structure enhancement point; The road survey module is used to identify the overlapping areas of tile boundary structures and establish anchor points, complete local rigid body registration, and then build a global error minimization model to uniformly optimize the pose and position of all tiles and conduct overall road space survey; It is used to divide the road space into continuous mapping units based on control points and direction information, set the spatial range and main axis direction and area attribution mapping for the mapping units, and construct independent mapping task packages for overlapping area mapping. It includes the following steps: Construct a spatial principal axis reference set based on the output control point sequence and the corresponding direction vector set. Each pair of adjacent control points is considered as a segment interval, and the direction vector represents the basic segment principal axis direction. After obtaining the main axis reference of each segment, for the kth segment, the spatial area center of the corresponding local mapping unit is recorded as: ; The boundary region of the mapping unit is constructed as follows: , where Box represents the spatial enclosing area defined by center and direction parameters; is the main direction vector; its length is L; its vertical direction is , the width W is set by the environmental constraints; the height direction is set to H; After the structure is constructed, each mapping unit area is used as a spatial carrier for independent image reconstruction; The overlapping length of the mapping unit is , do equidistant expansion on the original mapping unit length L, and determine the effective area of the mapping unit as: ; During the image frame attribution process, if the center point of the image frame falls into the valid area of the adjacent mapping unit, the image frame will be assigned to two mapping units for redundant mapping; After completing the division of spatial mapping units, the spatial parameters, image data, direction constraints and reconstruction configuration of each mapping unit are structured and packaged to form an independent mapping task package; Construct a task unit for each mapping unit: ,in, is the set of image frames belonging to the mapping unit; is the set of RTK trajectory points corresponding to the image frame; is the main direction vector; is the center position of the mapping unit.

2. The road scene spatial rapid survey system combining RTK and image reconstruction according to claim 1, characterized in that: It is used to collect road inspection tracks and synchronize image frames with track points. The specific steps are as follows: The three-dimensional spatial positioning data collected in the road scene is used as RTK trajectory data. The RTK trajectory data corresponds to the real-time spatial position of each frame of image acquisition in the road scene. The acquisition object is the installation location of the image acquisition terminal, including vehicle-mounted cameras, mobile work platforms, handheld devices, and trajectory scanning devices. The acquisition path is all spatial points passed along the road topology path during the road inspection process. Coverage areas include motor vehicle lanes, sidewalks, bicycle lanes, curbs, lane line intersections; turning corners, intersections, ramps, intersections, underpasses, culverts, and overpasses; Perform RTK track point collection as a track point sequence. The RTK track point includes the 3D positioning coordinates corresponding to the collection position and the corresponding timestamp; The collected image frame data includes the image frame and the image frame collection time; For each image frame, find the RTK track point closest to the timestamp and perform time binding between the image and the track point; Set the time matching tolerance threshold. If the absolute value of the difference between the image frame acquisition time and the timestamp of the closest RTK track point is less than or equal to the matching tolerance threshold, the frame image is bound to an RTK track point and the pairing data is retained.

3. The road scene spatial rapid survey system combining RTK and image reconstruction according to claim 2, characterized in that: Remove abnormal jump points, extract continuous control points, and build a road topology skeleton based on control points and direction vectors. The specific steps include: Perform RTK trajectory smoothing and pseudo-jump point removal, and convert three consecutive trajectory points into , the direction vector is calculated as ,in, It is from the point Pointing Point The direction vector, It is from the point Pointing Point The direction vector of Tectonic direction change angle: ,in, represents the Euclidean norm of a vector; represents the turning angle of the trajectory at point i; Calculate the spatial distance between adjacent trajectory points : ,like , there is a violent space jump, is the distance threshold; The pseudo jump point determination rules are set as follows: If the trajectory point satisfies both: and , it is determined to be an abnormal jump point and recorded as a pseudo jump point; is the set corner threshold; If the pseudo jump point is only an isolated jump, it is directly eliminated. If the trajectory points before and after the jump point are continuous, linear interpolation or spline reconstruction is used to fill the gap; Summarize the interpolation points to form the final set of control points: ,in, is a three-dimensional Euclidean real space, is the kth control point; n is the total number of extracted control points; Each control point from the trajectory point set or interpolation point is used as a space partition anchor point and skeleton graph node.

4. The road scene spatial rapid survey system combining RTK and image reconstruction according to claim 3, characterized in that: Each control point from the trajectory point set or interpolation point is used as a space partition anchor point and skeleton graph node. The specific steps are as follows: Construct a direction vector for any pair of consecutive k-th and k+1-th control points , C is the control point set, and the unit direction vector is defined as: ,in, , represents the main axis direction of the k-th space segment; Construct a set of posture vectors and a set of direction vectors: , each direction vector and corresponding control points Bind to form a posture reference pair: , the pose information is used as the orientation reference for the partition, and as the main vector for initializing the camera pose estimation for image reconstruction, and as the pose constraint input during the tile stitching process; Construct the road skeleton graph structure and define the road skeleton graph as: : ; A set of directed edges: ; Each edge records the following attribute tuple: ,in, is the side length, which represents the spatial distance between the control point pairs; is the direction angle, which represents the turning angle between road segments.

5. The road scene spatial rapid survey system combining RTK and image reconstruction according to claim 4, characterized in that: It is used to extract the structural semantic area and edge information in the image, construct a structure mask and map it to the three-dimensional space, and mark it as a structure enhancement point. The specific steps are as follows: Extracting road semantic and geometric structural region information from image frames assigned to local mapping units, and using a joint semantic segmentation and structural edge extraction mechanism to identify road structure regions in each image frame. Perform pixel-level structural classification on the image to obtain a semantic label map; At the same time, the Canny operator is used to perform gradient edge detection to obtain the image edge structure map; Combine the semantic label map with the edge structure map to construct a multi-structure joint mask; After image feature extraction, the camera pose and spatial orientation information obtained by RTK registration are used to perform three-dimensional projection of the high-confidence structure area in each image to obtain a set of structural guide points, which are used as structural enhancement points for structural enhancement.

6. The road scene spatial rapid survey system combining RTK and image reconstruction according to claim 5, characterized in that: After image feature extraction, the camera pose and spatial orientation information obtained from RTK registration is used to perform a 3D projection of the high-confidence structure areas in each image to obtain a set of structural guide points. These points are then used as structural enhancement points for structural enhancement. The specific steps are as follows: For each image frame, the pixel-level mask is projected into a set of three-dimensional space points using the camera's internal and external parameters. Convert the two-dimensional image coordinates into normalized camera coordinate system vectors, and map them into three-dimensional space projection directions through camera extrinsics; Combine the RTK trajectory points corresponding to the image frame to generate the line of sight space points, and record the 3D points after the projection of all significant structures as a 3D point set; In the point cloud generation stage, an adaptive sampling mechanism is used for the significant structure area to enhance the spatial position of the structure point area. Use the reconstruction pipeline as a sampling reference constraint to construct a spatial density adjustment factor : ,in, Indicates the candidate position for reconstruction of the current point cloud, Enhanced impact range threshold for structures; is the response slope control factor; q is the spatial position coordinate of the structural reinforcement point area; After the initial point cloud generation, the structural region undergoes geometric optimization and texture refinement. For each point cloud within a mapping unit, a local neighborhood subset close to the structural guide point is extracted, and geometric boundary repair and texture mapping optimization operations are performed. Based on a local optimization strategy for reprojection error, the optimization goal is to minimize the multi-view reprojection error of the structural region.

7. The road scene spatial rapid survey system combining RTK and image reconstruction according to claim 6, characterized in that: It is used to identify the overlapping areas of tile boundary structures and establish anchor points to complete local rigid body registration. The specific steps are as follows: Between each pair of adjacent tiles, the structural salient points of the boundary area are extracted, including lane corners, roadside boundaries, sudden corners or traffic sign outlines, to form splicing anchor points, and the boundary structural feature point sets of the kth and k+1th tiles are combined. Respectively expressed as: ,in, 、 Respectively represent the three-dimensional coordinate vectors of the i-th and j-th structural feature points in the k-th block; is a three-dimensional Euclidean real space; Determine whether there are significant structural coincidence point pairs by nearest neighbor matching: ; And form a candidate matching set: ,in, is the set of candidate matching structure point pairs between blocks k and k+1; If satisfied , then the two blocks have good spatial connectivity at the structural boundary, and further registration is performed, where is the minimum matching point pair number threshold; Initial rigid body registration is performed using structural coincident point pairs to unify adjacent tiles into a continuous road principal axis coordinate system; In obtaining the set of structural coincidence points Finally, the rigid body transformation is constructed to determine the rigid body space transformation matrix of tile k+1 relative to tile k: ,in, represents the rigid body transformation group in three-dimensional space, is the rotation matrix in three-dimensional space, which indicates the change of attitude direction; t is the translation vector, which indicates the displacement of the coordinate origin; Minimize the coincidence error function: , get the transformation relationship from block k+1 to block k coordinate system; Transform all point clouds in the k+1th tile into: ,in, is the original 3D coordinate of the j-th point cloud point in tile k+1; It means that the point in the k+1 block is mapped to the new coordinates in the block k coordinate system after rigid body transformation.

8. The road scene spatial rapid survey system combining RTK and image reconstruction according to claim 7, characterized in that: Then, we build a global error minimization model, uniformly optimize the pose and position of all tiles, and conduct an overall road space survey. The specific steps are as follows: After completing the local rough alignment, a global alignment optimization mechanism is used to make all tiles have a consistent coordinate system in the global road space; Transform the coordinates of each of the N tiles contained into: ,in, is the global pose of the kth tile; Describe the orientation / posture change of tile k in space; Represents the position offset of tile k relative to the global coordinate system; Minimize the structural point coincidence error between all tiles: ,in, 、 Represent the coordinates of the structural points of tiles i and j respectively; 、 Represent the global pose transformation of tiles i and j respectively; SY is the index set of all tile pairs with overlapping structural points; The structural point coincidence error between tiles is minimized, the optimal transformation of each tile is solved, and the optimal transformation result is used as the input for road scene modeling to conduct overall road space survey.

Images