Method and device for constructing real scene three-dimensional model of vehicle-mounted mobile measurement system and vehicle-mounted mobile measurement system

Abstract

The application discloses a real scene three-dimensional model construction method and device based on a vehicle-mounted mobile measurement system and the vehicle-mounted mobile measurement system, relates to the technical field of three-dimensional model construction, and comprises the following steps: collecting acceleration and angular velocity through inertial navigation after time synchronization of each sensor is completed; collecting mileage information of vehicle advancement based on a mileage counter; calculating a first position and a first attitude of a vehicle carrier which are not optimized according to the acceleration, the angular velocity and the mileage information; optimizing the first position and the first attitude based on a three-dimensional laser radar to obtain optimized pose information; calculating geographical coordinates of three-dimensional point cloud data and panoramic image pose according to the optimized pose information; processing the geographical coordinates of the three-dimensional point cloud data through the panoramic image pose to obtain target three-dimensional point cloud data, and calculating a digital camera image pose through the panoramic image pose; and constructing a real scene three-dimensional model through the target three-dimensional point cloud data and the digital camera image pose.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional model construction, in particular to a real scene three-dimensional model construction method and device based on a vehicle-mounted mobile measurement system and the vehicle-mounted mobile measurement system. BACKGROUND

[0002] Mobile mapping technology (MMS) is a new technology in the forefront of the surveying and mapping industry today, which is widely used in real scene three-dimensional ground multi-source data acquisition and processing, and is combined with aerial data and unmanned aerial vehicle data to construct a real scene three-dimensional model, so as to achieve the goal of expressing road and road surrounding landscape and building facade information. The vehicle-mounted mobile measurement system is an automatic data acquisition and processing system which takes a common vehicle or a patrol vehicle as a mobile platform, and is equipped with multiple types of sensors, core processors, synchronous controllers and other functional modules, so as to realize one-time acquisition, automatic fusion of image, laser point cloud, position and attitude data, and quickly meet the application requirements of multiple fields.

[0003] The core of the vehicle-mounted mobile measurement system is the high integration of multiple functional modules. The main sensors of the vehicle-mounted mobile measurement system include a global satellite navigation system (GNSS) positioning antenna and receiver for positioning and attitude determination, an inertial navigation system (IMU) and an odometer, a panoramic camera and a digital camera for acquiring image data, and a laser radar for acquiring three-dimensional point cloud. The time synchronization controller can realize synchronous control of multiple source sensors and record the trigger time of each sensor. The core processor can control and display the system state, unify the spatial coordinate reference system of different image data and point cloud data to realize fusion processing, and realize two-dimensional or three-dimensional expression of real world geographic scenes and geographic entities. At the same time, the platform design of the vehicle-mounted mobile measurement system should have practicality and flexibility, and can quickly respond and work efficiently in complex and variable scenes. The GNSS satellite signal and inertial navigation combined positioning technology used in the existing vehicle-mounted mobile measurement system can only achieve high-precision positioning and attitude determination under the condition of open field of view and no obvious influence of surrounding environment. In the complex urban scene, GNSS satellite signal shielding, high-power radio interference such as microwave towers and transmitting antennas, and serious multipath reflection in large-area water, iron sheds and metal areas seriously restrict high-precision positioning and attitude determination. GNSS satellite signal loss causes the inertial navigation and odometer drift to be unable to be corrected, and the positioning accuracy rapidly decreases to several meters to tens of meters, which cannot meet the application requirements.

[0004] The construction of the basic geographic entity model in the real three-dimensional construction requires the collection of high-resolution and high-overlap images to realize the joint modeling of aerial and ground images, so as to construct the real model of the street road and the landscape on both sides, and express the facade information of the street scene road and the landscape on both sides and the building structure. The existing vehicle-mounted mobile measurement system generally uses a panoramic camera to collect images. The panoramic camera can realize 360° horizontal field of view angle image collection and splicing, and is used for true color coloring of point cloud data. However, due to the low pixel resolution and obvious distortion of panoramic images (or segmented single-lens images), it cannot be used for real three-dimensional modeling. If a digital camera is mounted on the vehicle-mounted mobile measurement system, high-pixel-resolution images can be collected, but the field of view angle of the digital camera lens is small, and it is difficult to realize high overlap between the front and rear frames of images during the movement of the vehicle-mounted platform, thereby causing modeling failure. On the other hand, the digital camera and the laser radar are not rigidly connected, and the installation angle of the digital camera differs in each installation process, so the automatic calibration of the digital camera is difficult. SUMMARY

[0005] The main purpose of the present application is to provide a real three-dimensional model construction method, device and vehicle-mounted mobile measurement system based on a vehicle-mounted mobile measurement system, aiming at solving the technical problem that the current real three-dimensional model construction effect is poor.

[0006] To achieve the above-mentioned purpose, the present application provides a real three-dimensional model construction method based on a vehicle-mounted mobile measurement system. The vehicle-mounted mobile measurement system comprises an inertial navigation system, a GNSS receiver, an odometer, a panoramic camera, a plurality of digital cameras, a plurality of three-dimensional laser radars, a positioning antenna and a synchronous controller. The odometer is installed on the wheel, the plurality of digital cameras are installed around the vehicle, the plurality of three-dimensional laser radars are installed in front of and on both sides of the vehicle, and the panoramic camera and the plurality of three-dimensional laser radars are rigidly connected.

[0007] The real three-dimensional model construction method based on the vehicle-mounted mobile measurement system comprises:

[0008] After the time synchronization of the inertial navigation system, the GNSS receiver, the odometer, the panoramic camera, the plurality of three-dimensional laser radars, the plurality of digital cameras and the positioning antenna is completed, the acceleration and angular velocity are collected by the inertial navigation system;

[0009] The mileage information of the vehicle forward movement is collected based on the odometer;

[0010] The first position and the first attitude of the vehicle carrier which are not optimized are calculated according to the acceleration, the angular velocity and the mileage information;

[0011] The first position and the first attitude are optimized based on the three-dimensional laser radar, and the optimized pose information is obtained;

[0012] calculating three-dimensional point cloud data geographical coordinates and panoramic image poses according to the optimized pose information;

[0013] processing the three-dimensional point cloud data geographical coordinates through the panoramic image poses to obtain target three-dimensional point cloud data, and calculating a digital camera image pose through the panoramic image poses;

[0014] constructing a real scene three-dimensional model through the target three-dimensional point cloud data and the digital camera image pose.

[0015] In an embodiment, the step of calculating a first position and a first attitude of a vehicle carrier that are not optimized according to the acceleration, the angular velocity, and the mileage information comprises:

[0016] controlling the positioning antenna and the GNSS receiver to perform loop tracking based on the inertial navigation to obtain navigation satellite pseudo-range and carrier phase observation data;

[0017] performing navigation calculation on the navigation satellite pseudo-range and carrier phase observation data to obtain a geographical coordinate system position;

[0018] correcting the inertial navigation based on the geographical coordinate system position to obtain a corrected inertial navigation, and obtaining a first corrected acceleration and a first corrected angular velocity collected by the corrected inertial navigation;

[0019] performing tight combination filtering estimation on the first corrected acceleration, the first corrected angular velocity, and the mileage information to obtain a first position and a first attitude of a vehicle carrier that are not optimized.

[0020] In an embodiment, the plurality of three-dimensional laser radars comprises a first laser radar;

[0021] The step of optimizing the first position and the first attitude based on the three-dimensional laser radars to obtain optimized pose information comprises:

[0022] correcting the acceleration and the angular velocity accumulated over time by the inertial navigation through the first position and the first attitude to obtain a second corrected acceleration and a second corrected angular velocity;

[0023] collecting first laser point cloud data through the first laser radar;

[0024] performing feature matching on the first laser point cloud data to obtain a vehicle motion state;

[0025] optimizing the second corrected acceleration and the second corrected angular velocity in loose combination filtering through the vehicle motion state to obtain optimized pose information.

[0026] In an embodiment, the plurality of three-dimensional laser radars comprises a second laser radar, and the number of laser beams scanned by the second laser radar is greater than the number of laser beams scanned by the first laser radar;

[0027] The step of calculating the geographic coordinates of the three-dimensional point cloud data and the pose of the panoramic image according to the optimized pose information comprises:

[0028] The second laser three-dimensional point cloud data is acquired by the second laser radar;

[0029] The second laser three-dimensional point cloud data is subjected to multi-echo elimination and noise removal to obtain processed second laser three-dimensional point cloud data;

[0030] The processed second laser three-dimensional point cloud data is subjected to feature extraction and semantic processing by a deep learning algorithm to obtain geometric information and spatial structure information of building objects in a three-dimensional scene;

[0031] The geographic coordinates of the three-dimensional point cloud data are calculated by the optimized pose information, the geometric information, and the spatial structure information;

[0032] The panoramic image acquired by the panoramic camera is obtained;

[0033] The time stamp of each frame of the panoramic image is aligned with the time sequence of the optimized pose information, and the optimized pose information before and after the time stamp of the panoramic image is calculated by interpolation to obtain the pose of the panoramic image.

[0034] In an embodiment, the step of extracting features and performing semantic processing on the processed second laser three-dimensional point cloud data by a deep learning algorithm to obtain geometric information and spatial structure information of building objects in a three-dimensional scene comprises:

[0035] The processed second laser three-dimensional point cloud data is subjected to feature extraction by a deep learning algorithm to obtain point cloud features;

[0036] The point cloud features are used to perform semantic labeling on points or grid elements in a three-dimensional scene to obtain different semantic categories;

[0037] The regions of different semantic categories are segmented by a deep learning model to obtain segmented regions of different semantic categories;

[0038] The geometric information and spatial structure information of building objects in a three-dimensional scene are determined by the segmented regions of different semantic categories.

[0039] In an embodiment, the step of processing the geographic coordinates of the three-dimensional point cloud data by the pose of the panoramic image to obtain target three-dimensional point cloud data comprises:

[0040] Obtaining a conversion relationship between a panoramic image coordinate system and a three-dimensional point cloud data coordinate system;

[0041] Performing panoramic image coordinate conversion according to the conversion relationship, the panoramic image pose, and geographical coordinates of the three-dimensional point cloud data, to color the three-dimensional point cloud data by the panoramic image, and obtain target three-dimensional point cloud data.

[0042] In an embodiment, the plurality of digital cameras includes a first group of digital cameras and a second group of digital cameras, the number of cameras in the first group of digital cameras is the same as that in the second group of digital cameras, and the first group of digital cameras and the second group of digital cameras alternately capture images;

[0043] Before the step of calculating the digital camera image pose according to the panoramic image pose, the method further includes:

[0044] Controlling the first group of digital cameras and the second group of digital cameras to alternately capture images, and obtaining a placement distance of the cameras along a vehicle advancing direction, a camera field of view angle, an outward rotation angle, an imaging size, a capture distance, and a capture vehicle speed during the capturing process;

[0045] Calculating an image horizontal coverage value according to the camera field of view angle, the outward rotation angle, and the capture distance;

[0046] Calculating an image vertical coverage value according to the image horizontal coverage value and the imaging size;

[0047] Calculating an overlap degree between the first group of digital cameras and the second group of digital cameras according to the image horizontal coverage value, the image vertical coverage value, the placement distance, and the capture vehicle speed;

[0048] When the overlap degree is less than a preset overlap degree, reducing the capture vehicle speed, and marking images corresponding to the overlap degree less than the preset overlap degree;

[0049] Controlling the first group of digital cameras and the second group of digital cameras to alternately capture images at the reduced capture vehicle speed.

[0050] In an embodiment, the step of calculating the digital camera image pose according to the panoramic image pose includes:

[0051] Obtaining a plurality of panoramic images according to the panoramic image pose;

[0052] Obtaining digital camera images alternately captured by the first group of digital cameras and the second group of digital cameras;

[0053] Coarsely registering the plurality of panoramic images and the plurality of digital camera images according to timestamps, and extracting first feature points in the panoramic images and corresponding second feature points in the digital camera images;

[0054] match the first feature points and the second feature points by a feature matching algorithm to obtain a first matching feature point pair;

[0055] obtain a point cloud feature point corresponding to a second feature point of the digital camera according to a conversion relationship between a panoramic image coordinate system and a three-dimensional point cloud data coordinate system and the first matching feature point pair;

[0056] obtain a second matching feature point pair according to the second feature point and the corresponding point cloud feature point;

[0057] calculate a transformation matrix from a digital image coordinate system to a three-dimensional point cloud data coordinate system according to the second matching feature point pair;

[0058] obtain a digital camera image pose through the transformation matrix.

[0059] In addition, to achieve the above object, the application further provides a real scene three-dimensional model construction device based on a vehicle-mounted mobile measurement system, which comprises:

[0060] a time synchronization module, configured to collect acceleration and angular velocity through an inertial navigation system after time synchronization of the inertial navigation system, a GNSS receiver, an odometer, a panoramic camera, a plurality of three-dimensional laser radars, a plurality of digital cameras and a positioning antenna is completed;

[0061] a collection module, configured to collect mileage information of vehicle advancement based on the odometer;

[0062] a calculation module, configured to calculate a first position and a first attitude of a vehicle carrier which are not optimized according to the acceleration, the angular velocity and the mileage information;

[0063] an optimization module, configured to optimize the first position and the first attitude based on the three-dimensional laser radars to obtain optimized pose information;

[0064] the calculation module is further configured to calculate three-dimensional point cloud data geographical coordinates and panoramic image pose according to the optimized pose information;

[0065] a processing module, configured to process the three-dimensional point cloud data geographical coordinates through the panoramic image pose to obtain target three-dimensional point cloud data, and calculate a digital camera image pose through the panoramic image pose;

[0066] a construction module, configured to construct a real scene three-dimensional model through the target three-dimensional point cloud data and the digital camera image pose.

[0067] In addition, in order to achieve the above object, the application further provides a vehicle-mounted mobile measurement system, which comprises an inertial navigation system, a GNSS receiver, an odometer, a panoramic camera, a plurality of digital cameras, a plurality of three-dimensional laser radars, a positioning antenna and a synchronization controller, wherein the odometer is installed on a wheel, the plurality of digital cameras are installed around a vehicle, the plurality of three-dimensional laser radars are installed in front of and on both sides of the vehicle, the panoramic camera and the plurality of three-dimensional laser radars are rigidly connected, and the vehicle-mounted mobile measurement system is configured to realize the steps of the real-scene three-dimensional model construction method based on the vehicle-mounted mobile measurement system.

[0068] In addition, in order to achieve the above object, the application further provides a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium, and the computer program is executed by a processor to realize the steps of the real-scene three-dimensional model construction method based on the vehicle-mounted mobile measurement system.

[0069] In addition, in order to achieve the above object, the application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to realize the steps of the real-scene three-dimensional model construction method based on the vehicle-mounted mobile measurement system.

[0070] The method of the application has the following beneficial effects relative to the prior art:

[0071] 1) The application designs a vehicle-mounted mobile measurement system and method for real-scene three-dimensional construction. By highly integrating inertial navigation, positioning antenna and receiver, odometer, panoramic camera, digital camera and three-dimensional laser radar, automatic calibration of targetless laser radar-digital camera is realized.

[0072] 2) The application can more accurately estimate the initial pose (position and attitude) of the vehicle by synchronously collecting acceleration, angular velocity and vehicle forward information, and make up for the deficiency of a single sensor in a complex environment. The three-dimensional point cloud geographic coordinates are calculated based on the optimized pose: the generated point cloud data has real geographic position information, and provides reliable geographic reference for subsequent modeling. The point cloud data is processed in combination with panoramic image pose: the spatial consistency is further enhanced, and the point cloud data is more consistent with the actual scene structure.

[0073] 3) by obtaining the conversion relationship between the panoramic image coordinate system and the three-dimensional point cloud data coordinate system; and according to the conversion relationship, the panoramic image pose and the three-dimensional point cloud data geographic coordinates, the panoramic image coordinate conversion is carried out, so as to color the three-dimensional point cloud data by the panoramic image, obtain the target three-dimensional point cloud data, realize the true color attribute assignment of the three-dimensional point cloud data, and realize the efficient collection and processing of the true color three-dimensional laser point cloud data and high-resolution and high-overlap image data with high-precision pose attribute in a complex scene;

[0074] 4) The image and three-dimensional point cloud collected and processed by the laser radar both have high-precision pose attribute, and the digital camera is arranged to alternately collect digital camera images. The image collected and processed by the digital camera image grouping has the characteristics of high resolution and high overlap, can truly reflect the street scene road and both sides of the landscape, building facade information, realize the two-dimensional or three-dimensional expression of the three-dimensional model to the real world, facilitate the flexible operation of the operator, and meet the urgent application needs of the real scene three-dimensional construction data collection and processing. BRIEF DESCRIPTION OF DRAWINGS

[0075] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the application.

[0076] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0077] Figure 1 The flowchart provided by the first embodiment of the real scene three-dimensional model construction method based on the vehicle-mounted mobile measurement system of the present application;

[0078] Figure 2 The structural schematic diagram of the vehicle-mounted mobile measurement system;

[0079] Figure 3 The flowchart provided by the second embodiment of the real scene three-dimensional model construction method based on the vehicle-mounted mobile measurement system of the present application;

[0080] Figure 4 The flowchart provided by the third embodiment of the real scene three-dimensional model construction method based on the vehicle-mounted mobile measurement system of the present application;

[0081] Figure 5 The brief flowchart provided by the first embodiment of the real scene three-dimensional model construction method based on the vehicle-mounted mobile measurement system;

[0082] Figure 6This is a schematic diagram of the module structure of the real-scene 3D model construction device based on the vehicle-mounted mobile measurement system in an embodiment of this application.

[0083] Reference numerals: 4. Positioning antenna; 5. Synchronization controller;

[0084] Panoramic camera Q1, first group of digital cameras A1-A3, second group of digital cameras B1-B3, first lidar L2, second lidar L1.

[0085] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0086] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0087] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0088] The main solution of this application embodiment is as follows: After time synchronization of the inertial navigation system, GNSS receiver, odometer, panoramic camera, multiple 3D LiDARs, multiple digital cameras, and positioning antenna, acceleration and angular velocity are collected through the inertial navigation system; mileage information of the vehicle's forward movement is collected based on the odometer; the unoptimized first position and first attitude of the vehicle carrier are calculated based on the acceleration, angular velocity, and mileage information; the first position and first attitude are optimized based on the 3D LiDARs to obtain optimized pose information; the geographic coordinates of 3D point cloud data and the pose of panoramic image are calculated based on the optimized pose information; the geographic coordinates of the 3D point cloud data are processed through the panoramic image pose to obtain target 3D point cloud data, and the pose of digital camera images is calculated through the panoramic image pose; a real-scene 3D model is constructed through the target 3D point cloud data and the pose of digital camera images.

[0089] Current GNSS satellite signal and inertial navigation combined positioning technology can only achieve high-precision positioning and attitude determination under conditions of open field of view and no significant influence from the surrounding environment. In complex urban scenarios, GNSS satellite signal obstruction, high-power radio interference from microwave towers and transmitting antennas, and multi-path reflections from large areas of water, metal sheds, and metallic areas severely restrict high-precision positioning and attitude determination. Loss of GNSS satellite signal lock leads to the inability to correct inertial navigation and odometer drift, causing positioning accuracy to drop rapidly to a few meters to tens of meters, which cannot meet application requirements.

[0090] This application provides a solution, designing a vehicle-mounted mobile measurement system and method for real-scene 3D construction. It highly integrates sensors such as inertial navigation, positioning antenna and receiver, odometer, panoramic camera, digital camera, and 3D LiDAR, enabling automatic calibration of the targetless LiDAR-digital camera. It can efficiently acquire and process true-color 3D LiDAR point cloud data with high-precision pose attributes and high-resolution, high-overlap image data in complex scenes. The system platform adopts an intelligent modular design, facilitating flexible operation and meeting the urgent application needs of real-scene 3D construction data acquisition and processing.

[0091] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or vehicle-mounted mobile measurement system capable of performing the above functions. The following description uses a vehicle-mounted mobile measurement system as an example to illustrate this embodiment and the subsequent embodiments.

[0092] Based on this, embodiments of this application provide a method for constructing a real-scene 3D model based on a vehicle-mounted mobile measurement system, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the real-scene 3D model construction method based on a vehicle-mounted mobile measurement system of this application.

[0093] In this embodiment, as Figure 2 As shown, Figure 2 This is a schematic diagram of a vehicle-mounted mobile measurement system. The system includes: an inertial navigation system (not shown), a GNSS receiver (not shown), and an odometer (not shown). The inertial navigation system, GNSS receiver, and odometer are located under the vehicle body. It also includes a panoramic camera Q1, multiple digital cameras (A1, A2, A3, B1, B2, B3), multiple 3D LiDARs (L1 and L2), a positioning antenna 4, and a synchronization controller 5. The odometer is mounted on the wheels, the digital cameras are mounted around the vehicle, and the 3D LiDARs are mounted at the front and on the left and right sides of the vehicle. The panoramic camera Q1 and the 3D LiDARs are rigidly connected. It should be noted that digital cameras A1-A3 and B1-B3 are correspondingly located on the sides of the vehicle.

[0094] In this embodiment, the method for constructing a real-scene 3D model based on a vehicle-mounted mobile measurement system includes steps S10 to S70:

[0095] Step S10: After the time synchronization of the inertial navigation system, the GNSS receiver, the odometer, the panoramic camera, the multiple 3D LiDARs, the multiple digital cameras, and the positioning antenna is completed, the acceleration and angular velocity are collected by the inertial navigation system.

[0096] It should be noted that the vehicle-mounted mobile measurement system also includes a time-frequency signal driving device, which, together with the synchronization controller 5, constitutes a time synchronization functional module. Before constructing the real-scene 3D model, to improve the accuracy of data acquisition, all sensors can be synchronized in time. Therefore, the inertial navigation system, the GNSS receiver, the odometer, the panoramic camera, multiple 3D LiDARs, multiple digital cameras, and the positioning antenna can be synchronized in time through the time synchronization functional module composed of the time-frequency signal driving device and the synchronization controller 5. Specifically, the sensors of each functional module are first triggered, and then the panoramic camera is controlled to acquire each frame of image. After acquisition, the digital cameras are triggered to acquire digital images in groups through the synchronization controller, and the timestamps returned by each sensor are recorded. The returned timestamps are checked, and invalid timestamps are discarded, thereby completing the time synchronization.

[0097] In practical implementation, after the time synchronization of all sensors is completed, subsequent positioning and attitude determination can be performed. Specifically, the inertial navigation system is a high-precision inertial navigation system, and the number of laser beams scanned by the second and first lidars differs; for example, the first lidar is a 16-line / second lidar, while the second lidar is a 250-line / second lidar. The positioning and attitude determination is mainly composed of and implemented by a high-precision inertial navigation system, a set of positioning antennas and receivers, a set of odometers, and a 16-line / second lidar, and is used for positioning and attitude determination of the vehicle carrier.

[0098] Therefore, after time synchronization is completed, 200Hz high-frequency acceleration and angular velocity information can be collected through inertial navigation, thereby enabling the calculation of the position and attitude of the inertial navigation sensor.

[0099] Step S20: Collect the mileage information of the vehicle based on the odometer.

[0100] It should be noted that during the vehicle's operation, the odometer wheel speed device counts and collects the distance the vehicle has traveled, smoothing the vehicle's trajectory to obtain the distance traveled.

[0101] Step S30: Calculate the unoptimized first position and first attitude of the vehicle carrier based on the acceleration, the angular velocity and the mileage information.

[0102] It should be understood that the position and attitude of the vehicle can be estimated using fusion algorithms, such as Kalman filtering or particle filtering, based on acceleration, angular velocity, and odometer information, resulting in an unoptimized first position and first attitude. Acceleration and angular velocity information reflect the dynamic changes of the vehicle, while odometer information provides the absolute displacement information of the vehicle during travel. Combining these three factors can improve the accuracy of positioning and attitude determination.

[0103] In one feasible implementation, step S30 may include steps A11 to A14:

[0104] Step A11: Based on the inertial navigation system, perform loop tracking on the positioning antenna and the GNSS receiver to obtain navigation satellite pseudorange and carrier phase observation data;

[0105] It should be noted that inertial navigation systems (INS) can assist positioning antennas and GNSS receivers in loop tracking. In vehicle-mounted mobile measurement systems, INS, as one of the core sensors, provides real-time acceleration and angular velocity information of the vehicle. This information is used to assist positioning antennas and GNSS receivers in accurate loop tracking. Loop tracking is a crucial step in GNSS navigation, involving the reception and processing of signals transmitted by navigation satellites to obtain pseudorange and carrier phase observation data. This data forms the basis for subsequent navigation calculations and positioning corrections.

[0106] Step A12: Perform navigation calculations on the pseudorange and carrier phase observation data of the navigation satellites to obtain geographic coordinate system positioning;

[0107] In practice, after acquiring pseudorange and carrier phase observation data from navigation satellites, the vehicle-mounted motion measurement system uses this data for navigation calculation. Navigation calculation is a complex process involving multiple steps, including data processing, error correction, and coordinate system transformation. Ultimately, the navigation calculation outputs the vehicle's position information in the geographic coordinate system, which is crucial for real-time vehicle positioning.

[0108] Navigation solutions may specifically include:

[0109] (1) Data preprocessing: Cycle slip detection and repair are performed on the raw observation data output by the GNSS receiver to eliminate the influence of multipath effect.

[0110] An ionospheric-free combined observation equation was constructed using dual-frequency observations:

[0111] P_{IF} =frac{f_1^2 P_1 - f_2^2 P_2}{f_1^2 - f_2^2}

[0112] Where P_1 and P_2 are the pseudorange observations at frequencies L1 and L2, respectively, and f_1 and f_2 are the corresponding frequencies;

[0113] (2) Error correction: Ionospheric delay: eliminated by Klobuchar model or dual-frequency observations; Tropospheric delay: corrected by Saastamoinen model; Satellite orbit error: compensated by precise ephemeris data; Receiver clock error: solved by adjustment of observation equations;

[0114] (3) Positioning solution: Constructing pseudorange observation equations:

[0115] rho =sqrt{(X_{sv}-X)^2 + (Y_{sv}-Y)^2 + (Z_{sv}-Z)^2} + c·δt +ε

[0116] Where (X_{sv},Y_{sv},Z_{sv}) are satellite coordinates, (X,Y,Z) are GNSS receiver coordinates, δt is clock error, ε is residual term, and c is speed of light;

[0117] (4) Coordinate transformation: The positioning results in the WGS84 coordinate system are obtained by least squares method, and then transformed into the local geographic coordinate system by Gauss projection.

[0118] By performing navigation calculations on pseudorange and carrier phase observation data from navigation satellites, geographic coordinate system positioning is obtained. The inertial navigation system outputs 200Hz high-frequency pose data. GNSS navigation satellite data calculation can provide absolute geographic coordinate system positioning and correct inertial navigation system drift.

[0119] Step A13: Correct the inertial navigation system based on the geographic coordinate system positioning to obtain the corrected inertial navigation system, and acquire the first corrected acceleration and the first corrected angular velocity collected by the corrected inertial navigation system;

[0120] Since inertial navigation systems may accumulate errors after long-term operation, they need to be corrected using external information.

[0121] In this embodiment, the inertial navigation system (INS) is corrected using geographic coordinate system positioning information. The corrected INS will have higher accuracy and stability, and can more accurately reflect the motion state of the vehicle. Simultaneously, the corrected INS will collect a first corrected acceleration and a first corrected angular velocity; these data will be used for subsequent compact combination filtering estimation.

[0122] Step A14: Perform tight combination filtering estimation on the first corrected acceleration, the first corrected angular velocity, and the mileage information to obtain the unoptimized first position and first attitude of the vehicle carrier.

[0123] Compact combination filtering estimation is an advanced data processing method that can fuse data from multiple sensors to improve positioning accuracy and stability.

[0124] In this embodiment, the first corrected acceleration, the first corrected angular velocity, and the odometer information are estimated using a compact combination filter. This method fully utilizes the complementarity of various sensor data to improve the accuracy of vehicle positioning. Finally, the compact combination filter estimation outputs the unoptimized first position and first attitude of the vehicle, which will be used for subsequent optimization and construction of a realistic 3D model.

[0125] This embodiment designs a combined tight-combination and loose-combination processing method. The inertial navigation system outputs 200Hz high-frequency pose data, and GNSS navigation satellite data processing can provide absolute geographic coordinate system positioning and correct inertial navigation drift. The odometer can prevent abrupt changes in the vehicle system's movement trajectory and maintain trajectory smoothness. By fusing the raw data from inertial navigation, GNSS, and odometer through a tight-combination filter, unoptimized positioning and attitude data and a variance-covariance matrix representing accuracy are calculated.

[0126] Through the above steps, real-time and accurate positioning and attitude calculation of the vehicle can be achieved, providing strong support for the subsequent construction of a real-scene 3D model.

[0127] Step S40: Optimize the first position and the first pose based on the three-dimensional lidar to obtain optimized pose information.

[0128] In practical implementation, laser point cloud data can be collected by three-dimensional LiDAR, and the vehicle motion state can be estimated based on the collected laser point cloud data. Based on the vehicle motion state, the vehicle position and attitude information can be further optimized in loose combination filtering to improve the accuracy of pose information in complex scenes and output optimized high-frequency and high-precision pose information.

[0129] In one feasible implementation, step S40 may include steps A21 to A24:

[0130] Multiple 3D lidars include a first lidar, which is a 16-line / second lidar.

[0131] Step A21: Correct the acceleration and angular velocity accumulated by the inertial navigation system over time using the first position and the first attitude to obtain the second corrected acceleration and the second corrected angular velocity;

[0132] It should be understood that inertial navigation system (INS) drift can cause positioning errors due to offset. Therefore, the INS drift error can be compensated in real time using the first position and first attitude output by the GNSS / INS tight combination. For example, an error state equation can be established: δx_{k+1}=Φ_k·δx_k + w_k, where Φ_k is the state transition matrix and w_k is the process noise. Thus, the cumulative acceleration and angular velocity drift errors of the INS over time can be corrected using the error state equation, resulting in corrected acceleration and angular velocity, i.e., the second corrected acceleration and the second corrected angular velocity.

[0133] Step A22: Collect first laser point cloud data using the first lidar;

[0134] The first lidar (16 lines / second) acquires three-dimensional point cloud data at a frequency of 20Hz, with a point cloud density of 1600 points / square meter, thus obtaining the first lidar point cloud data.

[0135] Step A23: Perform feature matching on the first laser point cloud data to obtain the vehicle motion state;

[0136] It should be noted that feature matching can be performed on the point cloud data between frames in the first laser point cloud data to estimate the vehicle's motion state. Specifically, a feature extraction method based on curvature values ​​can be used to extract the corner points and planar features of ground landmarks from the point cloud, and a feature matching constraint condition can be constructed: Σ(θ_i·(n_i^T·(R·p_i + t - q_i)))^2<ε1, where n_i is the normal vector, θ_i is the weight coefficient, and ε1 is the matching threshold or error tolerance. By performing feature matching on the point cloud data between frames through the feature matching constraint condition, the pixel displacement and motion trajectory can be calculated by tracking the positional changes of the same physical feature point in consecutive image frames. Using geometric transformation models (such as rigid body motion and perspective transformation), the translational and rotational speeds and directions of the vehicle can be solved, thereby inferring the overall motion state of the vehicle.

[0137] Step A24: Optimize the second corrected acceleration and the second corrected angular velocity in the loose combination filter based on the vehicle motion state to obtain the optimized pose information.

[0138] In practical implementation, the vehicle motion state estimated from point cloud data can be further optimized in a loosely combined filter to improve the vehicle's position and attitude information. This allows for pose optimization using LiDAR data, building upon the unoptimized pose obtained through time synchronization and tightly combined filtering. Loosely combined filtering optimizes positioning and attitude data, improving reliability in complex GNSS positioning loss scenarios. The combined tight and loose processing method maintains high positioning and attitude accuracy while considering the high complexity of LiDAR data processing, ensuring real-time processing efficiency and enabling the acquisition of high-precision pose information from the vehicle system in complex environments.

[0139] Step S50: Calculate the geographic coordinates of the 3D point cloud data and the pose of the panoramic image based on the optimized pose information.

[0140] It's important to note that point cloud data is a collection of points in three-dimensional space obtained through sensors such as LiDAR or stereo vision. Based on optimized pose information, the coordinates of these points can be transformed from the sensor's local coordinate system to a global geographic coordinate system (such as the Earth coordinate system or a projected coordinate system). The purpose of this is to map the 3D point cloud data to its real-world location for subsequent processing.

[0141] Panoramic images are typically created by capturing a series of images with a camera and stitching them together. Optimized pose information also helps determine the shooting position and orientation of the panoramic image, i.e., the image's pose. This information is crucial for subsequent registration of the images and 3D point cloud data.

[0142] Step S60: Process the geographic coordinates of the 3D point cloud data using the panoramic image pose to obtain the target 3D point cloud data, and calculate the digital camera image pose using the panoramic image pose.

[0143] Geometric registration is performed by associating the pose of panoramic images with the geographic coordinates of 3D point cloud data. In other words, the pose information of the panoramic image is combined with the 3D point cloud data to ensure that these data are within the same spatial reference frame. For example, if the coordinates of the point cloud data are at a specific location, while the panoramic image was taken at another location, calibration and coordinate transformation are used to map them to the same scene. After panoramic image pose processing, the result is calibrated, adjusted, and colorized 3D point cloud data. These point clouds may have undergone filtering, noise reduction, registration, and colorization processes to generate true-color 3D point cloud data, which can better represent the 3D spatial structure of the real world.

[0144] In practice, the pose information from panoramic images can be used to further calculate the position and orientation of the digital camera during shooting. This is to map the camera's shooting information to the three-dimensional scene, enabling the construction of a real-world model through the combination of images and three-dimensional point clouds.

[0145] Step S70: Construct a real-scene 3D model using the target 3D point cloud data and the digital camera image pose.

[0146] A realistic 3D model is created by combining target 3D point cloud data with digital camera image poses. The 3D point cloud provides the spatial geometry of the scene, while the digital camera image poses provide the corresponding image texture information. The point cloud data describes the spatial position of each point in the scene, which forms the geometry of the scene. The poses of the digital camera images help determine the position and orientation of these images in 3D space.

[0147] By combining point cloud data and image pose, point cloud rendering and texture mapping techniques are typically used to project the texture information of the image onto the point cloud data. In this way, the generated 3D model not only has geometric shape but also reflects realistic visual effects and presents rich details.

[0148] This embodiment provides a method for constructing a real-scene 3D model based on an onboard mobile measurement system. By synchronously acquiring acceleration, angular velocity, and vehicle forward information, it can more accurately estimate the vehicle's initial pose (position and attitude), compensating for the shortcomings of a single sensor in complex environments. The method calculates the geographic coordinates of the 3D point cloud based on the optimized pose: ensuring that the generated point cloud data has realistic geographic location information, providing a reliable geographic reference for subsequent modeling. Combining panoramic image pose processing with the point cloud data further enhances spatial consistency, making the point cloud data more closely match the actual scene structure. The pose of the digital camera image is derived from the panoramic image: ensuring the spatiotemporal consistency between the image and the 3D point cloud, avoiding geometric distortion caused by asynchrony. Joint modeling of the target 3D point cloud and image pose achieves precise matching of texture and geometry, constructing a high-quality, highly realistic real-scene 3D model.

[0149] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 Step S50 includes steps S501 to S506:

[0150] In this embodiment, the three-dimensional lidar includes a second lidar, which is a 250 lines / second lidar, and the number of laser beams scanned by the second lidar is greater than the number of laser beams scanned by the first lidar.

[0151] Step S501: Collect second laser three-dimensional point cloud data using the second lidar.

[0152] It should be noted that the main function of the second lidar is point cloud processing. The second lidar can collect high-density three-dimensional point cloud data, namely the second lidar three-dimensional point cloud data.

[0153] Step S502: Perform multi-echo cancellation and noise removal on the second laser three-dimensional point cloud data to obtain the processed second laser three-dimensional point cloud data.

[0154] In practice, after acquiring the second laser three-dimensional point cloud data, the acquired point cloud data can be subjected to multi-echo cancellation and noise removal, thereby improving the accuracy of the acquired point cloud data.

[0155] Because the laser beam emitted by a lidar can be reflected by multiple objects, multiple reflected signals (echoes) are generated. Multi-echo cancellation technology is used to filter out valid echoes from different echoes, removing interfering echoes from irrelevant objects (such as reflections from the background or moving objects). Typically, the lidar selects the strongest or earliest echo, depending on the specific application. Point cloud data may contain some errors or irregular noise, such as erroneous data caused by environmental interference or sensor malfunctions. Noise removal uses algorithms to remove useless data points, retaining only information reflecting the true environment.

[0156] By using multi-echo cancellation and noise removal, more accurate point cloud data with noise and invalid echoes removed is obtained, which is more consistent with the actual physical environment.

[0157] Step S503: Use deep learning algorithms to extract features and perform semantic processing on the processed second laser 3D point cloud data to obtain the geometric and spatial structure information of building objects in the 3D scene.

[0158] Deep learning algorithms (such as Convolutional Neural Networks (CNN), PointNet, PointNet++, etc.) can be applied to 3D point cloud data to automatically identify and extract features. Through training, the algorithms can identify different objects in the point cloud data, such as buildings, roads, and trees.

[0159] Therefore, deep learning algorithms can be used to extract features and perform semantic processing on the processed second laser 3D point cloud data. Based on deep learning methods, feature extraction and automatic semantic recognition and classification of buildings in the 3D scene can be performed, thereby realizing the automatic extraction of geometric and spatial structural information of building objects in the 3D scene.

[0160] In one feasible implementation, step S503 may include steps B11~B1:4:

[0161] Step B11: Extract features from the processed second laser 3D point cloud data using a deep learning algorithm to obtain point cloud features;

[0162] In practical implementation, feature extraction can be performed on disordered laser point clouds using deep learning methods based on PointNet and PointNet++ point cloud processing networks to obtain point cloud features.

[0163] Point cloud features include geometric features and spatial structure features. Geometric features reflect the geometric shape and local structure of the point cloud, expressing its local characteristics. Spatial structure features reflect the topological and hierarchical structure of buildings in a 3D scene.

[0164] Step B12: Based on the point cloud features, perform semantic annotation on the points or mesh units in the 3D scene to obtain different semantic categories;

[0165] It should be noted that semantic annotation can be performed on each point or mesh unit in a 3D scene based on the feature information of the point cloud, classifying them into different semantic categories, such as buildings, roads, and vegetation. The purpose of semantic processing is to assign a label to each point or point cloud region, defining their specific meaning (such as walls, windows, doors, ceilings, etc.), thereby obtaining semantic information about buildings or other objects in the environment.

[0166] Step B13: Segment the regions of different semantic categories using a deep learning model to obtain the segmented regions of different semantic categories;

[0167] Understandably, deep learning models can be trained on large labeled datasets. They can automatically identify and segment regions of different semantic categories, thus obtaining segmented regions of different semantic categories. Furthermore, classification performance can be evaluated based on semantic segmentation metrics and instance segmentation metrics.

[0168] Step B14: Determine the geometric and spatial structural information of architectural objects in the 3D scene by segmenting different semantic category regions.

[0169] It's important to note that the segmented regions allow for independent analysis of each semantic category, enabling the extraction of its geometric information. This includes, for example, the shape, size, and relative positions of building walls, windows, roofs, etc. Geometric information typically includes the spatial location, dimensions, and angles of point clouds, accurately describing the building's shape. Spatial structural information describes the layout and relationships of buildings or other objects in three-dimensional space. This includes the relative positions, orientations, and distances between objects. By analyzing the relative positions of regions from different semantic categories, the system can understand the building's spatial structure, such as the relationships between walls and windows, and the size and layout of rooms.

[0170] Step S504: Calculate the geographic coordinates of the 3D point cloud data using the optimized pose information, the geometric information, and the spatial structure information.

[0171] Pose includes the spatial position (coordinates) and orientation (direction) of the camera or radar. By using optimized pose information and geometric structure data, point cloud data can be transformed from a local coordinate system to a global geographic coordinate system (such as the WGS-84 coordinate system). This allows us to obtain the actual position of the 3D point cloud on Earth, i.e., the geographic coordinates of the 3D point cloud data.

[0172] Step S505: Acquire the panoramic image captured by the panoramic camera.

[0173] A panoramic camera captures images from multiple perspectives and stitches them together to create a complete 360-degree panoramic image. In this step, the images acquired by the panoramic camera, typically a 360-degree view covering the scene, are used for subsequent image and 3D data fusion. For example, a panoramic camera might take photos at a fixed distance and stitch together images from six lenses to create a 360° panoramic image.

[0174] Step S506: Align the timestamp of each frame of the panoramic image with the time sequence of the optimized pose information, and perform interpolation calculation on the optimized pose information before and after the timestamp of the panoramic image to obtain the panoramic image pose.

[0175] Each frame of the panoramic image has a corresponding timestamp, indicating when it was captured. The optimized pose information also has a timestamp, indicating the sensor's position and orientation at a specific point in time. To accurately align the panoramic image with the 3D point cloud data, the timestamps of the panoramic image and pose data need to be matched. By aligning the timestamps, it can be ensured that the image and pose data are synchronized at the same time.

[0176] The time synchronization module unifies the time reference between the positioning and pose determination module and the panoramic camera. Therefore, it can align a high-precision 200Hz pose time series based on the timestamp of each panoramic image frame. Interpolation calculations are then performed using high-precision pose information from multiple epochs before and after the panoramic image timestamp, employing spline interpolation. Since the timestamps of the panoramic image and the pose information may not perfectly match, there may be temporal discrepancies. To obtain accurate panoramic image pose, an interpolation algorithm (such as linear interpolation) is needed to calculate the pose information between consecutive time points, thereby estimating the precise pose at a specific time point. This process ensures that the camera position and orientation at the time of image capture are accurately reflected during image and point cloud data fusion.

[0177] This embodiment acquires second laser 3D point cloud data using a second LiDAR; performs multi-echo cancellation and noise removal on the second laser 3D point cloud data to obtain processed second laser 3D point cloud data; uses a deep learning algorithm to extract features and perform semantic processing on the processed second laser 3D point cloud data to obtain the geometric and spatial structure information of building objects in the 3D scene; calculates the geographic coordinates of the 3D point cloud data using optimized pose information, the geometric information, and the spatial structure information; acquires panoramic images captured by the panoramic camera; aligns the timestamp of each frame of the panoramic image with the time sequence of the optimized pose information, and interpolates the optimized pose information before and after the timestamp of the panoramic image to obtain the panoramic image pose. By combining LiDAR data, deep learning algorithms, panoramic images, and optimized pose information, a precise 3D scene model is constructed. LiDAR provides detailed 3D point cloud data, deep learning can identify objects such as buildings in the scene, and panoramic images provide visual texture for the model. Through precise pose calibration, all data can be fused to generate an accurate 3D geographic model.

[0178] Based on the first embodiment of this application, in the third embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 Step S60 includes steps S601 to S610:

[0179] Step S601: Obtain the transformation relationship between the panoramic image coordinate system and the 3D point cloud data coordinate system.

[0180] It should be noted that the panoramic camera and the laser sensor are rigidly connected, so the transformation relationship between the panoramic image coordinate system and the 3D point cloud data coordinate system is fixed. The factory-provided calibration relationship can be reused to perform panoramic image coordinate transformation. Therefore, the transformation relationship between the panoramic image coordinate system and the 3D point cloud data coordinate system, i.e., the calibration relationship, can be obtained.

[0181] Step S602: Perform panoramic image coordinate transformation according to the transformation relationship, the panoramic image pose, and the geographic coordinates of the three-dimensional point cloud data, so as to colorize the three-dimensional point cloud data through the panoramic image and obtain the target three-dimensional point cloud data.

[0182] In practice, the coordinate transformation of the panoramic image can be performed based on the transformation relationship, the pose of the panoramic image, and the geographic coordinates of the 3D point cloud data. This allows the panoramic image to be used to colorize the 3D point cloud data, generating true-color 3D point cloud data, i.e., the target 3D point cloud data, thus achieving the assignment of true-color attributes to the 3D point cloud data.

[0183] Step S603: Obtain multiple panoramic images through the panoramic image pose.

[0184] It should be noted that, based on the targetless camera parameter calibration method, the calibration relationship between the digital camera and the panoramic camera can be constructed by matching visual feature points. Based on the calibration relationship between the panoramic camera of the mobile measuring vehicle and the laser, the corresponding points between the images acquired by the digital camera and the point cloud acquired by the laser can be constructed to establish the correspondence or geometric constraints of the connection points in an indirect manner. Then, the transformation matrix of the camera sensor in the reference coordinate system can be estimated. Therefore, multiple frames of panoramic images can be obtained first based on the pose of the panoramic images.

[0185] Step S604: Acquire digital camera images alternately captured by the first group of digital cameras and the second group of digital cameras.

[0186] It should be noted that in this embodiment, the digital camera is divided into two groups for alternating acquisition, thereby improving the acquisition effect. Therefore, digital camera images acquired alternately by the first group of digital cameras and the second group of digital cameras can be obtained.

[0187] In one possible implementation, the plurality of digital cameras includes a first group of digital cameras and a second group of digital cameras, wherein the number of cameras in the first group of digital cameras and the second group of digital cameras are the same, and the first group of digital cameras and the second group of digital cameras take turns shooting.

[0188] It should be noted that, continuing as Figure 2 As shown, this embodiment integrates six digital cameras: two on each side of the vehicle-mounted camera system and two ground-facing cameras. Cameras A1, A2, and A3 form group A, while cameras B1, B2, and B3 form group B. These cameras are controlled by a synchronization controller to acquire images alternately in groups, ensuring a high degree of overlap between group A and group B images. Taking the opposite-side cameras A1 and B1 as an example, the image coverage and overlap are calculated. The same applies to A2 and B2. The ground-facing cameras A3 and B3 are positioned closer together, with a shorter acquisition distance and higher overlap than the opposite-side cameras; therefore, only the opposite-side cameras are calculated.

[0189] Before the step of calculating the digital camera image pose using the panoramic image pose, the method further includes:

[0190] The system controls the first group of digital cameras and the second group of digital cameras to alternately acquire images, and during the acquisition process, it acquires the camera's placement distance along the vehicle's forward direction, camera field of view, outward rotation angle, image size, acquisition distance, and acquisition vehicle speed.

[0191] In specific implementation, the first group of digital cameras and the second group of digital cameras can be controlled to alternately acquire images, and relevant parameters can be obtained during the acquisition process. The distance between cameras A1 and B1 along the direction of vehicle movement is set to l=0.6m, the camera field of view d1=86°, the outward rotation angle d2=5°, the imaging size is 7360 pixels * 4912 pixels, the acquisition distance is L=10m, the minimum response time between frames is 1s, and the acquisition vehicle speed V during acquisition is obtained.

[0192] The image horizontal coverage value is calculated based on the camera field of view, the outward rotation angle, and the acquisition distance.

[0193] In practice, the horizontal coverage value of the image can be calculated based on the camera's field of view, the outward rotation angle, and the acquisition distance. The horizontal coverage value of the image is calculated as tan(d1 / 2+d2)*L+tan(d1 / 2-d2)*L=18.919 m.

[0194] Calculate the vertical coverage value of the image based on the horizontal coverage value and the imaging size;

[0195] In practice, the vertical coverage value of the image can be calculated based on the horizontal coverage value and the imaging size. The vertical coverage value of the image is 18.919 * 4912 / 7360 = 12.626 m, and the imaging size is higher than 36 million pixels.

[0196] The overlap between the first group of digital cameras and the second group of digital cameras is calculated based on the image horizontal coverage value, the image vertical coverage value, the placement distance, and the acquisition vehicle speed.

[0197] It should be noted that the overlap between A1 and B1 can be calculated based on the vehicle speed, horizontal coverage value, vertical coverage value, and installation distance, as shown in the following formula:

[0198] P=(18.919-((V / 3.6-l)-(tan(d1 / 2+d2)*L-tan(d1 / 2-d2)*L))) / 18.919=1-(V-6.8933) / 68.1084.

[0199] When the overlap is less than a preset overlap, the acquisition vehicle speed is reduced, and the images corresponding to the overlap less than the preset overlap are marked.

[0200] It should be noted that a preset overlap can be set, for example, the overlap is not less than 50%. When the preset overlap is 50%, the vehicle speed is collected as 48km / h.

[0201] By utilizing the relationship between vehicle speed and overlap, a vehicle speed warning value can be set according to different overlap requirements to prompt operators to reduce vehicle speed. Additionally, the relationship between vehicle speed and image overlap can be used for camera selection and installation settings. Based on the aforementioned derivation process, the expression for image overlap in vehicle-mounted digital camera images can be derived as follows:

[0202]

[0203] As can be seen from the formula, the image overlap depends on the vehicle speed, camera mounting distance, camera field of view, outward rotation angle, and camera shooting distance. Considering that vehicle speed and camera mounting distance are limited by actual road conditions and the length of the vehicle carrier along the forward direction, camera selection can also be based on the target overlap. The above formula provides preset values ​​for vehicle speed and camera mounting distance. Then, the camera field of view, outward rotation angle, and camera shooting distance are substituted into the formula to calculate the overlap, and it is determined whether the theoretical overlap meets the target requirements.

[0204] In practice, when the overlap is less than the preset overlap, it means that the digital camera images collected at this time may not meet the requirements. Therefore, the acquisition speed can be reduced and the images that do not meet the overlap requirements can be marked to facilitate manual intervention during modeling.

[0205] The reduced acquisition vehicle speed controls the first and second sets of digital cameras to acquire images alternately.

[0206] Understandably, by reducing the speed of the acquisition vehicle, the first and second sets of digital cameras can be controlled to acquire images alternately, thus addressing the problem that traditional methods only acquire single panoramic images with low resolution, large camera distortion, and small frame overlap, making them unsuitable for constructing real-world 3D models.

[0207] Step S605: Perform coarse registration on multiple frames of the panoramic image and multiple frames of the digital camera image according to the timestamp, and extract the first feature point in the panoramic image and the corresponding second feature point in the digital camera image.

[0208] In practice, both panoramic images and digital camera images have timestamps. The synchronization controller can achieve synchronous triggering of each frame of panoramic image and digital camera image. Therefore, coarse registration is first performed based on the timestamps to extract feature points from the panoramic image, i.e., the first feature points. At the same time, the corresponding feature points are extracted from the digital camera image, i.e., the second feature points.

[0209] Step S606: Match the first feature point and the second feature point using a feature matching algorithm to obtain a first matching feature point pair.

[0210] It should be noted that feature matching algorithms can be used to match feature points in panoramic images with feature points in digital camera images.

[0211] Feature points can be corner points, edge points, or other salient points in the image. Specifically, feature matching algorithms (such as SIFT, SURF, ORB, etc.) can be used to match the first set of feature points with the second set of feature points. These algorithms, based on local descriptors of features, find the similarity between the two sets of points and establish a matching relationship. After matching, the first pair of matched feature points is obtained, and the correspondence between these point pairs provides the basis for subsequent calculations.

[0212] Step S607: Based on the transformation relationship between the panoramic image coordinate system and the three-dimensional point cloud data coordinate system and the first matching feature point pair, obtain the point cloud feature points corresponding to the second feature points of the digital camera.

[0213] It should be understood that there is a fixed calibration relationship between the panoramic camera and the laser. Therefore, the transformation relationship between the panoramic image coordinate system and the 3D point cloud data coordinate system can be obtained based on this calibration relationship. Thus, the corresponding points in the laser point cloud, i.e., point cloud feature points, can be obtained through the transformation relationship and the first matching feature point pair. In this way, points in the image are mapped to point cloud feature points in 3D space through geometric relationships.

[0214] Step S608: Obtain the second matching feature point pair based on the second feature point and the corresponding point cloud feature point.

[0215] In practice, the second feature point and the corresponding point cloud feature point are combined to form a second matching feature point pair to ensure that subsequent transformation calculations can be more accurate.

[0216] Step S609: Calculate the transformation matrix from the digital image coordinate system to the three-dimensional point cloud data coordinate system based on the second matching feature point pair.

[0217] It should be noted that the transformation matrix from the digital image coordinate system to the point cloud coordinate system can be calculated using an adjustment algorithm using the second matching feature point pair. The transformation matrix describes the geometric relationship between the two coordinate systems, including rotation and translation. The calculation of the transformation matrix usually relies on known matching feature point pairs and is solved using optimization methods such as least squares method and RANSAC algorithm.

[0218] Step S610: Obtain the digital camera image pose through the transformation matrix.

[0219] In practice, the transformation matrix can be iteratively calculated using the post-hoc residual information to obtain the high-precision coordinate attributes of the digital camera images. Since the installation position and angle of each camera differ, steps S605-S609 are repeated for each camera's image data to calculate the pose information of all digital camera images.

[0220] This method uses a panoramic camera as an intermediary for LiDAR-digital camera calibration. The panoramic camera and LiDAR are integrated into a single module, and the calibration relationship between them can be reused after a single calibration. Furthermore, the synchronous controller triggering mechanism ensures near-synchronous triggering of panoramic and digital images. Therefore, the calibration relationship between the DSLR camera and the panoramic camera can be constructed using image visual feature point matching. Image feature matching can be fully automated. Based on the existing calibration relationship between the panoramic camera and the LiDAR on the mobile surveying vehicle, corresponding points between the digital image and the point cloud acquired by the LiDAR are constructed to indirectly establish the correspondence or geometric constraints of the connection points. Then, the transformation matrix of the camera sensor in the reference coordinate system is estimated. This method does not require additional target calibration work, requires no manual intervention, and can achieve optimal estimation of the calibration relationship matrix.

[0221] The images and 3D point clouds acquired and processed in this embodiment have high-precision pose attributes. At the same time, the images acquired and processed by grouping digital camera images have the characteristics of high resolution and high overlap, which can realistically reflect the street scene, roads and landscapes on both sides, and building facade information, realizing the 3D model to express the real world in two or three dimensions.

[0222] For example, to help understand the implementation process of the real-scene 3D model construction method based on the vehicle-mounted mobile measurement system obtained by combining this embodiment with the above embodiment one, please refer to... Figure 5 , Figure 5A simplified flowchart of a method for constructing a real-scene 3D model based on a vehicle-mounted mobile measurement system is provided. Specifically: Step 1: The time-frequency signal driving device synchronously triggers the sensors of each functional module; Step 2: The synchronization controller triggers the digital cameras in groups; Step 3: The timestamps of each sensor are returned and invalid timestamps are discarded; Step 4: The position and attitude of the inertial navigation sensor are estimated, and the positioning antenna and receiver loop tracking is assisted; Step 5: The positioning antenna and receiver track and decode, and navigation calculation is performed; Step 6: The mileage of the vehicle's movement is collected, and the vehicle's motion trajectory is smoothed; Step 7: Tightly combined filter estimation is used to calculate the unoptimized position and attitude information of the vehicle; Step 8: The influence of acceleration and angular velocity drift errors accumulated by the inertial navigation over time is corrected; Step 9: The point cloud data is used to estimate the vehicle's motion state through feature matching between frames; Step 10: The position and attitude information of the vehicle are optimized in a loosely combined filter, and the optimized result is output. Step 11: Take photos at a fixed distance with the panoramic camera and stitch the images from the 6 lenses into a 360° panoramic image; Step 12: Take photos alternately with the 6 digital cameras in two groups, ensuring uniform lighting and color in the images; Step 13: Calculate the image overlap, and issue warnings and markings when the modeling requirements are not met; Step 14: Collect high-density 3D point cloud data; Step 15: Perform multi-echo cancellation and noise removal on the high-density 3D point cloud data; Step 16: Perform feature extraction, semantic recognition, and classification based on deep learning methods; Step 17: Calculate the geographic coordinate attributes of the 3D point cloud data; Step 18: Calculate the high-precision pose attributes of the panoramic image; Step 19: Use the panoramic image to colorize the point cloud data to generate true-color 3D point cloud data; Step 20: Estimate the transformation matrix of the camera sensor in the reference coordinate system based on the targetless camera parameter calibration method; Step 21: Calculate the high-precision pose attributes of the digital camera images and construct a real-scene 3D model. Steps 1 to 3 are implemented by the time synchronization function module to perform time synchronization; steps 4 to 10 are implemented by the positioning and attitude determination function module to perform positioning and attitude determination; steps 11 to 13 are implemented by the image processing function module to perform image processing; steps 14 to 16 are implemented by the point cloud processing function module to perform point cloud processing; and steps 17 to 21 are implemented by the core processor function module to perform data processing and construction of the real scene 3D model.

[0223] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the method for constructing a real-scene 3D model based on a vehicle-mounted mobile measurement system. Any simple transformations based on this technical concept are within the scope of protection of this application.

[0224] This application also provides a device for constructing a real-scene 3D model based on a vehicle-mounted mobile measurement system. Please refer to [link / reference]. Figure 6 The real-scene 3D model construction device based on the vehicle-mounted mobile measurement system includes:

[0225] The time synchronization module 10 is used to collect acceleration and angular velocity through the inertial navigation system after the inertial navigation system, GNSS receiver, odometer, panoramic camera, multiple 3D LiDAR, multiple digital cameras and positioning antenna have been synchronized in time.

[0226] The acquisition module 20 is used to acquire mileage information of the vehicle based on the odometer.

[0227] Calculation module 30 is used to calculate the unoptimized first position and first attitude of the vehicle carrier based on the acceleration, the angular velocity and the mileage information;

[0228] Optimization module 40 is used to optimize the first position and the first posture based on the three-dimensional lidar to obtain optimized pose information;

[0229] The calculation module 30 is also used to calculate the geographic coordinates of the three-dimensional point cloud data and the pose of the panoramic image based on the optimized pose information.

[0230] Processing module 50 is used to process the geographic coordinates of the three-dimensional point cloud data through the panoramic image pose to obtain the target three-dimensional point cloud data, and to calculate the digital camera image pose through the panoramic image pose.

[0231] The construction module 60 is used to construct a real-scene 3D model using the target 3D point cloud data and the pose of the digital camera image.

[0232] The real-scene 3D model construction device based on a vehicle-mounted mobile measurement system provided in this application adopts the real-scene 3D model construction method based on a vehicle-mounted mobile measurement system in the above embodiments, which can solve the technical problem of poor real-scene 3D model construction effect in the current applications. Compared with the prior art, the beneficial effects of the real-scene 3D model construction device based on a vehicle-mounted mobile measurement system provided in this application are the same as the beneficial effects of the real-scene 3D model construction method based on a vehicle-mounted mobile measurement system provided in the above embodiments, and other technical features in the real-scene 3D model construction device based on a vehicle-mounted mobile measurement system are the same as the features disclosed in the methods of the above embodiments, and will not be repeated here.

[0233] This application provides a vehicle-mounted mobile measurement system, which includes an inertial navigation system 1, a GNSS receiver 2, an odometer (not shown in the figure), a panoramic camera Q1, multiple digital cameras including digital cameras A1, A2, A3, B1, B2, and B3, multiple three-dimensional lidars including a first lidar L2 and a second lidar L1, a positioning antenna 4, and a synchronization controller 5. The odometer is mounted on the wheels, the multiple digital cameras are mounted around the vehicle, and the multiple three-dimensional lidars are mounted at the front and left and right sides of the vehicle. The panoramic camera Q1 and the multiple three-dimensional lidars are rigidly connected.

[0234] The vehicle-mounted mobile measurement system provided in this application employs the real-scene 3D model construction method based on the vehicle-mounted mobile measurement system described in the above embodiments, which can solve the technical problem of poor real-scene 3D model construction effect in current applications. Compared with the prior art, the beneficial effects of the vehicle-mounted mobile measurement system provided in this application are the same as those of the real-scene 3D model construction method based on the vehicle-mounted mobile measurement system provided in the above embodiments, and other technical features of this vehicle-mounted mobile measurement system are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0235] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0236] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0237] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the real-scene 3D model construction method based on the vehicle-mounted mobile measurement system in the above embodiments.

[0238] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory or Flash Memory), optical fibers, CD-ROM (CD-Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0239] The aforementioned computer-readable storage medium may be included in the vehicle-mounted mobile measurement system; or it may exist independently and not be assembled into the vehicle-mounted mobile measurement system.

[0240] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the vehicle-mounted mobile measurement system, the vehicle-mounted mobile measurement system: after completing time synchronization of the inertial navigation system, the GNSS receiver, the odometer, the panoramic camera, the multiple 3D LiDARs, the multiple digital cameras, and the positioning antenna, acquires acceleration and angular velocity through the inertial navigation system; acquires mileage information of the vehicle's forward movement based on the odometer; calculates an unoptimized first position and first attitude of the vehicle carrier based on the acceleration, the angular velocity, and the mileage information; optimizes the first position and first attitude based on the 3D LiDARs to obtain optimized pose information; calculates the geographic coordinates of 3D point cloud data and the pose of the panoramic image based on the optimized pose information; processes the geographic coordinates of the 3D point cloud data using the panoramic image pose to obtain target 3D point cloud data, and calculates the pose of the digital camera image using the panoramic image pose.

[0241] A real-world 3D model is constructed using the target 3D point cloud data and the pose of the digital camera image.

[0242] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including LAN (Local Area Network) or WAN (Wide Area Network)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0243] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0244] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0245] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described method for constructing a real-scene 3D model based on a vehicle-mounted mobile measurement system. This method can solve the technical problem of poor real-scene 3D model construction results in current applications. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the real-scene 3D model construction method based on a vehicle-mounted mobile measurement system provided in the above embodiments, and will not be elaborated upon here.

[0246] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method for constructing a real-scene 3D model based on a vehicle-mounted mobile measurement system.

[0247] The computer program product provided in this application can solve the technical problem of poor effect in the current construction of real-scene 3D models. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the real-scene 3D model construction method based on vehicle-mounted mobile measurement system provided in the above embodiments, and will not be repeated here.

[0248] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for constructing a real scene three-dimensional model based on a vehicle-mounted mobile measurement system, characterized in that, The vehicle-mounted mobile measurement system comprises: an inertial navigation system, a GNSS receiver, an odometer, a panoramic camera, a plurality of digital cameras, a plurality of three-dimensional laser radars, a positioning antenna and a synchronization controller, wherein the odometer is mounted on a wheel, the plurality of digital cameras are mounted around the vehicle, the plurality of three-dimensional laser radars are mounted at the front and both sides of the vehicle, and the panoramic camera and the plurality of three-dimensional laser radars are rigidly connected; The real three-dimensional model construction method based on the vehicle-mounted mobile measurement system comprises: After time synchronization of the inertial navigation system, the GNSS receiver, the odometer, the panoramic camera, the plurality of three-dimensional laser radars, the plurality of digital cameras and the positioning antenna is completed, acceleration and angular velocity are collected by the inertial navigation system; Mileage information of vehicle advancement is collected based on the odometer; A first position and a first attitude of a vehicle carrier that are not optimized are calculated according to the acceleration, the angular velocity and the mileage information; The first position and the first attitude are optimized based on the three-dimensional laser radars to obtain optimized pose information; Three-dimensional point cloud data geographical coordinates and panoramic image poses are calculated according to the optimized pose information; The three-dimensional point cloud data geographical coordinates are processed through the panoramic image poses to obtain target three-dimensional point cloud data, and digital camera image poses are calculated through the panoramic image poses; A real three-dimensional model is constructed through the target three-dimensional point cloud data and the digital camera image poses; The step of calculating a first position and a first attitude of a vehicle carrier that are not optimized according to the acceleration, the angular velocity and the mileage information comprises: The positioning antenna and the GNSS receiver are controlled to perform loop tracking based on the inertial navigation system to obtain navigation satellite pseudo-range and carrier phase observation data; Navigation solution is performed on the navigation satellite pseudo-range and carrier phase observation data to obtain geographical coordinate system positioning; The inertial navigation system is corrected based on the geographical coordinate system positioning to obtain a corrected inertial navigation system, and a first corrected acceleration and a first corrected angular velocity collected by the corrected inertial navigation system are obtained; Tightly coupled filtering estimation is performed on the first corrected acceleration, the first corrected angular velocity and the mileage information to obtain a first position and a first attitude of a vehicle carrier that are not optimized; The plurality of three-dimensional laser radars comprises a first laser radar; The step of optimizing the first position and the first attitude based on the three-dimensional laser radars to obtain optimized pose information comprises: The acceleration and the angular velocity accumulated over time by the inertial navigation system are corrected through the first position and the first attitude to obtain a second corrected acceleration and a second corrected angular velocity; First laser point cloud data is collected by the first laser radar; Vehicle motion state is obtained through feature matching of the first laser point cloud data; The second corrected acceleration and the second corrected angular velocity are optimized in loose coupled filtering through the vehicle motion state to obtain optimized pose information.

2. The method of claim 1, wherein, The plurality of three-dimensional laser radars comprises a second laser radar, and the number of laser beams scanned by the second laser radar is greater than the number of laser beams scanned by the first laser radar; The step of calculating the geographic coordinates of the three-dimensional point cloud data and the panorama image pose according to the optimized pose information comprises: collecting second laser three-dimensional point cloud data by the second laser radar; performing multi-return elimination and noise removal on the second laser three-dimensional point cloud data to obtain processed second laser three-dimensional point cloud data; performing feature extraction and semantic processing on the processed second laser three-dimensional point cloud data by a deep learning algorithm to obtain geometric information and spatial structure information of building objects in a three-dimensional scene; calculating the geographic coordinates of the three-dimensional point cloud data by the optimized pose information, the geometric information and the spatial structure information; acquiring panorama images collected by the panorama camera; aligning the time stamp of each frame of the panorama images with the time sequence of the optimized pose information, and performing interpolation calculation on the optimized pose information before and after the time stamp of the panorama image to obtain the panorama image pose.

3. The method of claim 2, wherein, The step of performing feature extraction and semantic processing on the processed second laser three-dimensional point cloud data by a deep learning algorithm to obtain geometric information and spatial structure information of building objects in a three-dimensional scene comprises: performing feature extraction on the processed second laser three-dimensional point cloud data by a deep learning algorithm to obtain point cloud features; performing semantic labeling on points or grid elements in the three-dimensional scene according to the point cloud features to obtain different semantic categories; segmenting regions of different semantic categories by a deep learning model to obtain segmented regions of different semantic categories; determining the geometric information and the spatial structure information of building objects in the three-dimensional scene by the segmented regions of different semantic categories.

4. The method of claim 1, wherein, The step of processing the geographic coordinates of the three-dimensional point cloud data by the panorama image pose to obtain target three-dimensional point cloud data comprises: obtaining a conversion relationship between a panorama image coordinate system and a three-dimensional point cloud data coordinate system; performing panorama image coordinate conversion according to the conversion relationship, the panorama image pose and the geographic coordinates of the three-dimensional point cloud data to color the three-dimensional point cloud data by the panorama image to obtain target three-dimensional point cloud data.

5. The method of claim 1, wherein, The plurality of digital cameras comprises a first group of digital cameras and a second group of digital cameras, the number of cameras in the first group of digital cameras and the second group of digital cameras is the same, and the first group of digital cameras and the second group of digital cameras alternately take pictures; Before the step of calculating the digital camera image pose by the panorama image pose, it further comprises: controlling the first group of digital cameras and the second group of digital cameras to alternately collect images, and acquiring the installation distance of the cameras along the forward direction of the vehicle, the camera field of view angle, the outward rotation angle, the imaging size, the collection distance and the collection vehicle speed during the collection process; calculating the image horizontal coverage value according to the camera field of view angle, the outward rotation angle and the collection distance; calculating the image vertical coverage value according to the image horizontal coverage value and the imaging size; calculating the overlap degree between the first group of digital cameras and the second group of digital cameras according to the image horizontal coverage value, the image vertical coverage value, the installation distance and the collection vehicle speed; When the overlap degree is less than the preset overlap degree, the acquisition vehicle speed is reduced, and images less than the preset overlap degree are marked; The first group of digital cameras and the second group of digital cameras alternately acquire images at the reduced acquisition vehicle speed.

6. The method of claim 5, wherein, The step of calculating the digital camera image pose from the panoramic image pose includes: A plurality of panoramic images are obtained from the panoramic image pose; Digital camera images alternately acquired by the first group of digital cameras and the second group of digital cameras are obtained; According to the time stamp, a plurality of panoramic images and a plurality of digital camera images are coarsely registered, and first feature points in the panoramic images and corresponding second feature points in the digital camera images are extracted; The first feature points and the second feature points are matched by a feature matching algorithm to obtain a first matching feature point pair; According to the conversion relationship between the panoramic image coordinate system and the three-dimensional point cloud data coordinate system and the first matching feature point pair, a point cloud feature point corresponding to the second feature point of the digital camera is obtained; A second matching feature point pair is obtained according to the second feature point and the corresponding point cloud feature point; A transformation matrix from the digital image coordinate system to the three-dimensional point cloud data coordinate system is calculated according to the second matching feature point pair; The digital camera image pose is obtained through the transformation matrix.

7. A device for constructing a real three-dimensional model based on a vehicle-mounted mobile measurement system, characterized by, The device includes: A time synchronization module is configured to, after time synchronization of inertial navigation, GNSS receiver, odometer, panoramic camera, multiple three-dimensional laser radars, multiple digital cameras, and positioning antenna is completed, acquire acceleration and angular velocity by the inertial navigation; An acquisition module is configured to acquire mileage information of vehicle advancement based on the odometer; A calculation module is configured to calculate a first position and a first attitude of a vehicle carrier that are not optimized according to the acceleration, the angular velocity, and the mileage information; An optimization module is configured to optimize the first position and the first attitude based on the three-dimensional laser radars to obtain optimized pose information; The calculation module is further configured to calculate three-dimensional point cloud data geographical coordinates and a panoramic image pose according to the optimized pose information; A processing module is configured to process the three-dimensional point cloud data geographical coordinates by the panoramic image pose to obtain target three-dimensional point cloud data, and calculate a digital camera image pose by the panoramic image pose; A construction module is configured to construct a real scene three-dimensional model by the target three-dimensional point cloud data and the digital camera image pose; The step of calculating a first position and a first attitude of a vehicle carrier that are not optimized according to the acceleration, the angular velocity, and the mileage information includes: The positioning antenna and the GNSS receiver are controlled to perform loop tracking based on the inertial navigation to obtain navigation satellite pseudo-range and carrier phase observation data; The navigation satellite pseudo-range and carrier phase observation data are subjected to navigation solution to obtain geographical coordinate system positioning; The inertial navigation is corrected based on the geographical coordinate system positioning to obtain a corrected inertial navigation, and a first corrected acceleration and a first corrected angular velocity acquired by the corrected inertial navigation are obtained; performing tight combination filtering estimation on the first corrected acceleration, the first corrected angular velocity and the mileage information to obtain a first position and a first attitude of the vehicle carrier which are not optimized; the plurality of three-dimensional laser radars comprises a first laser radar; the step of optimizing the first position and the first attitude based on the three-dimensional laser radars comprises: correcting the acceleration and the angular velocity accumulated over time by the inertial navigation system by using the first position and the first attitude to obtain a second corrected acceleration and a second corrected angular velocity; collecting first laser point cloud data by using the first laser radar; performing feature matching on the first laser point cloud data to obtain a vehicle motion state; optimizing the second corrected acceleration and the second corrected angular velocity in loose combination filtering by using the vehicle motion state to obtain optimized attitude information.

8. A mobile measurement system on board a vehicle, characterized in that The vehicle-mounted mobile measurement system comprises an inertial navigation system, a GNSS receiver, an odometer, a panoramic camera, a plurality of digital cameras, a plurality of three-dimensional laser radars, a positioning antenna and a synchronization controller, wherein the odometer is mounted on a wheel, the plurality of digital cameras are mounted around the vehicle, the plurality of three-dimensional laser radars are mounted on the front and both sides of the vehicle, the panoramic camera and the plurality of three-dimensional laser radars are rigidly connected, and the vehicle-mounted mobile measurement system is configured to implement the steps of the real scene three-dimensional model construction method based on the vehicle-mounted mobile measurement system according to any one of claims 1 to 6.

Images