Three-dimensional model generation method and system based on multi-data fusion

Abstract

The invention relates to the technical field of three-dimensional reconstruction, and particularly discloses a three-dimensional model generation method and system based on multi-data fusion, and the method comprises the steps: controlling an unmanned plane to carry out the oblique photographing of a to-be-reconstructed real scene, and generating a first point cloud based on the data obtained through the oblique photographing, the method comprises the steps of collecting a second point cloud on a road of a to-be-reconstructed real scene through a handheld laser radar, controlling a vehicle carrying the laser radar to drive on the road to collect a third point cloud, fusing the first point cloud, the second point cloud and the third point cloud to obtain a multi-data fusion point cloud, and generating a three-dimensional model of the to-be-reconstructed real scene based on the multi-data fusion point cloud. A handheld laser radar scans a road surface and a vehicle-mounted laser radar scans a surrounding environment of a road to obtain point clouds, and first point clouds generated by oblique photography data acquisition are fused to generate a high-precision three-dimensional model, so that details of road fluctuation, gradient, corners, side slopes, guardrails, traffic signs and the like of a real scene can be accurately reproduced; and the real scene simulation effect of the game is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional reconstruction, and in particular relates to a three-dimensional model generation method and system based on multi-data fusion. Background Art

[0002] When making a game scene with reference to a real scene, a three-dimensional model of the game scene is usually made with reference to a photo or video of the real scene. For example, the three-dimensional model in the game scene is made through satellite images of the real scene. For example, a racing scene model of a racing game is made through a three-dimensional satellite map, image or video of a certain real scene area. However, the three-dimensional model made from photos, videos or satellite maps of the real scene is difficult to reflect the ups and downs, slopes and corners of the ground in the real scene, and it is difficult for users to experience a real simulation effect in the game. Summary of the invention

[0003] The purpose of the embodiments of the present invention is to provide a three-dimensional model generation method and system based on multi-data fusion, aiming to solve the problem that the existing generated three-dimensional models are difficult to reflect the ground ups and downs, slopes and corners of real scenes, affecting the user's simulation experience in the game.

[0004] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions: A three-dimensional model generation method based on multi-data fusion specifically comprises the following steps: Controlling a drone equipped with a camera to perform oblique photography on a real scene to be reconstructed to obtain oblique photography data, and generating a first point cloud of the real scene to be reconstructed based on the oblique photography data; Collecting a second point cloud of the road of the real scene to be reconstructed by a handheld laser radar; Controlling a vehicle equipped with a laser radar to drive on a road of the real scene to be reconstructed to collect a third point cloud; fusing the first point cloud, the second point cloud and the third point cloud to obtain a multi-data fused point cloud; A three-dimensional model of the real scene to be reconstructed is generated based on the multi-data fusion point cloud.

[0005] As a further technical solution of the present invention, before controlling the drone equipped with a camera to perform oblique photography on the real scene to be reconstructed to obtain oblique photography data, the following steps are also included: Acquire a satellite image of the real scene to be reconstructed; A flight path of the UAV is generated based on the satellite image, wherein the flight path includes UAV waypoints and image acquisition points.

[0006] As a further technical solution of the present invention, controlling a drone equipped with a camera to perform oblique photography on a real scene to be reconstructed to obtain oblique photography data, and generating a first point cloud of the real scene to be reconstructed based on the oblique photography data, specifically includes the following steps: Controlling a drone equipped with a camera to fly over the real scene to be reconstructed according to the flight path; When flying to the image acquisition point, controlling the camera to acquire an image to obtain an oblique photography image; A first point cloud of the real scene to be reconstructed is generated based on the oblique photography image and the position and posture data of the drone at the image acquisition point.

[0007] As a further technical solution of the present invention, before collecting the second point cloud of the road of the real scene to be reconstructed by a handheld laser radar, the following steps are specifically included: Generating an intermediate three-dimensional model of the real scene to be reconstructed based on the first point cloud; The intermediate three-dimensional model is input into a road recognition model to obtain at least one target road section to be reconstructed.

[0008] As a further technical solution of the present invention, collecting a second point cloud of the road of the real scene to be reconstructed by a handheld laser radar specifically includes the following steps: The target road section to be reconstructed is scanned by a handheld laser radar to obtain a second point cloud.

[0009] As a further technical solution of the present invention, the first point cloud, the second point cloud and the third point cloud are fused to obtain a multi-data fused point cloud, which specifically includes the following steps: Preprocessing the first point cloud, the second point cloud, and the third point cloud to obtain a preprocessed first point cloud, a second point cloud, and a third point cloud; The preprocessed first point cloud, the second point cloud and the third point cloud are fused by ICP registration to obtain a multi-data fused point cloud.

[0010] As a further technical solution of the present invention, the pre-processed first point cloud, the second point cloud and the third point cloud are subjected to ICP registration fusion to obtain a multi-data fusion point cloud, which specifically includes the following steps: Perform ICP registration and fusion on the preprocessed first point cloud and the second point cloud to obtain a fourth point cloud; The road surface point cloud is removed from the preprocessed third point cloud to obtain a fifth point cloud; The fourth point cloud and the fifth point cloud are fused by ICP registration to obtain a multi-data fused point cloud.

[0011] As a further technical solution of the present invention, generating the three-dimensional model of the real scene to be reconstructed based on the multi-data fusion point cloud specifically includes the following steps: Generate a triangular mesh based on the multi-data fusion point cloud to obtain an initial three-dimensional surface model of the real scene to be reconstructed; The initial three-dimensional surface model is smoothed to obtain a three-dimensional model of the real scene to be reconstructed.

[0012] A three-dimensional model generation system based on multi-data fusion, specifically including the following units: A first point cloud acquisition unit is used to control a drone equipped with a camera to perform oblique photography on the real scene to be reconstructed to obtain oblique photography data, and generate a first point cloud of the real scene to be reconstructed based on the oblique photography data; A second point cloud acquisition unit is used to collect a second point cloud of the road of the real scene to be reconstructed by using a handheld laser radar; A third point cloud acquisition unit, used to control a vehicle equipped with a laser radar to drive on the road of the real scene to be reconstructed to collect a third point cloud; A point cloud fusion unit, used to fuse the first point cloud, the second point cloud and the third point cloud to obtain a multi-data fused point cloud; A three-dimensional model generating unit is used to generate a three-dimensional model of the real scene to be reconstructed based on the multi-data fusion point cloud.

[0013] As a further technical solution of the present invention, the following units are also included: A satellite image acquisition unit, used to acquire a satellite image of the real scene to be reconstructed; A flight path generation unit is used to generate a flight path of the UAV based on the satellite image, wherein the flight path includes UAV waypoints and image acquisition points.

[0014] Compared with the prior art, the present invention has the following beneficial effects: In the embodiment of the present invention, a drone is used to perform oblique photography of a real scene to be reconstructed, a first point cloud is generated based on the data obtained from the oblique photography, a handheld laser radar is used to collect a second point cloud from a road in the real scene to be reconstructed, and a vehicle equipped with the laser radar is controlled to drive on the road in the real scene to be reconstructed to collect a third point cloud. After the first point cloud, the second point cloud and the third point cloud are fused to obtain a multi-data fused point cloud, a three-dimensional model of the real scene to be reconstructed is generated based on the multi-data fused point cloud, so that an overall and rough first point cloud of the area to be reconstructed is obtained through high-altitude oblique photography of the drone, and then a second point cloud is collected from the road by a handheld laser radar to collect details such as the ups and downs, slopes and corners of the road, and further a third point cloud is collected by a vehicle-mounted laser radar while driving on the road to collect details such as the slopes, guardrails, and traffic signs on the road, so that the fused multi-data fused point cloud can represent the real scene of the real scene to be reconstructed. After the high-precision three-dimensional model is applied to the game, the ups and downs, potholes, slopes, bumps, corners, slopes, guardrails, ditch depths and traffic signs of the road in the real scene can be realistically and accurately simulated, thereby improving the simulation effect of the game. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0016] Figure 1 A flow chart of a method for generating a three-dimensional model based on multi-data fusion provided by an embodiment of the present invention is shown.

[0017] Figure 2 This is a schematic diagram of drone aerial oblique photography; Figure 3 Schematic diagram of the true color point cloud obtained by the handheld laser radar; Figure 4 Schematic diagram of the point cloud obtained by the vehicle-mounted laser radar; Figure 5 Schematic diagram of generating a 3D model after generating a 3D mesh from multiple data fusion point clouds; Figure 6 An application architecture diagram of a three-dimensional model generation system based on multi-data fusion provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0019] Figure 1 The flowchart of the method for generating a three-dimensional model based on multi-data fusion provided by an embodiment of the present invention is shown. Specifically, the method for generating a three-dimensional model based on multi-data fusion provided by an embodiment of the present invention comprises the following steps: Step S101: Control a drone equipped with a camera to perform oblique photography of a real scene to be reconstructed to obtain oblique photography data, and generate a first point cloud of the real scene to be reconstructed based on the oblique photography data.

[0020] One application scenario of the present embodiment is to establish a three-dimensional model of a real scene, and apply the three-dimensional model to a game scene. For example, the game may be a racing game, and the real scene may be a scene with roads in reality and suitable for racing, such as a mountainous area with roads or a city street, so as to simulate and reproduce the real scene in the game for racing. The real scene that needs to be simulated in the game is the real scene to be reconstructed. The present embodiment can use a drone equipped with a camera to perform aerial oblique photography of the real scene to be reconstructed to obtain a first point cloud of the real scene to be reconstructed.

[0021] Specifically, a satellite image of the real scene to be reconstructed can be obtained first, and a flight path of the UAV can be generated based on the satellite image, where the flight path includes UAV waypoints and image acquisition points, wherein the image acquisition points can be points for controlling a camera on the UAV to acquire images, and the UAV equipped with a camera is controlled to fly over the real scene to be reconstructed according to the flight path, and when flying to the image acquisition point, the camera is controlled to acquire images to obtain an oblique photographic image, and a first point cloud of the real scene to be reconstructed is generated based on the oblique photographic image and the position data of the UAV at the image acquisition point.

[0022] The satellite image may be a stereoscopic image of the real scene to be reconstructed, or an elevation map of the real scene to be reconstructed, wherein the stereoscopic image includes the height of the objects on the surface of the real scene to be reconstructed, and the satellite image, the flight altitude, flight speed, and flight path overlap rate of the drone may be input into the path planning program of the drone to obtain the initial flight path of the drone when performing oblique photography of the real scene to be reconstructed, wherein the initial flight path includes waypoints and image acquisition points, wherein the waypoints are used to control the direction and altitude of the drone flight, and the image acquisition points are used to control the camera on the drone to acquire images, and the initial path is checked and adjusted by the surveyor to obtain the final flight path, and illustratively, the surveyor may add, delete or adjust the image acquisition points in the initial flight path to acquire images at the best image acquisition points. Of course, the surveyor may also select a target area on the two-dimensional map of the real scene to be reconstructed, and manually set the flight path in the target area, and this embodiment does not limit the method of generating the flight path.

[0023] The flight altitude of the UAV's flight path can change with the terrain of the real scene to be reconstructed, so that the UAV can perform oblique photography by flying in the area of ​​the real scene to be reconstructed.

[0024] The drone can be equipped with two or more cameras, each with a different shooting angle, so as to acquire images from different angles. Each camera can be pre-calibrated to obtain calibration parameters and then stored.

[0025] like Figure 2 The figure shows a schematic diagram of oblique photography of a mountainous area including roads, wherein the flight path is a plurality of parallel paths. When the UAV flies along the flight path, each time it reaches an image acquisition point, it controls the camera to expose and acquire an image, thereby obtaining oblique photography data. The oblique photography data includes the image, the UAV's posture data when acquiring the image, the camera's posture data, etc. The first point cloud of the real scene to be reconstructed can be calculated through the acquired image, the UAV's posture data, and the camera's posture data.

[0026] Specifically, for the multiple images collected at each image collection point, the images can be preprocessed such as lens distortion correction, color correction, brightness and contrast adjustment, and then the feature points in each image can be identified. The identified feature points can be matched with each other in the multiple images to establish a correspondence between the images. The feature points extracted from different images are further matched to establish a relative position relationship between the feature points of different images in space. The matched feature points are further triangulated to obtain a preliminary three-dimensional structure, i.e., a sparse point cloud. On the basis of the sparse point cloud, more scene details are further estimated through images to obtain a dense point cloud. The dense point cloud is the first point cloud. For details, please refer to the relevant technology of generating three-dimensional models using oblique camera technology, which will not be described in detail here.

[0027] Step S102, collecting a second point cloud of the road to be reconstructed in the real scene by using a handheld laser radar.

[0028] In practical applications, drone aerial oblique photography may be blocked or affected by photography accuracy. The first point cloud may not reflect the slope, corners, ups and downs of the road on the ground, and manual handheld laser radar is required to collect the second point cloud of the road to reconstruct the real scene.

[0029] In one embodiment, a person can walk on the road of the real scene to be reconstructed by using a handheld laser radar, and scan the road surface with the handheld laser radar to obtain a second point cloud. The second point cloud can reflect the slope, bumps, potholes, corners, ups and downs and other details of the road surface of all roads to be reconstructed.

[0030] In another embodiment, an intermediate three-dimensional model of the real scene to be reconstructed can be generated based on the first point cloud, and the intermediate three-dimensional model can be input into the road recognition model to obtain at least one target road section to be reconstructed, and the target road section to be reconstructed can be scanned by a handheld laser radar to obtain a second point cloud.

[0031] Specifically, the road recognition model can be pre-trained so that the road recognition model can recognize slope sections, corner sections, occluded sections, etc. of the road. For example, the three-dimensional model samples of other scenes in the existing calculation can be used to mark the slope sections, corner sections, occluded sections, etc. in the three-dimensional model samples, and then the three-dimensional model samples are input into the road recognition model to obtain various types of detection box positions and section types. The loss rate is obtained by performing loss calculations such as mean square error and cross entropy based on the position of the detection box and the section type. The parameters of the road recognition model are adjusted based on the loss rate, and the training is returned to continue until the loss rate is less than a preset value or the number of training times reaches a preset number.

[0032] After obtaining an intermediate three-dimensional model of the real scene to be reconstructed through drone aerial oblique photography, the intermediate three-dimensional model can be input into the road recognition model to obtain the slope sections, corner sections, obstructed sections and other sections of the road in the real scene to be reconstructed as target sections, wherein the road recognition model outputs the detection frame positions (such as the latitude and longitude coordinates of the detection frame) and types of various sections, so that the surveyor can reach each target section with a handheld laser radar according to the detection frame position, and then scan each target section with the handheld laser radar to collect the second point cloud, wherein the laser radar scanning and collection of point clouds can refer to the existing technology and will not be described in detail here.

[0033] In yet another embodiment, the intermediate three-dimensional model may also be displayed, and the target section may be determined in response to the surveyor's selection operation on the slope section, corner section, blocked section, etc. of the road in the intermediate three-dimensional model.

[0034] This embodiment identifies slope sections, corner sections, obstructed sections and other sections as target sections from the intermediate model generated by the first point cloud, and further scans and collects the second point cloud for each target section through a handheld laser radar. Point clouds can be re-collected for slope sections, corner sections, obstructed sections and other sections. On the one hand, there is no need to scan the entire road section with a handheld laser radar to collect point clouds, thereby reducing the workload of surveyors. On the other hand, the data volume of the second point cloud can be reduced, thereby reducing the complexity of subsequent point cloud fusion.

[0035] like Figure 3 Shown is a schematic diagram of the true color point cloud collected from the road surface. Since the handheld laser radar is mainly used to scan the road surface, the point cloud of the road surface and objects near the road (such as guardrails and curbs) is more prominent in the schematic diagram of the true color point cloud.

[0036] Step S103, controlling the vehicle equipped with the laser radar to travel on the road to be reconstructed to collect the third point cloud.

[0037] In this embodiment, a laser radar can be installed on the roof of the survey vehicle, and the vehicle can be controlled to travel on the road where the real scene is to be reconstructed. During driving, the vehicle-mounted laser radar scans the surrounding environment of the road where the vehicle is located at a preset frequency to obtain a third point cloud. By scanning the surrounding environment of the road through the vehicle-mounted radar, point clouds of objects such as road guardrails, road traffic signs, and ramps on both sides of the road can be scanned to make up for the details that cannot be collected by drone aerial oblique photography.

[0038] like Figure 4 The figure shows the schematic diagram of the point cloud obtained by scanning and solving the vehicle-mounted laser radar. Figure 4 The point cloud in the image can reflect the detailed features of the road guardrails, ditches beside the road, ramps, etc.

[0039] Step S104: fuse the first point cloud, the second point cloud and the third point cloud to obtain a multi-data fused point cloud.

[0040] Specifically, the first point cloud, the second point cloud and the third point cloud are preprocessed to obtain the preprocessed first point cloud, the second point cloud and the third point cloud, and the preprocessed first point cloud, the second point cloud and the third point cloud are ICP-registered and fused to obtain a multi-data fused point cloud.

[0041] Before fusing the first point cloud, the second point cloud and the third point cloud, the first point cloud, the second point cloud and the third point cloud can be preprocessed, for example, the point cloud can be preprocessed such as denoising and smoothing to remove abnormal points in the point cloud, so as to improve the quality of the first point cloud, the second point cloud and the third point cloud, reduce the fusion error and improve the fusion accuracy.

[0042] In one embodiment, the preprocessed first point cloud and the second point cloud can be firstly subjected to ICP registration and fusion to obtain a fourth point cloud, the road point cloud can be removed from the preprocessed third point cloud to obtain a fifth point cloud, and the fourth point cloud and the fifth point cloud can be subjected to ICP registration and fusion to obtain a multi-data fusion point cloud. It is achieved that the road surface point cloud collected by the handheld laser radar is first fused into the first point cloud collected by the drone aerial oblique photography to obtain the fourth point cloud, so that the road surface in the fourth point cloud can reflect the ups and downs, corners, slope and other detailed features of the road, and then the fifth point cloud is obtained by removing the road surface point cloud from the third point cloud collected by the vehicle-mounted laser radar, and then the fifth point cloud is fused into the fourth point cloud, so that the point clouds of objects such as curbs, guardrails, traffic signs on both sides of the road collected by the vehicle-mounted laser radar are integrated into the fourth point cloud, so that the final multi-data fusion point cloud includes both the global features of the scene to be reconstructed and displayed collected during the drone aerial oblique photography, and can also be fused by the ups and downs, corners, slope and other detailed features of the road surface collected by the handheld laser radar, and can also be fused by the curbs, guardrails, traffic signs, slopes and other features of the road collected by the vehicle-mounted radar.

[0043] Among them, the point cloud registration fusion can be ICP (Iterative Closest Point, iterative closest point algorithm) registration, which is mainly to determine the rigid body change relationship between the two point clouds. Taking the first point cloud and the second point cloud as an example, the first point cloud can be used as a reference. For a certain point q in the first point cloud i , according to the preset constraints, find the n nearest points p in the second point cloud i , calculate the optimal matching parameters R and t, and calculate the error function, as follows: ; After each iteration, the second point cloud is transformed by coordinate rotation and translation using parameters R and t to obtain a new second point cloud and calculate point q i and the n nearest points p i If the average distance is less than the threshold and the number of iterations reaches the preset number, the iteration is stopped to obtain the final parameters R and t. The final parameters R and t are used to transform the second point cloud, that is, the coordinates of the first point cloud and the second point cloud under the same coordinates can be obtained to complete the point cloud registration. When the optimization is minimum, point q can be used. i and point p i The coordinates of point q can also be used to calculate the distance. i and point p iThe difference is calculated by the normal vector (fitting the normal vector through the adjacent points). The KD-tree (k-dimensional Tree) algorithm can be used to find the nearest neighbor. KD-tree is a data structure that partitions the k-dimensional data space. It is mainly used for the nearest neighbor search (Nearest Neighbor) and approximate nearest neighbor search (Approximate Nearest Neighbor) of key data in multidimensional space.

[0044] Similarly, after the first point cloud and the second point cloud are fused through ICP registration based on the first point cloud to obtain the fourth point cloud, when fusing the fourth point cloud and the fifth point cloud, the fourth point cloud containing the first point cloud can be used as a reference, and the fourth point cloud and the fifth point cloud can be further fused through ICP registration to obtain a multi-data fused point cloud.

[0045] Step S105, generating a three-dimensional model of the real scene to be reconstructed based on the multi-data fusion point cloud.

[0046] Specifically, a triangular mesh can be generated based on multi-data fusion point cloud to obtain an initial three-dimensional surface model of the real scene to be reconstructed, and the initial three-dimensional surface model is smoothed to obtain a three-dimensional model of the real scene to be reconstructed. For details, please refer to the method of generating a three-dimensional model from a point cloud, which will not be described in detail here.

[0047] like Figure 5 The figure shows a schematic diagram of generating a triangular mesh by fusion of multi-data point clouds, that is, connecting three adjacent points in the multi-data fusion point clouds to obtain a triangular mesh, and multiple triangular meshes form an initial three-dimensional surface model of the real scene to be reconstructed, and after smoothing, a three-dimensional model of the real scene to be reconstructed is obtained.

[0048] In the embodiment of the present invention, a first point cloud is obtained by performing oblique photography of a real scene to be reconstructed by a drone, a second point cloud is collected by a handheld laser radar on a road of the real scene to be reconstructed, and a third point cloud is collected by controlling a vehicle equipped with a laser radar to drive on the road of the real scene to be reconstructed. After the first point cloud, the second point cloud and the third point cloud are fused to obtain a multi-data fused point cloud, a three-dimensional model of the real scene to be reconstructed is generated based on the multi-data fused point cloud, thereby obtaining an overall and rough first point cloud of the area to be reconstructed by high-altitude oblique photography of the drone, and then collecting a second point cloud of the road by a handheld laser radar to collect details such as the ups and downs, slopes and corners of the road, and further collecting a third point cloud by driving on the road by a vehicle-mounted laser radar to collect details such as slopes, guardrails, and traffic signs on the road, so that the fused multi-data fused point cloud can represent the real scene of the real scene to be reconstructed. After the three-dimensional model is applied to the game, the ups and downs, potholes, bumps, slopes, corners, slopes, guardrails, ditch depths and traffic signs of the road in the real scene can be realistically and accurately simulated, thereby improving the simulation effect of the game.

[0049] Figure 6 The application architecture diagram of the 3D model generation system based on multi-data fusion provided by an embodiment of the present invention is shown. The 3D model generation system based on multi-data fusion in this embodiment specifically includes the following units: A first point cloud acquisition unit 601 is used to control a drone equipped with a camera to perform oblique photography on a real scene to be reconstructed to obtain oblique photography data, and to generate a first point cloud of the real scene to be reconstructed based on the oblique photography data; A second point cloud acquisition unit 602 is used to collect a second point cloud of the road of the real scene to be reconstructed by using a handheld laser radar; A third point cloud acquisition unit 603 is used to control the vehicle equipped with the laser radar to drive on the road of the real scene to be reconstructed to collect a third point cloud; A point cloud fusion unit 604 is used to fuse the first point cloud, the second point cloud and the third point cloud to obtain a multi-data fused point cloud; The three-dimensional model generating unit 605 is used to generate the three-dimensional model of the real scene to be reconstructed based on the multi-data fusion point cloud.

[0050] As a further technical solution of the present invention, the following units are also included: A satellite image acquisition unit, used to acquire a satellite image of the real scene to be reconstructed; A flight path generation unit is used to generate a flight path of the UAV based on the satellite image, wherein the flight path includes UAV waypoints and image acquisition points.

[0051] As a further technical solution of the present invention, the first point cloud acquisition unit 601 specifically includes the following modules: A UAV flight control module, used to control a UAV equipped with a camera to fly over the real scene to be reconstructed according to the flight path; An image acquisition control module, used for controlling the camera to acquire images to obtain oblique photography images when flying to the image acquisition point; The first point cloud generation module is used to generate a first point cloud of the real scene to be reconstructed based on the oblique photography image and the posture data of the drone at the image acquisition point.

[0052] As a further technical solution of the present invention, the following units are specifically included: An intermediate three-dimensional model generating unit, configured to generate an intermediate three-dimensional model of the real scene to be reconstructed based on the first point cloud; The intermediate three-dimensional model detection unit is used to input the intermediate three-dimensional model into the road recognition model to obtain at least one target road section to be reconstructed.

[0053] As a further technical solution of the present invention, the second point cloud acquisition unit 602 specifically includes the following modules: The target road section scanning module is used to scan the target road section to be reconstructed by using a handheld laser radar to obtain a second point cloud.

[0054] As a further technical solution of the present invention, the point cloud fusion unit 604 specifically includes the following modules: A point cloud preprocessing module, used for preprocessing the first point cloud, the second point cloud and the third point cloud to obtain the preprocessed first point cloud, the second point cloud and the third point cloud; The point cloud fusion module is used to perform ICP registration and fusion on the preprocessed first point cloud, the second point cloud and the third point cloud to obtain a multi-data fused point cloud.

[0055] As a further technical solution of the present invention, the point cloud fusion module specifically includes the following submodules: The first fusion submodule is used to perform ICP registration fusion on the preprocessed first point cloud and the second point cloud to obtain a fourth point cloud; A road surface point cloud removal sub-module is used to remove the road surface point cloud from the pre-processed third point cloud to obtain a fifth point cloud; The second fusion submodule is used to perform ICP registration fusion on the fourth point cloud and the fifth point cloud to obtain a multi-data fusion point cloud.

[0056] As a further technical solution of the present invention, the three-dimensional model generation unit 605 specifically includes the following modules: An initial three-dimensional model generation module is used to generate a triangular mesh based on the multi-data fusion point cloud to obtain an initial three-dimensional surface model of the real scene to be reconstructed; The model smoothing module is used to smooth the initial three-dimensional surface model to obtain the three-dimensional model of the real scene to be reconstructed.

[0057] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0058] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0059] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0060] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the attached claims.

[0061] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A three-dimensional model generation method based on multi-data fusion, characterized in that: The specific steps include: Controlling a drone equipped with a camera to perform oblique photography on a real scene to be reconstructed to obtain oblique photography data, and generating a first point cloud of the real scene to be reconstructed based on the oblique photography data; Collecting a second point cloud of the road of the real scene to be reconstructed by a handheld laser radar; Controlling a vehicle equipped with a laser radar to drive on a road of the real scene to be reconstructed to collect a third point cloud; fusing the first point cloud, the second point cloud and the third point cloud to obtain a multi-data fused point cloud; A three-dimensional model of the real scene to be reconstructed is generated based on the multi-data fusion point cloud.

2. The method for generating a three-dimensional model based on multi-data fusion according to claim 1, characterized in that: Before controlling the drone equipped with a camera to perform oblique photography on the real scene to be reconstructed to obtain oblique photography data, the following steps are also included: Acquire a satellite image of the real scene to be reconstructed; A flight path of the UAV is generated based on the satellite image, wherein the flight path includes UAV waypoints and image acquisition points.

3. The method for generating a three-dimensional model based on multi-data fusion according to claim 2, characterized in that: Controlling a drone equipped with a camera to perform oblique photography on a real scene to be reconstructed to obtain oblique photography data, and generating a first point cloud of the real scene to be reconstructed based on the oblique photography data, specifically includes the following steps: Controlling a drone equipped with a camera to fly over the real scene to be reconstructed according to the flight path; When flying to the image acquisition point, controlling the camera to acquire an image to obtain an oblique photography image; A first point cloud of the real scene to be reconstructed is generated based on the oblique photography image and the position and posture data of the drone at the image acquisition point.

4. The method for generating a three-dimensional model based on multi-data fusion according to claim 1, characterized in that: Before collecting the second point cloud of the road of the real scene to be reconstructed by the handheld laser radar, the following steps are specifically included: Generating an intermediate three-dimensional model of the real scene to be reconstructed based on the first point cloud; The intermediate three-dimensional model is input into a road recognition model to obtain at least one target road section to be reconstructed.

5. The method for generating a three-dimensional model based on multi-data fusion according to claim 4, characterized in that: Collecting a second point cloud of the road of the real scene to be reconstructed by a handheld laser radar specifically includes the following steps: The target road section to be reconstructed is scanned by a handheld laser radar to obtain a second point cloud.

6. The method for generating a three-dimensional model based on multi-data fusion according to claim 4, characterized in that: The first point cloud, the second point cloud and the third point cloud are fused to obtain a multi-data fused point cloud, which specifically includes the following steps: Preprocessing the first point cloud, the second point cloud, and the third point cloud to obtain a preprocessed first point cloud, a second point cloud, and a third point cloud; The preprocessed first point cloud, the second point cloud and the third point cloud are fused by ICP registration to obtain a multi-data fused point cloud.

7. The method for generating a three-dimensional model based on multi-data fusion according to claim 6, characterized in that: The preprocessed first point cloud, the second point cloud and the third point cloud are subjected to ICP registration and fusion to obtain a multi-data fusion point cloud, which specifically includes the following steps: Perform ICP registration and fusion on the preprocessed first point cloud and the second point cloud to obtain a fourth point cloud; The road surface point cloud is removed from the preprocessed third point cloud to obtain a fifth point cloud; The fourth point cloud and the fifth point cloud are fused by ICP registration to obtain a multi-data fused point cloud.

8. The method for generating a three-dimensional model based on multi-data fusion according to any one of claims 1 to 7, characterized in that: Generating the three-dimensional model of the real scene to be reconstructed based on the multi-data fusion point cloud specifically includes the following steps: Generate a triangular mesh based on the multi-data fusion point cloud to obtain an initial three-dimensional surface model of the real scene to be reconstructed; The initial three-dimensional surface model is smoothed to obtain a three-dimensional model of the real scene to be reconstructed.

9. A three-dimensional model generation system based on multi-data fusion, characterized in that: It includes the following units: A first point cloud acquisition unit is used to control a drone equipped with a camera to perform oblique photography on the real scene to be reconstructed to obtain oblique photography data, and generate a first point cloud of the real scene to be reconstructed based on the oblique photography data; A second point cloud acquisition unit is used to collect a second point cloud of the road of the real scene to be reconstructed by using a handheld laser radar; A third point cloud acquisition unit, used to control a vehicle equipped with a laser radar to drive on the road of the real scene to be reconstructed to collect a third point cloud; A point cloud fusion unit, used for fusing the first point cloud, the second point cloud and the third point cloud to obtain a multi-data fused point cloud; A three-dimensional model generating unit is used to generate a three-dimensional model of the real scene to be reconstructed based on the multi-data fusion point cloud.

10. The three-dimensional model generation system based on multi-data fusion according to claim 9, characterized in that: Also includes the following units: A satellite image acquisition unit, used to acquire a satellite image of the real scene to be reconstructed; A flight path generation unit is used to generate a flight path of the UAV based on the satellite image, wherein the flight path includes UAV waypoints and image acquisition points.

Images