Laser point cloud fusion method and device, server and computer readable storage medium

Abstract

This application relates to the field of image processing technology and provides a laser point cloud fusion method, apparatus, server, and computer-readable storage medium. The method includes: acquiring a point cloud to be processed; wherein the point cloud to be processed includes a first point cloud to be processed and a second point cloud to be processed; fusing the first point cloud to be processed and the second point cloud to be processed to obtain a target point cloud. Therefore, this application can improve the accuracy of laser point cloud fusion.

Description

Technical Field

[0001] This application belongs to the field of point cloud technology, and particularly relates to a laser point cloud fusion method, device, server, and computer-readable storage medium. Background Technology

[0002] With the continuous development of sensing technology and surveying equipment, data sources have become increasingly diverse. In the field of 3D reconstruction, comprehensive point cloud data is needed to achieve better reconstruction results. However, the fusion accuracy between existing different types of point cloud data is relatively low. Summary of the Invention

[0003] This application provides a laser point cloud fusion method, apparatus, server, and computer-readable storage medium, which can solve the technical problem of low point cloud fusion accuracy in the prior art.

[0004] In a first aspect, embodiments of this application provide a laser point cloud fusion method, including:

[0005] Obtain a point cloud to be processed; wherein the point cloud to be processed includes a first point cloud to be processed and a second point cloud to be processed;

[0006] The target point cloud is obtained by fusing the first point cloud to be processed and the second point cloud to be processed.

[0007] In one possible implementation of the first aspect, acquiring the point cloud to be processed includes:

[0008] Acquire the first image to be processed and the first laser data;

[0009] Acquire the second image to be processed and the second laser data;

[0010] Coordinate system calibration is performed on the first coordinate system in which the first laser data is located and the second coordinate system in which the second laser data is located;

[0011] Match the first image to be processed and the second image to be processed;

[0012] A first point cloud to be processed is generated based on the matched first image to be processed and the first laser data after coordinate system calibration.

[0013] A second point cloud to be processed is generated based on the matched second image to be processed and the second laser data after coordinate system calibration.

[0014] In one possible implementation of the first aspect, matching the first image to be processed and the second image to be processed includes:

[0015] Extract the first feature points from the first image to be processed;

[0016] Extract the second feature points from the second image to be processed;

[0017] Select the target feature points that are common to the first feature point and the second feature point.

[0018] In one possible implementation of the first aspect, fusing the first point cloud to be processed and the second point cloud to be processed to obtain a target point cloud includes:

[0019] Obtain pose information based on the target feature points;

[0020] Based on the pose information, the first point cloud to be processed and the second point cloud to be processed are fused to obtain the target point cloud.

[0021] In one possible implementation of the first aspect, after fusing the first point cloud to be processed and the second point cloud to be processed to obtain the target point cloud, the method further includes:

[0022] Optimize the target point cloud.

[0023] Secondly, embodiments of this application provide a laser point cloud fusion device, comprising:

[0024] An acquisition module is used to acquire a point cloud to be processed; wherein the point cloud to be processed includes a first point cloud to be processed and a second point cloud to be processed.

[0025] The fusion module is used to fuse the first point cloud to be processed and the second point cloud to be processed to obtain the target point cloud.

[0026] In an optional implementation of the second aspect, the acquisition module includes:

[0027] The first acquisition submodule is used to acquire the first image to be processed and the first laser data;

[0028] The second acquisition submodule is used to acquire the second image to be processed and the second laser data;

[0029] The calibration submodule is used to perform coordinate system calibration on the first coordinate system in which the first laser data is located and the second coordinate system in which the second laser data is located.

[0030] A matching submodule is used to match the first image to be processed and the second image to be processed.

[0031] The first generation submodule is used to generate a first point cloud to be processed based on the matched first image to be processed and the first laser data after coordinate system calibration.

[0032] The second generation submodule is used to generate a second point cloud to be processed based on the matched second image to be processed and the second laser data after coordinate system calibration.

[0033] In an alternative implementation of the second aspect, the matching submodule includes:

[0034] The first extraction unit is used to extract the first feature points of the first image to be processed;

[0035] The second extraction unit is used to extract the second feature points of the second image to be processed;

[0036] The filtering unit is used to filter out target feature points that are common to the first feature point and the second feature point.

[0037] In an alternative implementation of the second aspect, the fusion module includes:

[0038] The third acquisition submodule is used to acquire pose information based on the target feature points;

[0039] The fusion submodule is used to fuse the first point cloud to be processed and the second point cloud to be processed according to the pose information to obtain the target point cloud.

[0040] In an alternative embodiment of the second aspect, the apparatus further includes:

[0041] An optimization module is used to optimize the target point cloud.

[0042] Thirdly, embodiments of this application provide a server, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0043] Fourthly, embodiments of this application provide a readable computer-readable storage medium, wherein the computer program, when executed by a processor, implements the method described in the first aspect above.

[0044] The beneficial effects of the embodiments in this application compared with the prior art are:

[0045] This application embodiment obtains a point cloud to be processed; wherein the point cloud to be processed includes a first point cloud to be processed and a second point cloud to be processed; the first point cloud to be processed and the second point cloud to be processed are fused to obtain a target point cloud, thereby improving the accuracy of laser point cloud fusion. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a schematic flowchart of the laser point cloud fusion method provided in the embodiments of this application;

[0048] Figure 2 This is a structural block diagram of the laser point cloud fusion device provided in the embodiments of this application;

[0049] Figure 3 This is a schematic diagram of the server structure provided in an embodiment of this application. Detailed Implementation

[0050] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0051] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0052] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0053] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0054] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0055] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0056] The technical solutions provided in the embodiments of this application will be described below through specific examples.

[0057] See Figure 1 This is a flowchart illustrating the laser point cloud fusion method provided in this application embodiment. It is intended as an example and not a limitation. This method can be applied to a server and may include the following steps:

[0058] Step S101: Obtain the point cloud to be processed.

[0059] The point cloud to be processed includes a first point cloud to be processed and a second point cloud to be processed.

[0060] It should be noted that the first and second point clouds to be processed are generated based on images of the same target object taken by different types of laser scanning equipment.

[0061] The first point cloud to be processed and the second point cloud to be processed correspond to the same shooting object. The first point cloud to be processed can be obtained by a ground laser scanner, and the second point cloud to be processed can be obtained by drone aerial photography.

[0062] In practical applications, acquiring the point cloud to be processed includes:

[0063] Step S101-1: Obtain the first image to be processed and the first laser data.

[0064] Step S101-2: Obtain the second image to be processed and the second laser data.

[0065] The first image to be processed and the first laser data can be obtained by a ground-based laser scanner. The first image to be processed can be a panoramic image, and the first laser data contains the depth value of each pixel in the first image to be processed. The second image to be processed and the second laser data can be obtained by aerial photography from a drone. The second image to be processed can be a regular image, and the second laser data contains the depth value of each pixel in the second image to be processed. In addition, the ground-based laser scanner can also acquire the first GPS data of the target object, and the drone can also acquire the second GPS data of the target object.

[0066] Step S101-3: Perform coordinate system calibration on the first coordinate system where the first laser data is located and the second coordinate system where the second laser data is located.

[0067] Specifically, the first GPS data is located in the first coordinate system, and the second GPS data is located in the second coordinate system. The first GPS data and the second GPS data are aligned in coordinates, that is, coordinate system calibration is performed on the first coordinate system where the first laser data is located and the second coordinate system where the second laser data is located.

[0068] Step S101-4: Match the first image to be processed and the second image to be processed.

[0069] For example, matching the first image to be processed and the second image to be processed includes:

[0070] Step S101-4-1: Extract the first feature points of the first image to be processed.

[0071] For example, a feature extraction algorithm is used to extract the first feature points of the first image to be processed.

[0072] Step S101-4-2: Extract the second feature points of the second image to be processed.

[0073] For example, a feature extraction algorithm is used to extract the first feature points of the first image to be processed.

[0074] It should be noted that the feature extraction algorithm used is the AKAZE algorithm. The AKAZE feature algorithm is an improved version of the SIFT feature algorithm, but it does not use Gaussian blur to construct the scale space because Gaussian blur has the disadvantage of losing edge information. Instead, it uses nonlinear diffusion filtering to construct the scale space, thereby preserving more edge features of the image.

[0075] Step S101-4-3: Filter out the target feature points that are common to the first feature point and the second feature point.

[0076] For example, the first feature point and the second feature point are input into a pre-trained neural network, and the common target feature point is output.

[0077] Optionally, before filtering out the target feature points shared by the first and second feature points, the process may also include training a neural network.

[0078] It is understandable that the existing algorithm is first used to extract a rough match, the SFM algorithm is used for dense reconstruction, effective points are selected, and a neural network is trained to achieve the matching of drone shooting with panoramic images.

[0079] The training process of a neural network includes an initial step and an iterative step. The initial step involves a coarse matching between the ordinary image and the panoramic image. This can be done by slicing the panoramic image already in the model and matching the feature points of the resulting ordinary image with those slices. Alternatively, the initial step can convert a drone image into a portion of the panoramic image, achieving panoramic-to-panoramic matching. The iterative step directly uses the matching between the drone image and the panoramic image for iterative processing. The neural network model can be a neural network model that performs feature point matching between ordinary and panoramic images, such as the SuperGlue model, a SuperGlue variant, or the OANet model.

[0080] Step S101-5: Generate a first point cloud to be processed based on the matched first image to be processed and the first laser data after coordinate system calibration.

[0081] For example, the three-dimensional coordinates of the first point cloud to be processed are obtained according to the following formula:

[0082] ,

[0083] Where (u, v) are the pixel coordinates of each feature point in the first image to be processed, d is the depth value of each pixel in the first image to be processed, K is the intrinsic parameter of the ground laser scanner, and (X, Y, Z) are the three-dimensional coordinates of the first point cloud to be processed.

[0084] Step S101-6: Generate a second point cloud to be processed based on the matched second image to be processed and the second laser data after coordinate system calibration.

[0085] For example, the three-dimensional coordinates of the second point cloud to be processed are obtained according to the following formula:

[0086] ,

[0087] Where (u, v) are the pixel coordinates of each feature point in the second image to be processed, d is the depth value of each pixel in the second image to be processed, K is the intrinsic parameter of the UAV, and (X, Y, Z) are the three-dimensional coordinates of the second point cloud to be processed.

[0088] Step S102: Fuse the first point cloud to be processed and the second point cloud to be processed to obtain the target point cloud.

[0089] In practical applications, the first and second point clouds to be processed are fused to obtain the target point cloud, including:

[0090] Step S102-1: Obtain pose information based on target feature points.

[0091] Specifically, the pose information is obtained by matching target feature points using the SFM algorithm.

[0092] Step S102-2: Based on the pose information, fuse the first point cloud to be processed and the second point cloud to be processed to obtain the target point cloud.

[0093] For example, based on the pose information, the first point cloud to be processed and the second point cloud to be processed are fused using the ICP algorithm to obtain the target point cloud.

[0094] In one optional implementation, after fusing the first point cloud to be processed and the second point cloud to be processed to obtain the target point cloud, the method further includes:

[0095] Optimize the target point cloud.

[0096] Understandably, after fusing the first and second point clouds to obtain the target point cloud, some point cloud overlap will occur. First, the synthesized point cloud is rasterized. For each raster, the density A and visibility B (the angle between the point cloud normal and the line connecting the camera position and the point cloud) of the two parts of the point cloud are calculated, along with the confidence C = A * (cosB). The part with higher confidence is selected for the final synthesized point cloud, thus optimizing the target point cloud.

[0097] In this embodiment of the application, a point cloud to be processed is obtained; wherein the point cloud to be processed includes a first point cloud to be processed and a second point cloud to be processed; the first point cloud to be processed and the second point cloud to be processed are fused to obtain a target point cloud, thereby improving the accuracy of laser point cloud fusion.

[0098] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0099] Corresponding to the method described in the above embodiments, Figure 2 A structural block diagram of the laser point cloud fusion device provided in the embodiments of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0100] Reference Figure 2 The device includes:

[0101] The acquisition module 21 is used to acquire the point cloud to be processed; wherein, the point cloud to be processed includes a first point cloud to be processed and a second point cloud to be processed.

[0102] The fusion module 22 is used to fuse the first point cloud to be processed and the second point cloud to be processed to obtain the target point cloud.

[0103] In one optional implementation, the acquisition module includes:

[0104] The first acquisition submodule is used to acquire the first image to be processed and the first laser data;

[0105] The second acquisition submodule is used to acquire the second image to be processed and the second laser data;

[0106] The calibration submodule is used to perform coordinate system calibration on the first coordinate system in which the first laser data is located and the second coordinate system in which the second laser data is located.

[0107] A matching submodule is used to match the first image to be processed and the second image to be processed.

[0108] The first generation submodule is used to generate a first point cloud to be processed based on the matched first image to be processed and the first laser data after coordinate system calibration.

[0109] The second generation submodule is used to generate a second point cloud to be processed based on the matched second image to be processed and the second laser data after coordinate system calibration.

[0110] In one optional implementation, the matching submodule includes:

[0111] The first extraction unit is used to extract the first feature points of the first image to be processed;

[0112] The second extraction unit is used to extract the second feature points of the second image to be processed;

[0113] The filtering unit is used to filter out target feature points that are common to the first feature point and the second feature point.

[0114] In one optional implementation, the fusion module includes:

[0115] The third acquisition submodule is used to acquire pose information based on the target feature points;

[0116] The fusion submodule is used to fuse the first point cloud to be processed and the second point cloud to be processed according to the pose information to obtain the target point cloud.

[0117] In one optional embodiment, the device further includes:

[0118] An optimization module is used to optimize the target point cloud.

[0119] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0120] Figure 3 This is a schematic diagram of the server structure provided in an embodiment of this application. Figure 3 As shown, the server 3 in this embodiment includes: at least one processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the at least one processor 30. When the processor 30 executes the computer program 32, it implements the steps in any of the above-described method embodiments.

[0121] The server 3 may be a computing device such as a cloud server. This server may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will understand that... Figure 3 This is merely an example of server 3 and does not constitute a limitation on server 3. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0122] The processor 30 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0123] In some embodiments, the memory 31 may be an internal storage unit of the server 3, such as a hard drive or memory of the server 3. In other embodiments, the memory 31 may be an external storage device of the server 3, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the server 3. Furthermore, the memory 31 may include both internal storage units and external storage devices of the server 3. The memory 31 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0124] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0125] This application also provides a readable computer-readable storage medium, preferably a computer-readable storage medium, which stores a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0126] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0127] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0128] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0129] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0130] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0131] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0132] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A laser point cloud fusion method, characterized in that, include: The process involves acquiring a point cloud to be processed, comprising a first point cloud and a second point cloud; acquiring a first image to be processed and first laser data; the first image to be processed and the first laser data are obtained by a ground-based laser scanner, the first image to be processed being a panoramic image, and the first laser data containing the depth value of each pixel in the first image to be processed; acquiring a second image to be processed and second laser data; the second image to be processed and the second laser data are obtained by aerial photography from a drone, the second image to be processed being a regular image, and the second laser data containing the depth value of each pixel in the second image to be processed; the ground-based laser scanner also acquires the target... The first GPS data of the target object is acquired by the drone, and the second GPS data of the target object is acquired by the drone. Coordinate system calibration is performed on the first coordinate system where the first laser data is located and the second coordinate system where the second laser data is located. The first GPS data is located in the first coordinate system, and the second GPS data is located in the second coordinate system. Coordinate alignment is performed on the first GPS data and the second GPS data, i.e., coordinate system calibration is performed on the first coordinate system where the first laser data is located and the second coordinate system where the second laser data is located. The first image to be processed and the second image to be processed are matched, including: extracting the first feature points of the first image to be processed; extracting the second... The second feature point of the image to be processed; the feature extraction algorithm is the AKAZE algorithm, which does not use Gaussian blur to construct the scale space, but uses nonlinear diffusion filtering to construct the scale space, thereby preserving more edge features of the image; the target feature points common to the first feature point and the second feature point are selected; the first feature point and the second feature point are input into a pre-trained neural network, and the common target feature points are output; a first point cloud to be processed is generated according to the matched first image to be processed and the first laser data after coordinate system calibration; a second point cloud to be processed is generated according to the matched second image to be processed and the second laser data after coordinate system calibration; the training process of the neural network includes an initial step and an iterative step, wherein the initial step is a coarse matching between the ordinary image and the panoramic image, slicing the panoramic image already in the neural network, and matching the feature points of the ordinary image after the panoramic image slices with the feature points of the ordinary image; the initial step also converts the image of the UAV into a part of the panoramic image to achieve panoramic matching; the iterative step directly uses the matching of the UAV image and the panoramic image for iteration; wherein, the model of the neural network is a neural network model for feature point matching between ordinary images and panoramic images, including the SuperGlue model, SuperGlue model variants or OANet model; The target point cloud is obtained by fusing the first point cloud to be processed and the second point cloud to be processed.

2. The laser point cloud fusion method as described in claim 1, characterized in that, By fusing the first point cloud to be processed and the second point cloud to be processed, a target point cloud is obtained, including: Obtain pose information based on the target feature points; Based on the pose information, the first point cloud to be processed and the second point cloud to be processed are fused to obtain the target point cloud.

3. The laser point cloud fusion method as described in claim 1, characterized in that, After fusing the first point cloud to be processed and the second point cloud to be processed to obtain the target point cloud, the process further includes: Optimize the target point cloud.

4. A laser point cloud fusion device, characterized in that, include: The acquisition module is used to acquire point clouds to be processed; wherein the point clouds to be processed include a first point cloud and a second point cloud; by acquiring a first image to be processed and first laser data; the first image to be processed and the first laser data are obtained by a ground laser scanner, the first image to be processed is a panoramic image, and the first laser data contains the depth value of each pixel in the first image to be processed; acquiring a second image to be processed and second laser data; the second image to be processed and the second laser data are obtained by aerial photography from a UAV, the second image to be processed is a normal image, and the second laser data contains the depth value of each pixel in the second image to be processed; a ground laser scanner; The system also acquires first GPS data of the target object, and the drone acquires second GPS data of the target object; coordinate system calibration is performed on the first coordinate system where the first laser data is located and the second coordinate system where the second laser data is located; the first GPS data is located in the first coordinate system and the second GPS data is located in the second coordinate system, and the first GPS data and the second GPS data are aligned, that is, coordinate system calibration is performed on the first coordinate system where the first laser data is located and the second coordinate system where the second laser data is located; the system matches the first image to be processed and the second image to be processed, including: extracting the first feature points of the first image to be processed; extracting... The second feature point of the second image to be processed; the feature extraction algorithm is the AKAZE algorithm, which does not use Gaussian blur to construct the scale space, but uses nonlinear diffusion filtering to construct the scale space, thereby preserving more edge features of the image; the target feature point common to the first feature point and the second feature point is selected; the first feature point and the second feature point are input into the pre-trained neural network, and the common target feature point is output; the first point cloud to be processed is generated according to the matched first image to be processed and the first laser data after coordinate system calibration; the second point cloud to be processed is generated according to the matched second image to be processed and the second laser data after coordinate system calibration; the training process of the neural network includes an initial step and an iterative step, wherein the initial step is a coarse matching between the ordinary image and the panoramic image, the panoramic image already in the neural network is sliced, and the feature points of the ordinary image after the panoramic image slice are matched with the feature points of the ordinary image; the initial step also converts the image of the UAV into a part of the panoramic image to achieve panoramic matching; the iterative step directly uses the matching of the UAV image and the panoramic image for iteration; wherein, the model of the neural network is a neural network model for feature point matching between ordinary images and panoramic images, including the SuperGlue model, SuperGlue model variants or OANet model; The fusion module is used to fuse the first point cloud to be processed and the second point cloud to be processed to obtain the target point cloud.

5. A server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 3.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 3.

Images