Method, device and system for registering point cloud data

Abstract

The application provides a point cloud data registration method, device and system, the method comprising: obtaining historical point cloud data and historical orthographic image data; obtaining current point cloud data and current orthographic image data; using the historical orthographic image data, the historical point cloud data and the current orthographic image data to perform registration processing on the current point cloud data to obtain registered current point cloud data. For the first time, the orthographic image data is included in the point cloud data registration step, improving the accuracy of the registered current point cloud data, achieving high-precision and high-efficiency registration of multi-period point cloud data, and combining with the image data, improving the accuracy of ground monitoring point position layout selection, thereby solving the problem of low registration accuracy of current point cloud data and historical point cloud data in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of point cloud data processing, in particular to a point cloud data registration method and device, a computer readable storage medium and a point cloud data registration system. BACKGROUND

[0002] A three-dimensional laser scanner records the laser information reflected by a target object through an instrument, thereby establishing a three-dimensional model of the target. The scanner scans a specified range through a rapidly rotating mirror, determines the distance of the target by using the time of a laser pulse to and fro, and obtains the three-dimensional coordinates of the measured target through encoder conversion. The three-dimensional laser scanning technology can reflect the displacement of a slope relatively intuitively, but is susceptible to environmental influences, has a relatively limited monitoring range, and is subject to many factors such as manual intervention, which jointly restrict the registration accuracy of current point cloud data and historical point cloud data. SUMMARY

[0003] The main purpose of the present application is to provide a point cloud data registration method, device, computer readable storage medium and point cloud data registration system, so as to solve the problem of low registration accuracy of current point cloud data and historical point cloud data in the prior art.

[0004] According to an aspect of an embodiment of the present application, a point cloud data registration method is provided, which comprises: obtaining historical point cloud data and historical orthographic image data, wherein the historical point cloud data is used to represent the point cloud data of a monitoring area obtained in a historical time period, and the historical orthographic image data is used to represent the orthographic image data of the monitoring area obtained in the historical time period; obtaining current point cloud data and current orthographic image data; performing registration processing on the current point cloud data by using the historical orthographic image data, the historical point cloud data and the current orthographic image data, to obtain registered current point cloud data.

[0005] Optionally, the current point cloud data is registered by using the historical orthographic image data, the historical point cloud data and the current orthographic image data, to obtain registered current point cloud data, including: performing matching processing on the historical orthographic image data and the current orthographic image data to obtain a matched current image feature point set and a matched historical image feature point set, points in the matched current image feature point set corresponding to points in the matched historical image feature point set one by one; obtaining a current point cloud feature point set according to the matched current image feature point set and the current point cloud data; obtaining a historical point cloud feature point set according to the matched historical image feature point set and the historical point cloud data; determining a target translation and a target rotation matrix according to the historical point cloud feature point set and the current point cloud feature point set, wherein the target translation is used to represent a translation related to the current point cloud feature point set and the historical point cloud feature point set, and the target rotation matrix is used to represent a rotation matrix related to the current point cloud feature point set and the historical point cloud feature point set; and performing registration on the current point cloud data according to the current point cloud feature point set, the target rotation matrix and the target translation, to obtain the registered current point cloud data.

[0006] Optionally, determining the target translation and the target rotation matrix according to the historical point cloud feature point set and the current point cloud feature point set includes: determining a plurality of initial nearest neighbor points, the initial nearest neighbor points being used to represent a point in the historical point cloud feature point set closest to a point in the current point cloud feature point set; determining a distance between the initial nearest neighbor point and a target point as a neighbor distance, the target point being a point in the current point cloud feature point set matched with the initial nearest neighbor point, the initial nearest neighbor point and the target point having the same values of a first coordinate axis and a second coordinate axis; in a case where the initial nearest neighbor point distance is less than a first distance threshold, taking the initial nearest neighbor point as a target nearest neighbor point; determining an average distance of the target nearest neighbor points, the average distance of the target nearest neighbor points being used to represent an average value of the neighbor distances of all the target nearest neighbor points; and determining the target translation and the target rotation matrix according to the average distance of the target nearest neighbor points, the historical point cloud feature point set and the current point cloud feature point set.

[0007] Optionally, the current point cloud data is registered according to the current point cloud feature point set, the target rotation matrix and the target translation amount, to obtain registered current point cloud data, including: registering the current point cloud data according to the current point cloud feature point set, the target rotation matrix and the target translation amount, to obtain initial registered current point cloud data; determining a plurality of feature point distances, the feature point distance being used to represent the distance between one point in the initial registered current point cloud data and a corresponding feature point in the historical point cloud feature point set; determining an average distance of feature points according to all the feature point distances; in a case where the average distance of feature points is less than a second distance threshold, taking the initial registered current point cloud data as the registered current point cloud data; in a case where the average distance of feature points is greater than or equal to the second distance threshold, determining a plurality of initial nearest neighbor points again.

[0008] Optionally, the current point cloud feature point set is obtained according to the matched current image feature point set and the current point cloud data, including: obtaining a plurality of first coordinate values, wherein the first coordinate value is used to represent the coordinate value of a point in the matched current image feature point set in the direction of a first coordinate axis and the direction of a second coordinate axis; constructing a first cylindrical bounding box according to the first coordinate value and the current point cloud data, wherein in the first cylindrical bounding box, the first coordinate value is taken as the center of the first cylindrical bounding box, a threshold radius is taken as the radius of the first cylindrical bounding box, and the difference between the highest point and the lowest point of the third coordinate axis in the point corresponding to the first coordinate value in the current point cloud data is taken as the height of the first cylindrical bounding box; determining the point with the maximum third coordinate axis in the first cylindrical bounding box as a current point cloud feature point; determining the current point cloud feature point set, wherein the current point cloud feature point set includes all the current point cloud feature points.

[0009] Optionally, the obtaining the historical point cloud feature point set according to the matched historical image feature point set and the historical point cloud data comprises: obtaining a plurality of second coordinate values, wherein the second coordinate values are used to represent the coordinate values of a point in the matched historical image feature point set in the direction of a first coordinate axis and the direction of a second coordinate axis; constructing a second cylinder bounding box according to the second coordinate values and the historical point cloud data, wherein in the second cylinder bounding box, the second coordinate values are taken as the center of the second cylinder bounding box, a threshold radius is taken as the radius of the second cylinder bounding box, and the difference between the highest point and the lowest point of the third coordinate axis in the points corresponding to the second coordinate values in the historical point cloud data is taken as the height of the second cylinder bounding box; determining the point with the largest third coordinate axis in the second cylinder bounding box as a historical point cloud feature point; and determining the historical point cloud feature point set, wherein the historical point cloud feature point set comprises all the historical point cloud feature points.

[0010] Optionally, after the current point cloud data is registered by using the historical orthographic image data, the historical point cloud data and the current orthographic image data to obtain registered current point cloud data, the method further comprises: constructing a difference model according to the registered current point cloud data and the historical point cloud data, wherein the difference model is used to represent the positional difference relationship between the registered current point cloud data and the historical point cloud data; and determining the region where the open-pit coal mine slope is undergoing displacement according to the difference model.

[0011] According to another aspect of the embodiment of the present application, a point cloud data registration device is also provided, which comprises a first obtaining unit, a second obtaining unit and a registration unit. The first obtaining unit is used to obtain historical point cloud data and historical orthographic image data, wherein the historical point cloud data is used to represent the point cloud data of a monitoring region obtained in a historical time period, and the historical orthographic image data is used to represent the orthographic image data of the monitoring region obtained in the historical time period. The second obtaining unit is used to obtain current point cloud data and current orthographic image data. The registration unit is used to register the current point cloud data by using the historical orthographic image data, the historical point cloud data and the current orthographic image data to obtain registered current point cloud data.

[0012] According to another aspect of the embodiment of the present application, a computer readable storage medium is also provided, which comprises a stored program, wherein the program executes any one of the point cloud data registration methods.

[0013] According to another aspect of the embodiments of the present application, there is also provided a system for registering point cloud data, comprising one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs comprise a program for performing any of the methods for registering point cloud data.

[0014] In the embodiments of the present application, orthographic image data is first introduced into the point cloud data registration step, which improves the accuracy of the current point cloud data after registration, realizes high-precision and high-efficiency registration of multi-period point cloud data, and improves the accuracy of ground monitoring point site layout selection in combination with image data, thereby solving the problem of low registration accuracy of current point cloud data and historical point cloud data in the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and are used to interpret the present application together with the description of the illustrative embodiments of the present application and their description. The accompanying drawings do not constitute an undue limitation on the present application. In the drawings:

[0016] Figure 1 A flowchart of a method for registering point cloud data according to an embodiment of the present application is shown;

[0017] Figure 2 A schematic diagram of a device for registering point cloud data according to an embodiment of the present application is shown;

[0018] Figure 3 A flowchart of a registration scheme for point cloud data according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0019] It should be noted that the embodiments and features in the present application can be combined with each other without conflict. The technical solutions of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0020] In order to enable those skilled in the art to better understand the present application, the technical solutions of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0021] It should be noted that the terms "first", "second", and the like in the description and in the claims of the present application and above accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular sequential or chronological order. It should be understood that the data thus used can be interchanged, where appropriate, to describe the embodiments of the present application described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a list of steps or units is not necessarily limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to such processes, methods, products, or apparatus.

[0022] It should be understood that when an element (such as a layer, film, region, or substrate) is referred to as being "on" another element, it can be directly on the other element, or intervening elements can also be present. In addition, in the specification and claims, when an element is described as being "connected" to another element, it can be "directly connected" to the other element, or "connected" to the other element through a third element.

[0023] For ease of description, the following describes some nouns or terms related to the embodiments of the present application:

[0024] Orthophoto data: the professional term should be: digital orthophoto. Orthophotography refers to the photography of unmanned aerial vehicles, in which the principal axes of two adjacent image planes are parallel to each other and perpendicular to the photography baseline. The orthophotography is obtained by digital differential rectification and mosaicking of the image, and is cropped according to a certain sheet range to generate a digital orthophoto.

[0025] Digital orthophoto: a digital orthophoto set generated by digital differential rectification and mosaicking of aerospace images, and cropped according to a certain sheet range.

[0026] GNSS: Global Navigation Satellite System (Global Navigation Satellite System).

[0027] ICP algorithm: Iterative Closest Point, based on data registration method, using nearest point search method to solve a free-form surface based algorithm.

[0028] As described in the background, the three-dimensional laser scanning technology can intuitively reflect the displacement of the slope, but is easily affected by the environment, has a relatively limited monitoring range, and is subject to many factors such as manual intervention. The current point cloud data and the historical point cloud data are registered with low accuracy. In order to solve the problem of low registration accuracy of the current point cloud data and the historical point cloud data in the prior art, in a typical embodiment of the present application, a point cloud data registration method, device, computer readable storage medium and point cloud data registration system are provided.

[0029] According to an embodiment of the present application, a point cloud data registration method is provided.

[0030] Figure 1 is a flowchart of the point cloud data registration method according to an embodiment of the present application. As shown in Figure 1 , the method comprises the following steps:

[0031] Step S101, obtaining historical point cloud data and historical orthographic image data, wherein the historical point cloud data is used to represent the point cloud data of the monitoring area obtained in the historical time period, and the historical orthographic image data is used to represent the orthographic image data of the monitoring area obtained in the historical time period;

[0032] Step S102, obtaining current point cloud data and current orthographic image data;

[0033] Step S103, using the historical orthographic image data, the historical point cloud data and the current orthographic image data to perform registration processing on the current point cloud data, to obtain the registered current point cloud data.

[0034] In the above steps, the orthographic image data is first introduced into the point cloud data registration step, which improves the accuracy of the registered current point cloud data, realizes high-precision and high-efficiency registration of multi-period point cloud data, and improves the accuracy of ground monitoring point layout selection by combining with image data, thereby solving the problem of low registration accuracy of the current point cloud data and the historical point cloud data in the prior art.

[0035] In an embodiment of the present application, the historical orthographic image data, the historical point cloud data and the current orthographic image data are used to perform registration processing on the current point cloud data to obtain registered current point cloud data. The registration processing includes: performing matching processing on the historical orthographic image data and the current orthographic image data to obtain a matched current image feature point set and a matched historical image feature point set, the points in the matched current image feature point set correspond one-to-one to the points in the matched historical image feature point set; obtaining a current point cloud feature point set according to the matched current image feature point set and the current point cloud data; obtaining a historical point cloud feature point set according to the matched historical image feature point set and the historical point cloud data; determining a target translation and a target rotation matrix according to the historical point cloud feature point set and the current point cloud feature point set, wherein the target translation is used to represent a translation related to the current point cloud feature point set and the historical point cloud feature point set, and the target rotation matrix is used to represent a rotation matrix related to the current point cloud feature point set and the historical point cloud feature point set; and performing registration on the current point cloud data according to the current point cloud feature point set, the target rotation matrix and the target translation to obtain the registered current point cloud data.

[0036] Specifically, the historical point cloud data is rotated and translated according to the target rotation matrix and the target translation, so that the position of the historical point cloud data can be accurately matched, and the ICP algorithm is optimized.

[0037] In an embodiment of the present application, determining the target translation and the target rotation matrix according to the historical point cloud feature point set and the current point cloud feature point set includes: determining a plurality of initial nearest neighbor points, the initial nearest neighbor points being used to represent a point in the historical point cloud feature point set that is closest to a point in the current point cloud feature point set; determining a neighbor distance between the initial nearest neighbor point and a target point, the target point being a point in the current point cloud feature point set that is matched to the initial nearest neighbor point, the initial nearest neighbor point and the target point having the same values of a first coordinate axis and a second coordinate axis; in a case where the initial nearest neighbor point distance is less than a first distance threshold, taking the initial nearest neighbor point as a target nearest neighbor point; determining an average distance of the target nearest neighbor points, the average distance of the target nearest neighbor points being used to represent an average value of the neighbor distances of all the target nearest neighbor points; and determining the target translation and the target rotation matrix according to the average distance of the target nearest neighbor points, the historical point cloud feature point set and the current point cloud feature point set. The coordinate of the historical coordinate scale is transplanted to the average value as the coordinate origin, and the direction scale is unified.

[0038] In an embodiment of the present application, the current point cloud data is registered according to the current point cloud feature point set, the target rotation matrix and the target translation amount to obtain registered current point cloud data, including: registering the current point cloud data according to the current point cloud feature point set, the target rotation matrix and the target translation amount to obtain initial registered current point cloud data; determining a plurality of feature point distances, the feature point distance being used to represent the distance between a point in the initial registered current point cloud data and a corresponding feature point in the historical point cloud feature point set; determining an average distance of feature points according to all the feature point distances; in a case where the average distance of feature points is less than a second distance threshold, taking the initial registered current point cloud data as the registered current point cloud data; in a case where the average distance of feature points is greater than or equal to the second distance threshold, determining a plurality of initial nearest neighbor points again. The numerical iteration algorithm and the gradient algorithm are used to optimize the numerical value.

[0039] In an embodiment of the present application, the current point cloud feature point set is obtained according to the matched current image feature point set and the current point cloud data, including: obtaining a plurality of first coordinate values, wherein the first coordinate value is used to represent the coordinate value of a point in the matched current image feature point set in the direction of the first coordinate axis and the direction of the second coordinate axis; constructing a first cylindrical bounding box according to the first coordinate value and the current point cloud data, wherein in the first cylindrical bounding box, the first coordinate value is taken as the center of the first cylindrical bounding box, the threshold radius is taken as the radius of the first cylindrical bounding box, and the difference between the highest point and the lowest point of the third coordinate axis in the point corresponding to the first coordinate value in the current point cloud data is taken as the height of the first cylindrical bounding box; determining the point with the maximum third coordinate axis in the first cylindrical bounding box as the current point cloud feature point; determining the current point cloud feature point set, wherein the current point cloud feature point set includes all the current point cloud feature points.

[0040] In an embodiment of the present application, the historical point cloud feature point set is obtained according to the matched historical image feature point set and the historical point cloud data, including: obtaining a plurality of second coordinate values, wherein the second coordinate values are used to represent the coordinate values of a point in the matched historical image feature point set in the direction of the first coordinate axis and the direction of the second coordinate axis; constructing a second cylinder bounding box according to the second coordinate values and the historical point cloud data, wherein in the second cylinder bounding box, the second coordinate values are taken as the center of the second cylinder bounding box, the threshold radius is taken as the radius of the second cylinder bounding box, and the difference between the highest point and the lowest point of the third coordinate axis in the points corresponding to the second coordinate values in the historical point cloud data is taken as the height of the second cylinder bounding box; determining the point with the largest third coordinate axis in the second cylinder bounding box as a historical point cloud feature point; and determining the historical point cloud feature point set, wherein the historical point cloud feature point set includes all the historical point cloud feature points.

[0041] In an embodiment of the present application, after the current point cloud data is registered by using the historical orthographic image data, the historical point cloud data and the current orthographic image data, the method further includes: constructing a difference model according to the registered current point cloud data and the historical point cloud data, wherein the difference model is used to represent the positional difference relationship between the registered current point cloud data and the historical point cloud data; and determining the area where the open-pit coal mine slope is in displacement according to the difference model. Thus, the purpose of uninterrupted, high-precision, high-efficiency and low-cost space-ground integrated monitoring of the open-pit coal mine slope displacement is achieved.

[0042] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0043] The embodiment of the present application also provides a point cloud data registration device. It should be noted that the point cloud data registration device of the embodiment of the present application can be used to execute the point cloud data registration method provided by the embodiment of the present application. The point cloud data registration device provided by the embodiment of the present application is introduced as follows.

[0044] Figure 2 FIG. 1 is a schematic diagram of a point cloud data registration device according to an embodiment of the present application. As shown in FIG. 1, the point cloud data registration device includes a processor 1001, a memory 1002 and a communication interface 1003. Figure 2As shown, the device comprises a first acquisition unit 21, a second acquisition unit 22 and a registration unit 23, the first acquisition unit 21 is configured to acquire historical point cloud data and historical orthographic image data, wherein the historical point cloud data is used to represent point cloud data of a monitoring area acquired in a historical time period, and the historical orthographic image data is used to represent orthographic image data of the monitoring area acquired in the historical time period; the second acquisition unit 22 is configured to acquire current point cloud data and current orthographic image data; and the registration unit 23 is configured to perform registration processing on the current point cloud data by using the historical orthographic image data, the historical point cloud data and the current orthographic image data, to obtain registered current point cloud data.

[0045] In the device, the orthographic image data is first introduced into the point cloud data registration step, the accuracy of the registered current point cloud data is improved, multi-period point cloud data high-precision and high-efficiency registration is realized, and the accuracy of ground monitoring point position layout selection is improved in combination with image data, thereby solving the problem of low registration accuracy of current point cloud data and historical point cloud data in the prior art.

[0046] In an embodiment of the present application, the registration unit comprises a first processing module, a second processing module, a third processing module, a determination module and a registration module, the first processing module is configured to perform matching processing on the historical orthographic image data and the current orthographic image data to obtain a matched current image feature point set and a matched historical image feature point set, and the points in the matched current image feature point set correspond one-to-one to the points in the matched historical image feature point set; the second processing module is configured to obtain a current point cloud feature point set according to the matched current image feature point set and the current point cloud data; the third processing module is configured to obtain a historical point cloud feature point set according to the matched historical image feature point set and the historical point cloud data; the determination module is configured to determine a target translation and a target rotation matrix according to the historical point cloud feature point set and the current point cloud feature point set, wherein the target translation is used to represent a translation related to the current point cloud feature point set and the historical point cloud feature point set, and the target rotation matrix is used to represent a rotation matrix related to the current point cloud feature point set and the historical point cloud feature point set; and the registration module is configured to perform registration on the current point cloud data according to the current point cloud feature point set, the target rotation matrix and the target translation, to obtain registered current point cloud data.

[0047] In an embodiment of the present application, the determining module comprises a first determining submodule, a second determining submodule, a first processing submodule, a third determining submodule and a fourth determining submodule, the first determining submodule is configured to determine a plurality of initial nearest neighbor points, the initial nearest neighbor point is configured to represent a nearest point in the historical point cloud feature point set to a point in the current point cloud feature point set; the second determining submodule is configured to determine a distance between the initial nearest neighbor point and a target point as a neighbor distance, the target point is a point in the current point cloud feature point set matched with the initial nearest neighbor point, the initial nearest neighbor point and the target point have the same values of the first coordinate axis and the second coordinate axis; the first processing submodule is configured to, in a case that the initial nearest neighbor point distance is less than a first distance threshold, take the initial nearest neighbor point as a target nearest neighbor point; the third determining submodule is configured to determine an average distance of the target nearest neighbor point, the average distance of the target nearest neighbor point is configured to represent an average value of the neighbor distances of all the target nearest neighbor points; and the fourth determining submodule is configured to determine a target translation and a target rotation matrix according to the average distance of the target nearest neighbor point, the historical point cloud feature point set and the current point cloud feature point set.

[0048] In an embodiment of the present application, the registration module comprises a registration submodule, a fifth determining submodule, a sixth determining submodule, a second processing submodule and a seventh determining submodule, the registration submodule is configured to register the current point cloud data according to the current point cloud feature point set, the target rotation matrix and the target translation, to obtain initial registered current point cloud data; the fifth determining submodule is configured to determine a plurality of feature point distances, the feature point distance is configured to represent a distance between a point in the initial registered current point cloud data and a corresponding feature point in the historical point cloud feature point set; the sixth determining submodule is configured to determine an average distance of the feature points according to all the feature point distances; the second processing submodule is configured to, in a case that the average distance of the feature points is less than a second distance threshold, take the initial registered current point cloud data as the registered current point cloud data; and the seventh determining submodule is configured to, in a case that the average distance of the feature points is greater than or equal to the second distance threshold, determine a plurality of initial nearest neighbor points again.

[0049] In an embodiment of the present application, the second processing module comprises a first obtaining sub-module, a first constructing sub-module, an eighth determining sub-module and a ninth determining sub-module. The first obtaining sub-module is configured to obtain a plurality of first coordinate values, wherein the first coordinate values are used to represent the coordinate values of a point in the matched current image feature point set in the direction of the first coordinate axis and the direction of the second coordinate axis. The first constructing sub-module is configured to construct a first cylinder bounding box according to the first coordinate values and the current point cloud data, wherein in the first cylinder bounding box, the first coordinate values are taken as the center of the first cylinder bounding box, a threshold radius is taken as the radius of the first cylinder bounding box, and the difference between the highest point and the lowest point of the third coordinate axis in the point corresponding to the first coordinate values in the current point cloud data is taken as the height of the first cylinder bounding box. The eighth determining sub-module is configured to determine the point with the maximum value of the third coordinate axis in the first cylinder bounding box as the current point cloud feature point. The ninth determining sub-module is configured to determine the current point cloud feature point set, wherein the current point cloud feature point set comprises all the current point cloud feature points.

[0050] In an embodiment of the present application, the third processing module comprises a second obtaining sub-module, a second constructing sub-module, a tenth determining sub-module and an eleventh determining sub-module. The second obtaining sub-module is configured to obtain a plurality of second coordinate values, wherein the second coordinate values are used to represent the coordinate values of a point in the matched historical image feature point set in the direction of the first coordinate axis and the direction of the second coordinate axis. The second constructing sub-module is configured to construct a second cylinder bounding box according to the second coordinate values and the historical point cloud data, wherein in the second cylinder bounding box, the second coordinate values are taken as the center of the second cylinder bounding box, a threshold radius is taken as the radius of the second cylinder bounding box, and the difference between the highest point and the lowest point of the third coordinate axis in the point corresponding to the second coordinate values in the historical point cloud data is taken as the height of the second cylinder bounding box. The tenth determining sub-module is configured to determine the point with the maximum value of the third coordinate axis in the second cylinder bounding box as the historical point cloud feature point. The eleventh determining sub-module is configured to determine the historical point cloud feature point set, wherein the historical point cloud feature point set comprises all the historical point cloud feature points.

[0051] In an embodiment of the present application, the device further comprises a constructing unit and a determining unit, after the current point cloud data is subjected to the registration processing by using the historical orthographic image data, the historical point cloud data and the current orthographic image data, the constructing unit is configured to construct a differential model according to the registered current point cloud data and the historical point cloud data, the differential model is configured to represent a positional difference relationship between the registered current point cloud data and the historical point cloud data; and the determining unit is configured to determine a region where the open-pit coal mine slope is undergoing displacement according to the differential model.

[0052] The registration device for the point cloud data comprises a processor and a memory, the first obtaining unit, the second obtaining unit and the registration unit are all stored in the memory as program units, and the corresponding functions are realized by the processor executing the program units stored in the memory.

[0053] The processor comprises a core, and the core is configured to call the corresponding program units from the memory. One or more than one core can be set, and the problem of low registration accuracy of the current point cloud data and the historical point cloud data in the prior art can be solved by adjusting the core parameters.

[0054] The memory can comprise a non-permanent memory in a computer readable medium, a random access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory comprises at least one memory chip.

[0055] The embodiment of the present application provides a computer readable storage medium, which stores a program, and the program is executed by a processor to realize the registration method for the point cloud data.

[0056] The embodiment of the present application provides a processor, and the processor is configured to run a program, and the program is executed to perform the registration method for the point cloud data.

[0057] The embodiment of the present application provides a device, and the device comprises a processor, a memory and a program stored in the memory and capable of running on the processor, and the processor is configured to perform at least the following steps when the program is executed: obtaining historical point cloud data and historical orthographic image data, wherein the historical point cloud data is configured to represent point cloud data of a monitored region obtained in a historical time period, and the historical orthographic image data is configured to represent orthographic image data of the monitored region obtained in the historical time period; obtaining current point cloud data and current orthographic image data; and performing registration processing on the current point cloud data by using the historical orthographic image data, the historical point cloud data and the current orthographic image data, to obtain registered current point cloud data. The device in the present application can be a server, a PC, a PAD, a mobile phone and the like.

[0058] The application also provides a computer program product suitable for executing a program having at least the following method steps when executed on a data processing device: obtaining historical point cloud data and historical orthographic image data, wherein the historical point cloud data is used to represent point cloud data of a monitoring area obtained in a historical time period, and the historical orthographic image data is used to represent orthographic image data of the monitoring area obtained in the historical time period; obtaining current point cloud data and current orthographic image data; performing registration processing on the current point cloud data by using the historical orthographic image data, the historical point cloud data and the current orthographic image data, to obtain registered current point cloud data.

[0059] The application also provides a point cloud data registration system, which comprises one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs comprise a program for executing any of the above point cloud data registration methods. For the first time, orthographic image data is introduced into the point cloud data registration step, which improves the accuracy of the registered current point cloud data, realizes high-precision and high-efficiency registration of multi-period point cloud data, and improves the accuracy of ground monitoring point layout selection by combining with image data, thereby solving the problem of low registration accuracy of current point cloud data and historical point cloud data in the prior art.

[0060] In order for those skilled in the art to have a clearer understanding of the technical solutions of the application, the technical solutions and technical effects of the application will be described below in conjunction with specific embodiments.

[0061] Embodiments

[0062] The application also provides a point cloud data registration scheme, which comprises the following steps:

[0063] Step 1: presetting a monitoring area range, the monitoring area is positionally corresponding to a task target area on a map (the geometric center point of the monitoring area is located in the task target area range), and the monitoring area completely covers the task target area;

[0064] Specifically, the ground station generates a flight path with a relative altitude ≤ 300 meters according to the terrain and task requirements, the optimal relative altitude is below 280 meters, obtains relative altitude data and point density indicators, and sets laser frequency and line scanning speed, image overlap rate according to the relative altitude and point density indicators;

[0065] Step 2: setting multiple image control points in the monitoring area, obtaining the latitude and longitude coordinates and elevation of the image control points for later data accuracy inspection;

[0066] Step 3: Install the tripod and control base station on the known point, measure the slope height; integrate and assemble the unmanned aerial vehicle body and laser radar system, turn on the power, start data storage, and place the ground for five minutes. The aircraft takes off and flies in the first "8" shape. After entering the preset route, data collection begins;

[0067] Step 4: After the aircraft completes the survey area operation, the laser scanner is turned off. After completing the tail "8" shape flight, the aircraft lands. After landing, the ground is placed for five minutes, the data storage is turned off, the radar, inertial navigation, and camera related data are downloaded, the reference station related data is downloaded after the laser data is downloaded, and the whole system is turned off and powered off;

[0068] Step 5: Convert the collected raw data, including static data.txt, GPS data.gnss, IMU raw data.imu, and laser raw data.rxp, to IMU data.imr and laser data.sdc;

[0069] Step 6: Use IE software to solve and tightly couple GPS data.gnss, IMU data.imr, and static data.txt, complete POS data solving, and export high-precision differential flight route trajectory.txt in jolidar format. Open JoLiDAR software developed by Chengdu Longheng Research and Development, combine converted laser data.sdc and high-precision differential flight route trajectory.txt in jolidar format, select distance (distance from the aircraft), width (point cloud width), and angle filter (rotating angle) to generate effective point cloud.las file in the survey area. Then use the data cleaning function to clean up isolated noise points in the point cloud data, and quickly generate reference laser point cloud data P (discrete point cloud);

[0070] Step 7: Use Tiantong software to solve the image data collected by the camera to generate POS data. After air triangulation, dense point cloud matching, and texture mapping, the current orthographic image data of the monitoring area is generated;

[0071] Step 8: As shown in Figure 3 , obtain historical point cloud data and historical orthographic image data, wherein the historical point cloud data represents the point cloud data of the monitoring area obtained in the historical time period, and the historical orthographic image data represents the orthographic image data of the monitoring area obtained in the historical time period; obtain current point cloud data and current orthographic image data;

[0072] Step 9: Match the historical orthographic image data and the current orthographic image data to obtain a matched current image feature point set and a matched historical image feature point set. The points in the matched current image feature point set correspond one-to-one to the points in the matched historical image feature point set;

[0073] Specifically, the oFAST corner points in the orthographic image are extracted (if a pixel is different from a large number of pixels in its neighborhood, the pixel can be a corner point), first, Gaussian blur and down-sampling are performed on the image at different scales, and the FAST feature point detection is performed on each scale image, and the total number of extracted feature points is taken as the oFAST feature points of the image; the BRIEF descriptor (BRIEF is a binary coded descriptor for the detected feature points, which discards the traditional method of describing feature points by using regional gray histogram, greatly speeds up the speed of establishing feature descriptors, and greatly reduces the time of feature matching, and is a very fast and potential algorithm) is calculated according to the position of the above-mentioned oFAST corner point; the feature point descriptors of the two images are matched, and the matching point pairs are screened out;

[0074] Step 10: obtaining a plurality of first coordinate values, wherein the first coordinate values are used to represent the coordinate values of a point in the matched current image feature point set in the direction of the first coordinate axis and the direction of the second coordinate axis; according to the first coordinate values and the current point cloud data, a first cylindrical bounding box is constructed, wherein in the first cylindrical bounding box, the first coordinate values are taken as the center of the first cylindrical bounding box, the threshold radius is taken as the radius of the first cylindrical bounding box, and the difference between the highest point and the lowest point of the third coordinate axis in the points corresponding to the first coordinate values in the current point cloud data is taken as the height of the first cylindrical bounding box; the point with the maximum third coordinate axis in the first cylindrical bounding box is determined as the current point cloud feature point; the current point cloud feature point set is determined, wherein the current point cloud feature point set includes all the current point cloud feature points;

[0075] The current point cloud feature point set p i The formula is:

[0076]

[0077] Where (x, y, z) is the coordinate system, x ui is the X-axis coordinate corresponding to the i-th point in the current point cloud feature point set, y ui is the Y-axis coordinate corresponding to the i-th point in the current point cloud feature point set, r is the threshold radius, is the minimum value of the Z-axis coordinate corresponding to the i-th point in the current point cloud feature point set, is the maximum value of the Z-axis coordinate corresponding to the i-th point in the current point cloud feature point set; the specific threshold r and the point cloud density p obtained by the laser radar satisfy and r can be adjusted according to the point cloud data density;

[0078] obtaining a plurality of second coordinate values, wherein the second coordinate values are used to represent the coordinates of a point in the matched historical image feature point set in the direction of the first coordinate axis and the direction of the second coordinate axis; constructing a second cylindrical bounding box according to the second coordinate values and the historical point cloud data, wherein in the second cylindrical bounding box, the second coordinate values are taken as the center of the second cylindrical bounding box, the threshold radius is taken as the radius of the second cylindrical bounding box, and the difference between the highest point and the lowest point of the third coordinate axis in the points corresponding to the second coordinate values in the historical point cloud data is taken as the height of the second cylindrical bounding box; determining the point with the maximum third coordinate axis in the second cylindrical bounding box as a historical point cloud feature point; and determining the historical point cloud feature point set, wherein the historical point cloud feature point set includes all the historical point cloud feature points.

[0079] historical point cloud feature point set q i The formula is:

[0080]

[0081] wherein x vi is the X-axis coordinate corresponding to the i-th point in the historical point cloud feature point set, y vi is the Y-axis coordinate corresponding to the i-th point in the historical point cloud feature point set, and r is the threshold radius, is the minimum value of the Z-axis coordinate corresponding to the i-th point in the historical point cloud feature point set, is the maximum value of the Z-axis coordinate corresponding to the i-th point in the historical point cloud feature point set.

[0082] Step 11: determining a plurality of initial nearest neighbor points, wherein the initial nearest neighbor points are used to represent the nearest point in the historical point cloud feature point set to the current point cloud feature point set; determining the distance between the initial nearest neighbor point and a target point as a neighbor distance, wherein the target point is the point in the current point cloud feature point set matched with the initial nearest neighbor point, and the initial nearest neighbor point and the target point have the same values of the first coordinate axis and the second coordinate axis; in the case that the distance between the initial nearest neighbor point and the target point is less than a first distance threshold, taking the initial nearest neighbor point as a target nearest neighbor point; determining the average distance of the target nearest neighbor point, wherein the average distance of the target nearest neighbor point is used to represent the average value of the neighbor distances of all the target nearest neighbor points; and determining a target translation and a target rotation matrix according to the average distance of the target nearest neighbor point, the historical point cloud feature point set and the current point cloud feature point set.

[0083] wherein the rigid body transformation with the minimum average distance of the corresponding points (the corresponding points are the corresponding feature point sets) is calculated, and the centroid of the current point cloud feature point set is calculated and the centroid of the historical point cloud feature point set and converted to the centroid coordinate system, the centroid coordinate system: the origin of the rectangular coordinate system is always selected on the centroid of the point group, and the coordinate axis direction is always parallel to the coordinate axis of a certain fixed reference system (inertial system) ;

[0084]

[0085] H = U∑V T ;

[0086] R * = UV T ;

[0087] t * = p i -R * q i ;

[0088] wherein, is the previous point cloud feature point set, is the historical point cloud feature point set, H is the ICP solving process matrix, j is the jth feature point of the current feature point set, the value range of j is [1, the total number of feature points of the current feature point set], U is a unit orthogonal matrix containing current orthographic image data information, V is a unit orthogonal matrix containing historical orthographic image data information, R * is the target rotation matrix, and t * is the target translation;

[0089] Step 12: registering the current point cloud data according to the above current point cloud feature point set, the above target rotation matrix and the above target translation, to obtain initial registration of the current point cloud data; determining a plurality of feature point distances, the feature point distance being used to represent the distance between a point in the initial registration of the current point cloud data and a corresponding feature point in the historical point cloud feature point set; determining the average distance of the feature points according to all the feature point distances; in the case that the average distance of the feature points is less than a second distance threshold, taking the initial registration of the current point cloud data as the registration of the current point cloud data; in the case that the average distance of the feature points is greater than or equal to the second distance threshold, determining a plurality of initial nearest neighbor points again;

[0090]

[0091] wherein, f(R * ,t * ) is the target optimization function, and n is the total number of matching points.

[0092] Step 13: For the registered current point cloud data, using DEM (Digital Terrain Model) data generation function, automatically extracting ground points and constructing a triangular net, calculating the accurate historical DEM data and current DEM data, subtracting the current DEM data from the historical DEM data to generate a difference model, displaying the calculation results on the model in the form of a chromatogram, and the difference model has a natural display range and a small deformation display range;

[0093] Step 14: Based on the difference model operation results, the slope deformation range and size can be intuitively extracted, and these areas will be the key areas of attention required for early warning monitoring. According to the texture features of high-resolution image data, the surface features such as rear edge cracks are further interpreted, and the deformation and image change features are comprehensively analyzed. Crack meters or GNSS ground online monitoring equipment are arranged to realize 24-hour monitoring of the key areas. When a certain monitoring value is greater than the set safety threshold, the system automatically sends a warning information (for example, controls the alarm to alarm).

[0094] In the above embodiments of the present application, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0095] In several embodiments provided in the present application, it should be understood that the disclosed technical content can be implemented by other ways. Among them, the device embodiments described above are only schematic, for example, the division of the above units can be a logical function division, and actual implementation can have another division way, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, indirect coupling or communication connection between units or modules, which can be electrical or other forms.

[0096] The units described above as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed to multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0097] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or software functional unit.

[0098] The integrated unit described above, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the above-mentioned method of each embodiment of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0099] From the above description, it can be seen that the above-mentioned embodiments of the present application achieve the following technical effects:

[0100] 1) The point cloud data registration method of the present application first introduces orthographic image data into the point cloud data registration step, improves the accuracy of the registered current point cloud data, realizes high-precision and high-efficiency registration of multi-period point cloud data, and combines image data to improve the accuracy of ground monitoring point layout selection, thereby solving the problem of low registration accuracy of current point cloud data and historical point cloud data in the prior art.

[0101] 2) The point cloud data registration device of the present application first introduces orthographic image data into the point cloud data registration step, improves the accuracy of the registered current point cloud data, realizes high-precision and high-efficiency registration of multi-period point cloud data, and combines image data to improve the accuracy of ground monitoring point layout selection, thereby solving the problem of low registration accuracy of current point cloud data and historical point cloud data in the prior art.

[0102] 3) The point cloud data registration system of the present application first introduces orthographic image data into the point cloud data registration step, improves the accuracy of the registered current point cloud data, realizes high-precision and high-efficiency registration of multi-period point cloud data, and combines image data to improve the accuracy of ground monitoring point layout selection, thereby solving the problem of low registration accuracy of current point cloud data and historical point cloud data in the prior art.

[0103] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method of registering point cloud data, characterized by, The method comprises the following steps: acquiring historical point cloud data and historical orthographic image data, wherein the historical point cloud data is used to represent point cloud data of a monitoring area acquired in a historical time period, and the historical orthographic image data is used to represent orthographic image data of the monitoring area acquired in the historical time period; acquiring current point cloud data and current orthographic image data; performing registration processing on the current point cloud data by using the historical orthographic image data, the historical point cloud data and the current orthographic image data, to obtain registered current point cloud data; the registration processing on the current point cloud data by using the historical orthographic image data, the historical point cloud data and the current orthographic image data to obtain the registered current point cloud data comprises the following steps: performing matching processing on the historical orthographic image data and the current orthographic image data to obtain a matched current image feature point set and a matched historical image feature point set, wherein the points in the matched current image feature point set correspond to the points in the matched historical image feature point set one by one; obtaining a current point cloud feature point set according to the matched current image feature point set and the current point cloud data; obtaining a historical point cloud feature point set according to the matched historical image feature point set and the historical point cloud data; determining a target translation amount and a target rotation matrix according to the historical point cloud feature point set and the current point cloud feature point set, wherein the target translation amount is used to represent a translation amount related to the current point cloud feature point set and the historical point cloud feature point set, and the target rotation matrix is used to represent a rotation matrix related to the current point cloud feature point set and the historical point cloud feature point set; performing registration on the current point cloud data according to the current point cloud feature point set, the target rotation matrix and the target translation amount, to obtain registered current point cloud data; after the registration processing on the current point cloud data by using the historical orthographic image data, the historical point cloud data and the current orthographic image data to obtain the registered current point cloud data, the method further comprises the following steps: constructing a differential model according to the registered current point cloud data and the historical point cloud data, wherein the differential model is used to represent a positional difference relationship between the registered current point cloud data and the historical point cloud data; determining a region where displacement of an open-pit coal mine slope is occurring according to the differential model.

2. The method of claim 1, wherein, determining the target translation amount and the target rotation matrix according to the historical point cloud feature point set and the current point cloud feature point set comprises the following steps: determining a plurality of initial nearest neighbor points, wherein the initial nearest neighbor points are used to represent a nearest point in the historical point cloud feature point set to the current point cloud feature point set; determining a neighbor distance between the initial nearest neighbor point and a target point, wherein the target point is a point in the current point cloud feature point set matched with the initial nearest neighbor point, and the initial nearest neighbor point and the target point have the same values of a first coordinate axis and a second coordinate axis; in a case where the initial nearest neighbor point distance is less than a first distance threshold, taking the initial nearest neighbor point as a target nearest neighbor point; determining an average distance of target nearest neighbor points, the average distance of target nearest neighbor points being used to represent an average value of neighbor distances of all the target nearest neighbor points; determining a target translation and a target rotation matrix according to the average distance of target nearest neighbor points, the historical point cloud feature point set and the current point cloud feature point set.

3. The method of claim 1, wherein, registering the current point cloud data according to the current point cloud feature point set, the target rotation matrix and the target translation, to obtain registered current point cloud data, including: registering the current point cloud data according to the current point cloud feature point set, the target rotation matrix and the target translation, to obtain initial registered current point cloud data; determining a plurality of feature point distances, the feature point distance being used to represent a distance between a point in the initial registered current point cloud data and a corresponding feature point in the historical point cloud feature point set; determining an average distance of feature points according to all the feature point distances; in a case where the average distance of feature points is less than a second distance threshold, taking the initial registered current point cloud data as the registered current point cloud data; in a case where the average distance of feature points is greater than or equal to the second distance threshold, determining a plurality of initial nearest neighbor points again.

4. The method of claim 1, wherein, obtaining a current point cloud feature point set according to the matched current image feature point set and the current point cloud data, including: obtaining a plurality of first coordinate values, wherein the first coordinate value is used to represent a coordinate value of a point in the matched current image feature point set in the direction of the first coordinate axis and the direction of the second coordinate axis; constructing a first cylindrical bounding box according to the first coordinate value and the current point cloud data, wherein in the first cylindrical bounding box, the first coordinate value is taken as the center of the first cylindrical bounding box, a threshold radius is taken as the radius of the first cylindrical bounding box, and the difference between the highest point and the lowest point of the third coordinate axis in the point corresponding to the first coordinate value in the current point cloud data is taken as the height of the first cylindrical bounding box; determining the point with the maximum third coordinate axis in the first cylindrical bounding box as the current point cloud feature point; determining the current point cloud feature point set, wherein the current point cloud feature point set includes all the current point cloud feature points.

5. The method of claim 1, wherein, obtaining a historical point cloud feature point set according to the matched historical image feature point set and the historical point cloud data, including: obtaining a plurality of second coordinate values, wherein the second coordinate value is used to represent a coordinate value of a point in the matched historical image feature point set in the direction of the first coordinate axis and the direction of the second coordinate axis; constructing a second cylindrical bounding box according to the second coordinate value and the historical point cloud data, wherein in the second cylindrical bounding box, the second coordinate value is taken as the center of the second cylindrical bounding box, a threshold radius is taken as the radius of the second cylindrical bounding box, and the difference between the highest point and the lowest point of the third coordinate axis in the point corresponding to the second coordinate value in the historical point cloud data is taken as the height of the second cylindrical bounding box; The point in the third coordinate axis with the largest value in the second cylinder bounding box is determined as a historical point cloud feature point; A historical point cloud feature point set is determined, wherein the historical point cloud feature point set includes all the historical point cloud feature points.

6. An apparatus for registering point cloud data, characterized by Comprise: A first obtaining unit is configured to obtain historical point cloud data and historical orthographic image data, wherein the historical point cloud data is used to represent point cloud data of a monitoring area obtained in a historical time period, and the historical orthographic image data is used to represent orthographic image data of the monitoring area obtained in the historical time period; A second obtaining unit is configured to obtain current point cloud data and current orthographic image data; A registration unit is configured to perform registration processing on the current point cloud data by using the historical orthographic image data, the historical point cloud data and the current orthographic image data, to obtain registered current point cloud data; The registration unit comprises: A first processing module is configured to perform matching processing on the historical orthographic image data and the current orthographic image data, to obtain a matched current image feature point set and a matched historical image feature point set, wherein points in the matched current image feature point set correspond to points in the matched historical image feature point set one by one; A second processing module is configured to obtain a current point cloud feature point set according to the matched current image feature point set and the current point cloud data; A third processing module is configured to obtain a historical point cloud feature point set according to the matched historical image feature point set and the historical point cloud data; A determination module is configured to determine a target translation amount and a target rotation matrix according to the historical point cloud feature point set and the current point cloud feature point set, wherein the target translation amount is used to represent a translation amount related to the current point cloud feature point set and the historical point cloud feature point set, and the target rotation matrix is used to represent a rotation matrix related to the current point cloud feature point set and the historical point cloud feature point set; A registration module is configured to perform registration on the current point cloud data according to the current point cloud feature point set, the target rotation matrix and the target translation amount, to obtain registered current point cloud data; The device further comprises: After the registration processing on the current point cloud data by using the historical orthographic image data, the historical point cloud data and the current orthographic image data is performed to obtain registered current point cloud data, a construction unit is configured to construct a difference model according to the registered current point cloud data and the historical point cloud data, wherein the difference model is used to represent a positional difference relationship between the registered current point cloud data and the historical point cloud data; A determination unit is configured to determine a region in which displacement of an open-pit coal mine slope is occurring according to the difference model.

7. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises a stored program, wherein the program performs the point cloud data registration method in any one of claims 1 to 5.

8. A registration system of point cloud data, characterized by, Comprise: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including programs for performing the method of registering point cloud data according to any one of claims 1 to 5.

Images