Point cloud processing method, device, equipment and storage medium

Abstract

The present application discloses a point cloud processing method, device, equipment and storage medium, and belongs to the field of computer vision technology. The method includes: obtaining a first point cloud and a second point cloud of a target area; based on the posture transformation relationship between the first coordinate system of the first point cloud and the second coordinate system of the second point cloud, adding the first point cloud to the second point cloud to obtain an initial point cloud including multiple first coordinate points and multiple second coordinate points; determining a first connecting line between any first coordinate point and the phase center of a laser radar used to collect the first point cloud; determining coordinate points to be removed from the multiple second coordinate points included in the initial point cloud to obtain a target point cloud. In addition to adding the first point cloud to the second point cloud, the coordinate points to be removed from the initial point cloud will also be removed, the update range is comprehensive, and the accuracy of the updated target point cloud is high.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer vision technology, and in particular to a point cloud processing method, device, equipment and storage medium. Background Art

[0002] With the development of computer vision technology, point clouds are being used in more and more scenarios. For example, point clouds are used to build maps. In the process of building maps, the target area needs to be scanned to obtain point clouds. However, in the scanned target area, the content of some areas will change over time. In this case, it is necessary to rescan the areas with changed content to obtain new point clouds, and update the old point clouds collected previously based on the new point clouds through the corresponding point cloud processing methods. Summary of the invention

[0003] The embodiments of the present application provide a point cloud processing method, apparatus, device and storage medium, which can be used to update the old point cloud collected previously based on the new point cloud. The technical solution is as follows:

[0004] On the one hand, an embodiment of the present application provides a point cloud processing method, the method comprising:

[0005] Acquire a first point cloud and a second point cloud of a target area, wherein the acquisition time of the first point cloud is later than the acquisition time of the second point cloud;

[0006] Based on the posture transformation relationship between the first coordinate system of the first point cloud and the second coordinate system of the second point cloud, the first point cloud is added to the second point cloud to obtain an initial point cloud including a plurality of first coordinate points and a plurality of second coordinate points, wherein the first coordinate points are coordinates of the corresponding first point cloud in the second coordinate system, and the second coordinate points are coordinates of the corresponding second point cloud in the second coordinate system;

[0007] For any first coordinate point, determine a first connecting line between the any first coordinate point and a phase center of a laser radar used to collect the first point cloud;

[0008] A coordinate point to be removed from a plurality of second coordinate points included in the initial point cloud is determined to obtain a target point cloud, wherein the coordinate point to be removed is a second coordinate point whose distance from the first connecting line satisfies a removal condition.

[0009] In a possible implementation manner, determining the coordinate points to be removed from the plurality of second coordinate points included in the initial point cloud includes:

[0010] Acquire a reference point set corresponding to any one of the first coordinate points, where the reference point set corresponding to any one of the first coordinate points includes a plurality of reference points located on a first connecting line corresponding to any one of the first coordinate points;

[0011] Based on the existence of a reference point in the reference point set corresponding to the plurality of first coordinate points, the distance from any second coordinate point being less than a distance threshold, determining the any second coordinate point as a candidate coordinate point;

[0012] The distance between the candidate coordinate point and a first connecting line corresponding to the candidate coordinate point is determined, and based on the distance satisfying a removal condition, the candidate coordinate point is determined to be the coordinate point to be removed.

[0013] In a possible implementation manner, determining the coordinate points to be removed from the plurality of second coordinate points included in the initial point cloud includes:

[0014] Determining a curvature of a candidate coordinate point among the plurality of second coordinate points, wherein the candidate coordinate point is determined based on the first connecting line;

[0015] Based on the curvature of the candidate coordinate point being greater than a first threshold, calculating a first distance between the candidate coordinate point and a first connecting line corresponding to the candidate coordinate point; based on the first distance being less than a second threshold, determining that the first distance satisfies the removal condition, and determining that the candidate coordinate point is a coordinate point to be removed;

[0016] Alternatively, based on the curvature of the candidate coordinate point being less than a third threshold, a second distance between the plane where the candidate coordinate point is located and the first coordinate point corresponding to the candidate coordinate point is calculated; based on the second distance being greater than a fourth threshold, it is determined that the second distance satisfies the removal condition, and the candidate coordinate point is determined to be a coordinate point to be removed.

[0017] In a possible implementation, the determining, based on the existence of a reference point whose distance from any second coordinate point is less than a distance threshold in a reference point set corresponding to the plurality of first coordinate points, any second coordinate point as a candidate coordinate point includes:

[0018] Searching for a reference point closest to any second coordinate point from a plurality of reference points included in a reference point set corresponding to the plurality of first coordinate points as a target reference point;

[0019] Based on the fact that the distance between the any second coordinate point and the target reference point corresponding to the any second coordinate point is less than the distance threshold, the any second coordinate point is determined to be the candidate coordinate point.

[0020] In a possible implementation, the step of acquiring a first point cloud and a second point cloud of a target area includes:

[0021] Acquire a third point cloud and a fourth point cloud of the target area, wherein the acquisition time of the third point cloud is later than the acquisition time of the fourth point cloud;

[0022] The third point cloud is downsampled according to the voxel size to obtain the first point cloud with uniform distribution, and the fourth point cloud is downsampled according to the voxel size to obtain the second point cloud with uniform distribution.

[0023] In a possible implementation, based on the pose transformation relationship between the first coordinate system of the first point cloud and the second coordinate system of the second point cloud, before adding the first point cloud to the second point cloud, the method further includes:

[0024] For any first initial coordinate point of the first point cloud in the first coordinate system, searching for a second initial coordinate point corresponding to the any first initial coordinate point among the second initial coordinate points included in the second point cloud, and determining a posture transformation relationship between the first coordinate system and the second coordinate system based on a plurality of first initial coordinate points and a second initial coordinate point corresponding to each of the plurality of first initial coordinate points;

[0025] Alternatively, a posture transformation relationship between the first coordinate system and the second coordinate system is obtained from a posture acquisition device, and the posture acquisition device obtains the posture transformation relationship through measurement.

[0026] In a possible implementation manner, after obtaining the target point cloud, the method further includes:

[0027] Acquire a fifth point cloud of the target area, wherein a collection posture of the fifth point cloud is different from a collection posture of the first point cloud;

[0028] The target point cloud is updated based on the fifth point cloud.

[0029] In another aspect, a point cloud processing device is provided, the device comprising:

[0030] An acquisition module, used to acquire a first point cloud and a second point cloud of a target area, wherein the acquisition time of the first point cloud is later than the acquisition time of the second point cloud;

[0031] An adding module, configured to add the first point cloud to the second point cloud based on a posture transformation relationship between a first coordinate system of the first point cloud and a second coordinate system of the second point cloud, to obtain an initial point cloud including a plurality of first coordinate points and a plurality of second coordinate points, wherein the first coordinate points are coordinates of the corresponding first point cloud in the second coordinate system, and the second coordinate points are coordinates of the corresponding second point cloud in the second coordinate system;

[0032] A determination module, used for determining, for any first coordinate point, a first connecting line between the any first coordinate point and a phase center of a laser radar used to collect the first point cloud;

[0033] The removal module is used to determine a coordinate point to be removed from a plurality of second coordinate points included in the initial point cloud to obtain a target point cloud, wherein the coordinate point to be removed is a second coordinate point whose distance from the first connecting line satisfies a removal condition.

[0034] In a possible implementation, the removal module is used to obtain a reference point set corresponding to any first coordinate point, the reference point set corresponding to any first coordinate point including multiple reference points located on a first connecting line corresponding to any first coordinate point; based on the existence of a reference point in the reference point set corresponding to multiple first coordinate points whose distance from any second coordinate point is less than a distance threshold, determine the any second coordinate point as a candidate coordinate point; determine the distance between the candidate coordinate point and the first connecting line corresponding to the candidate coordinate point, and based on the distance satisfying a removal condition, determine the candidate coordinate point as the coordinate point to be removed.

[0035] In one possible implementation, the removal module is used to determine the curvature of a candidate coordinate point among the multiple second coordinate points, the candidate coordinate point being determined based on the first connecting line; based on the curvature of the candidate coordinate point being greater than a first threshold, calculating a first distance between the candidate coordinate point and the first connecting line corresponding to the candidate coordinate point; based on the first distance being less than a second threshold, determining that the first distance satisfies the removal condition, and determining that the candidate coordinate point is a coordinate point to be removed; or, based on the curvature of the candidate coordinate point being less than a third threshold, calculating a second distance between the plane where the candidate coordinate point is located and the first coordinate point corresponding to the candidate coordinate point; based on the second distance being greater than a fourth threshold, determining that the second distance satisfies the removal condition, and determining that the candidate coordinate point is a coordinate point to be removed.

[0036] In a possible implementation, the removal module is used to search for a reference point that is closest to any second coordinate point from a plurality of reference points included in a set of reference points corresponding to a plurality of first coordinate points, as a target reference point; and determine that any second coordinate point is the candidate coordinate point based on the distance between any second coordinate point and the target reference point corresponding to the any second coordinate point being less than the distance threshold.

[0037] In a possible implementation, the acquisition module is used to acquire a third point cloud and a fourth point cloud of the target area, and the acquisition time of the third point cloud is later than the acquisition time of the fourth point cloud; the third point cloud is downsampled according to the voxel size to obtain the first point cloud with uniform distribution, and the fourth point cloud is downsampled according to the voxel size to obtain the second point cloud with uniform distribution.

[0038] In a possible implementation, the acquisition module is also used to search for a second initial coordinate point corresponding to any first initial coordinate point of the first point cloud in the first coordinate system among the second initial coordinate points included in the second point cloud, and determine a posture transformation relationship between the first coordinate system and the second coordinate system based on multiple first initial coordinate points and second initial coordinate points corresponding to each of the multiple first initial coordinate points; or, obtain the posture transformation relationship between the first coordinate system and the second coordinate system from a posture acquisition device, and the posture acquisition device obtains the posture transformation relationship through measurement.

[0039] In a possible implementation, the device further includes: an updating module, configured to acquire a fifth point cloud of the target area, wherein a collection pose of the fifth point cloud is different from a collection pose of the first point cloud; and update the target point cloud based on the fifth point cloud.

[0040] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor so that the computer device implements any of the above-mentioned point cloud processing methods.

[0041] On the other hand, a computer-readable storage medium is also provided, wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor so that a computer implements any of the above-mentioned point cloud processing methods.

[0042] On the other hand, a computer program product or a computer program is also provided, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs any of the above-mentioned point cloud processing methods.

[0043] The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:

[0044] Since the first line between any first coordinate point and the phase center of the laser radar reflects the emission trajectory of any first coordinate point, the emission trajectory of the first point cloud is reconstructed in the second coordinate system based on the first line. Since the first point cloud is a coordinate point successfully collected, it will not be blocked during the collection process. The coordinate points to be removed that block the first point cloud are determined based on the first line, with high accuracy. By determining the coordinate points to be removed, in the process of updating the second point cloud based on the first point cloud, in addition to adding the first point cloud to the second point cloud, the coordinate points to be removed in the initial point cloud will also be removed, so that the update range is more comprehensive and the accuracy of the updated target point cloud is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0047] Figure 2 is a flow chart of a point cloud processing method provided in an embodiment of the present application;

[0048] Figure 3 It is a schematic diagram of the position relationship of a region provided in an embodiment of the present application;

[0049] Figure 4 This is a schematic diagram of a laser radar scanning process provided by an embodiment of the present application;

[0050] Figure 5 is a structural schematic diagram of a point cloud processing device provided in an embodiment of the present application;

[0051] Figure 6 It is a structural diagram of a server provided in an embodiment of the present application;

[0052] Figure 7 It is a structural schematic diagram of a point cloud processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0054] This application embodiment provides a point cloud processing method, please refer to Figure 1, which shows a schematic diagram of an implementation environment of the method provided in an embodiment of the present application. The implementation environment may include: a terminal 11 and a server 12.

[0055] Optionally, the method may be independently executed by the terminal 11 or the server 12, or may be interactively executed by the terminal 11 and the server 12. For example, the process of interactive execution between the terminal 11 and the server 12 is that the terminal 11 is installed with an application capable of acquiring the first point cloud and the second point cloud. After the application acquires the first point cloud and the second point cloud, the first point cloud and the second point cloud may be sent to the server 12. The server 12 applies the method provided in the embodiment of the present application to determine the coordinate points to be removed based on the first point cloud and the second point cloud. The server 12 obtains the target point cloud based on the coordinate points to be removed and stores the target point cloud. Optionally, the server 12 sends the target point cloud to the terminal 11, and the terminal 11 stores the target point cloud.

[0056] Optionally, the terminal 11 can be any electronic product that can interact with the user through one or more methods such as keyboard, touchpad, touch screen, remote control, voice interaction or handwriting device, such as PC (Personal Computer), mobile phone, smart phone, PDA (Personal Digital Assistant), wearable device, PPC (Pocket PC), tablet computer, smart car machine, smart TV, smart speaker, etc. The server 12 can be a server, a server cluster composed of multiple servers, or a cloud computing service center. The terminal 11 establishes a communication connection with the server 12 through a wired or wireless network.

[0057] Those skilled in the art should understand that the above-mentioned terminal 11 and server 12 are only examples, and other existing or future terminals or servers that are applicable to the present application should also be included in the protection scope of the present application and are included here by reference.

[0058] The present application embodiment provides a point cloud processing method, which can be based on the above Figure 1 In the implementation environment shown, the method can be executed independently by the terminal or the server, or can be implemented by the terminal and the server interacting with each other. Taking the method applied to the server as an example, the flowchart of the method is as follows Figure 2 As shown, it includes steps 201 to 204.

[0059] In step 201, a first point cloud and a second point cloud of a target area are acquired, and the acquisition time of the first point cloud is later than the acquisition time of the second point cloud.

[0060] In one possible implementation, the process of the server acquiring the first point cloud and the second point cloud includes: acquiring a third point cloud and a fourth point cloud of the target area, wherein the acquisition time of the third point cloud is later than the acquisition time of the fourth point cloud; downsampling the third point cloud according to the voxel size to obtain a first point cloud with a uniform distribution, and downsampling the fourth point cloud according to the voxel size to obtain a second point cloud with a uniform distribution.

[0061] Exemplarily, the third point cloud and the fourth point cloud are point cloud data obtained based on the laser radar scanning the target area. The first laser radar used to collect the third point cloud and the second laser radar used to collect the fourth point cloud can be the same laser radar or different laser radars. The posture of the first laser radar when collecting the third point cloud and the posture of the second laser radar when collecting the fourth point cloud can be the same or different, and the embodiments of the present application do not limit this. In addition, when the laser radar including the first laser radar and the second laser radar collects point cloud data, the collection principle is, for example, TOF (Time of flight) or other principles. The laser beam emitted by the laser radar during the collection process is not limited in this application, including but not limited to infrared light near the 950nm (Nanometer) band.

[0062] The embodiment of the present application does not limit the triggering method of collecting the third point cloud based on the first laser radar. Optionally, the collection of the third point cloud is triggered based on the need to periodically collect the point cloud of the target area. Taking the application scenario of high-precision map as an example, the information of the regional buildings provided by the high-precision map should reflect the current latest situation as much as possible. Therefore, it is necessary to perform periodic point cloud collection on the regional buildings included in the high-precision map according to the collection cycle, so as to update the high-precision map in time. Based on this, the maintenance object of the high-precision map will use the first laser radar to scan the target area based on the collection cycle to realize the collection of the third point cloud, and the collection cycle can be set based on experience, for example, the collection cycle is set to three days, one week or one month based on experience.

[0063] Optionally, the triggering method for collecting the third point cloud can also be to trigger the collection of the third point cloud based on a collection instruction. Still taking the application scenario of high-precision maps as an example, when the high-precision maps are put into use, error correction controls will be provided. When the user finds that some scenes in the high-precision maps are different from the reality during the use of the high-precision maps, the error correction controls can be triggered for reporting. The maintenance object receives the collection instruction based on the triggering of the error correction control, and goes to the location reported by the user according to the collection instruction to collect the point cloud, that is, goes to the target area to collect the point cloud and obtain the third point cloud. For the target area in the above embodiment, it can be any area that has a need to collect point clouds. The embodiment of the present application does not limit the content included in the target area, and it can include a variety of content such as buildings, roads, trees and street lights.

[0064] Based on the above examples, it can be understood that the acquisition time of the third point cloud refers to the time when the first laser radar scans the target area. After the first laser radar acquires the third point cloud, the server acquires the third point cloud based on the communication connection with the first laser radar. It should be noted that the acquisition time of the third point cloud acquired by the server may be the same as or different from the acquisition time of the third point cloud. Among them, the acquisition time of the third point cloud is the same as the acquisition time. For example, after the first laser radar scans the target area to obtain the third point cloud, the third point cloud is sent to the server based on the communication connection between the server and the first laser radar, so that the server updates the point cloud based on the third point cloud.

[0065] If the acquisition time of the third point cloud is different from the acquisition time, for example, due to the limited scanning range of the first laser radar. Therefore, the first laser radar needs to adjust multiple postures to scan the target area and obtain multi-frame point cloud data including the third point cloud. Although the first laser radar has completed the acquisition of the third point cloud, the first laser radar chooses to store the third point cloud in the storage space until the scanning of multiple postures of the target area is completed, and then the multi-frame point cloud data in the storage space is sent to the server. Alternatively, the number of frames of point cloud data stored in the storage space reaches the storage threshold set based on experience, and the multi-frame point cloud data including the third point cloud is sent to the server.

[0066] It should be noted that the above example is intended to distinguish the acquisition time and collection time of the third point cloud, rather than to limit the process of the server acquiring the third point cloud. The interaction process between the server and the first laser radar can be as shown in the above embodiment, or it can be other situations. The embodiment of the present application does not limit this.

[0067] Optionally, after acquiring the third point cloud, the server may acquire a fourth point cloud of the target area based on the third point cloud. The acquisition time of the fourth point cloud is earlier than the acquisition time of the third point cloud, and the acquisition time of the fourth point cloud refers to the time when the second laser radar scans the target area. It should be noted that since the fourth point cloud has two situations, single-frame point cloud data and multi-frame point cloud data, and the corresponding acquisition time is different for different situations, for ease of understanding, the two situations of the fourth point cloud are now explained.

[0068] In the first case, the fourth point cloud can be a single-frame point cloud data collected by the second laser radar based on a posture. For example, when obtaining the experimental effect of the point cloud processing method provided in the embodiment of the present application, the execution purpose of the server is to obtain the accuracy of the determined coordinate points to be removed, and there is no requirement for whether the updated target point cloud needs to be put into use. Therefore, the fourth point cloud can be a single-frame point cloud data that has not been processed. In this case, the collection time of the fourth point cloud is also the time when the second laser radar collects the single-frame point cloud data.

[0069] In the second case, the fourth point cloud may be point cloud data obtained after processing multi-frame point cloud data of the target area. Multi-frame point cloud processing may be to unify the multi-frame point cloud data into a world coordinate system to obtain a reference point cloud for representing the three-dimensional form of the target area. Multi-frame point cloud processing may be to update the reference point cloud based on a newly collected frame of point cloud data of the target area using the method provided in the embodiment of the present application after unifying the multi-frame point cloud into a world coordinate system to obtain the reference point cloud, that is, before updating the fourth point cloud based on the third point cloud, the reference point cloud is also updated based on other point clouds to obtain the fourth point cloud. In the two cases in the second case above, the acquisition time of the fourth point cloud is the latest time among the acquisition times of each frame of point cloud data in the multi-frame point cloud data.

[0070] However, no matter which of the above embodiments the fourth point cloud is, the acquisition time of the third point cloud is later than that of the fourth point cloud. Since the acquisition time of the third point cloud is later than that of the fourth point cloud, the target area represented by the third point cloud is closer to the current target area than the target area represented by the fourth point cloud. After acquiring the fourth point cloud based on the third point cloud, the server can update the fourth point cloud through the third point cloud, thereby improving the accuracy of the updated fourth point cloud.

[0071] Regarding the process of obtaining the fourth point cloud based on the third point cloud, for example, the target area is determined based on the third point cloud, and the fourth point cloud including the target area and not smaller than the target area is determined according to the range of the target area. Taking the application scenario as a high-precision map, the target area is area A as an example, and the high-precision map includes area A, area B and area C. The positional relationship between area A, area B and area C is as follows: Figure 3 As shown, region B refers to an elliptical region including region A. After determining that the target region is region A based on the third point cloud, the server determines that the point cloud corresponding to region B is the fourth point cloud according to the range of region A, and extracts the point cloud data corresponding to region B from the point cloud data used to construct the high-precision map as the fourth point cloud, thereby achieving the acquisition of the fourth point cloud.

[0072] Exemplarily, after acquiring the third point cloud and the fourth point cloud, the server may downsample the third point cloud and the fourth point cloud according to the voxel size, and obtain a uniformly distributed first point cloud by downsampling the third point cloud, and obtain a uniformly distributed second point cloud by downsampling the fourth point cloud. In addition, in addition to having the uniform distribution characteristics shown in the above embodiments, the downsampled first point cloud and the second point cloud also have a reduced amount of data relative to the unprocessed third point cloud and the fourth point cloud. Based on the first point cloud and the second point cloud with less data, the efficiency of the subsequent update of the second point cloud based on the first point cloud is improved. Among them, the voxel size used for downsampling processing can be set based on experience, for example, the voxel size is set to a cube with a side length of 2 cm.

[0073] Of course, the server can choose to obtain the third point cloud and the fourth point cloud as shown in the above embodiment, and then obtain the first point cloud and the second point cloud for subsequent acquisition of the target point cloud by downsampling the third point cloud and the fourth point cloud. The server can also choose not to downsample the acquired point cloud data, that is, to use the third point cloud as the first point cloud and the fourth point cloud as the second point cloud. However, no matter how the first point cloud and the second point cloud are obtained based on the third point cloud and the fourth point cloud, since the first point cloud is obtained based on the third point cloud, the laser radar used to collect the first point cloud is also the first laser radar for collecting the third point cloud, and the collection time of the first point cloud is also the collection time of the third point cloud. The laser radar used to collect the second point cloud is also the second laser radar for collecting the fourth point cloud, and the collection time of the second point cloud is also the collection time of the fourth point cloud. Since the collection time of the third point cloud is later than the collection time of the fourth point cloud, the collection time of the first point cloud is also later than the collection time of the second point cloud.

[0074] In step 202, based on the posture transformation relationship between the first coordinate system of the first point cloud and the second coordinate system of the second point cloud, the first point cloud is added to the second point cloud to obtain an initial point cloud including multiple first coordinate points and multiple second coordinate points, the first coordinate point is the coordinate of the corresponding first point cloud in the second coordinate system, and the second coordinate point is the coordinate of the corresponding second point cloud in the second coordinate system.

[0075] In one possible implementation, before adding the first point cloud to the second point cloud, the server also needs to obtain the posture transformation relationship between the first coordinate system and the second coordinate system, and the acquisition process includes: for any first initial coordinate point of the first point cloud in the first coordinate system, searching for a second initial coordinate point corresponding to any first initial coordinate point among the second initial coordinate points included in the second point cloud, and determining the posture transformation relationship between the first coordinate system and the second coordinate system based on multiple first initial coordinate points and the second initial coordinate points corresponding to each of the multiple first initial coordinate points.

[0076] The second initial coordinate point corresponding to the first initial coordinate point means that the reflection point indicated by the first initial coordinate point and the reflection point indicated by the second initial coordinate point are the same point. Figure 4 The schematic diagram of the laser radar scanning process is shown as an example. Figure 4 (1) is a schematic diagram of the process of the second laser radar scanning the target area to obtain the second point cloud. Figure 4 (2) is a schematic diagram of the process of obtaining the first point cloud by scanning the target area with the first laser radar. Figure 4 The reflection point indicated by the first initial coordinate point A shown in the figure and the reflection point indicated by the second initial coordinate point A are located at the same position of the street lamp, that is, the reflection point indicated by the first initial coordinate point A and the reflection point indicated by the second initial coordinate point A are the same point, and the second initial coordinate point corresponding to the first initial coordinate point A is the second initial coordinate point A.

[0077] By obtaining the first initial coordinate point and the second initial coordinate point corresponding to the first initial coordinate point, the coordinates of the same point in the first coordinate system and the second coordinate system are obtained. The posture transformation relationship between the first coordinate system and the second coordinate system is calculated based on the coordinates of the same point in different coordinate systems. Exemplarily, the posture transformation relationship between the first coordinate system and the second coordinate system refers to a posture transformation matrix including rotation parameters and translation parameters. Among them, the translation parameter is used to indicate the position relationship between the coordinate origin of the first coordinate system and the coordinate origin of the second coordinate system, and the rotation parameter is used to indicate the posture relationship between the first coordinate system and the second coordinate system.

[0078] Optionally, in addition to calculating the pose transformation relationship based on the method shown in the above embodiment, the server can also obtain the pose transformation relationship between the first coordinate system and the second coordinate system from a pose acquisition device, and the pose acquisition device obtains the pose transformation relationship through measurement. Among them, the pose acquisition device can be a device provided inside the first laser radar, or it can be other devices installed on the first laser radar, so that when the first laser radar collects the first point cloud, the pose of the first coordinate system is measured, and then the pose transformation relationship between the first coordinate system and the second coordinate system is determined based on the pose of the first coordinate system and the pose of the second coordinate system.

[0079] Exemplarily, the pose of a coordinate system including the first coordinate system and the second coordinate system refers to the position of the coordinate origin of the coordinate system in the world coordinate system and the orientation of the coordinate system. In addition, the pose of the second coordinate system can also be measured by a pose acquisition device, and the acquisition of the pose of the second pose coordinate system can be synchronously acquired with the second point cloud, or asynchronously acquired. Optionally, the second coordinate system can be a world coordinate system, or the second coordinate system can be different from the world coordinate system, which is not limited in the embodiments of the present application. The pose acquisition device is, for example, an IMU (Inertial Measurement Unit) or other device that can be used to measure pose.

[0080] In a possible implementation, after obtaining the posture transformation relationship between the first coordinate system and the second coordinate system, the server adds the first point cloud in the second point cloud based on the posture transformation relationship. Adding the first point cloud in the second point cloud refers to determining the coordinates of the first point cloud in the second coordinate system. Since the first point cloud is a set of multiple first initial coordinate points, the coordinates of the first point cloud in the second coordinate system are also the first coordinate points corresponding to the multiple first initial coordinate points in the second coordinate system. Optionally, for any first initial coordinate point, the process of determining its corresponding first coordinate point includes: performing matrix multiplication on the coordinates of any first initial coordinate point and the posture transformation relationship, and the product obtained by the multiplication is the first coordinate point corresponding to the any first initial coordinate point. By performing the above steps on each first initial coordinate point included in the first point cloud, the first coordinate point of the first point cloud in the second coordinate system can be determined, thereby realizing the addition of the first point cloud in the second point cloud and obtaining an initial point cloud including the first coordinate point and the second coordinate point.

[0081] Similar to the meaning of the first coordinate point, the second coordinate point represents the coordinate of the corresponding second point cloud in the second coordinate system. The second point cloud corresponding to the second coordinate point is also the second initial coordinate point corresponding to the second coordinate point. It should be noted that if both the first point cloud and the second point cloud are point cloud data corresponding to the target area, some of the first coordinate points will be the same as the second initial coordinate point. Figure 4 Taking the schematic diagram of the laser radar scanning process as an example, the street lamp involved in the first laser radar scanning target area also exists in the second laser radar scanning target area. Therefore, in the second coordinate system, the first coordinate point falling on the street lamp will coincide with the second initial coordinate point falling on the street lamp. In this case, after determining multiple first coordinate points, the server can filter out the coordinate points in the second initial coordinate points that coincide with the first coordinate points, and use the filtered second initial coordinate points as the second coordinate points.

[0082] Optionally, the server may also choose not to filter the repeated second initial coordinate points, that is, determine multiple second initial coordinate points as second coordinate points, and then remove these repeated second initial coordinate points through the subsequent step of determining the coordinate points to be removed. The principle of removing the repeated second initial coordinate points will be explained in the process of determining the coordinates to be removed in step 204, and will not be repeated here.

[0083] In step 203, for any first coordinate point, a first connecting line between any first coordinate point and the phase center of the laser radar used to collect the first point cloud is determined.

[0084] In a possible implementation, the laser radar used to collect the first point cloud, that is, the first laser radar in the above embodiment, has a phase center that refers to the position where the first laser radar emits a laser beam. Figure 4 , which shows the phase center of the first laser radar. However, it should be noted that since the first coordinate point is the coordinate of the first point cloud in the second coordinate system, the phase center of the first laser radar used to establish the connection also needs to be moved from the first coordinate system to the second coordinate system. That is, based on the posture transformation relationship between the first coordinate system and the second coordinate system, the phase center of the first laser radar in the second coordinate system is determined. The process of determining the phase center is similar to that of determining the first coordinate point, and can also be done by matrix multiplication, which will not be repeated here.

[0085] By determining the phase center of the first laser radar in the second coordinate system, a first line is established between any first coordinate point and the phase center. In this case, the first line is used to represent the emission trajectory of the laser beam of the first laser radar in the process of collecting the first coordinate point. Figure 4 For example, after adding the first point cloud to the second point cloud, the second initial coordinate point A is the first coordinate point of the first initial coordinate point A in the second coordinate system, and the position of the first laser radar in the second coordinate system is Figure 4 The laser radar shown in (1) in this case, Figure 4 The line connecting the laser radar and the second initial coordinate point A in (1) reflects the emission trajectory of the laser beam of the first laser radar when collecting the first initial coordinate point A. By establishing a first line connecting each first coordinate point included in the initial point cloud with the phase center of the first laser radar, the emission trajectory of the laser beam corresponding to each first coordinate point can be reconstructed in the second coordinate system.

[0086] In step 204, the coordinate points to be removed from the plurality of second coordinate points included in the initial point cloud are determined to obtain a target point cloud, where the coordinate points to be removed are second coordinate points whose distances from the first connecting line satisfy a removal condition.

[0087] Exemplarily, the coordinate point to be removed is a second coordinate point that blocks the first coordinate point. Figure 4 Taking as an example, the case where the second coordinate point blocks the first coordinate point is described. Figure 4 (2) shows the emission trajectory of the laser beam corresponding to the first initial coordinate point B. The emission trajectory of the laser beam corresponding to the first initial coordinate point B reconstructed in the second coordinate system can be seen in Figure 4 The first line connecting the first coordinate point B and the phase center of the first laser radar in (1). The first line corresponding to the first coordinate point B passes through the bushes, that is, in the second coordinate system, the laser beam emitted by the first laser radar according to the scanning angle corresponding to the first coordinate point B cannot be projected onto the first coordinate point B, but will be reflected back to the first laser radar by the second coordinate point B after being projected onto the second coordinate point B located in the bushes, that is, the second coordinate point B blocks the first coordinate point B. However, since the first initial coordinate point B corresponding to the first coordinate point B is a real coordinate point collected by the first laser radar scanning the target area, the first coordinate point B should not be blocked by the second coordinate point, that is, the second coordinate point B does not exist during the process of the first laser radar collecting the first point cloud, and is a coordinate point to be removed.

[0088] It should be noted that the above examples are intended to illustrate the principle of determining the coordinate point to be removed in the embodiment of the present application, and are not intended to limit the relationship between the coordinate point to be removed and the first connecting line. Figure 4 The second coordinate point B shown is located on the first connecting line, that is, the distance between the second coordinate point B and the first connecting line is 0. It may also be a second coordinate point whose distance between the second coordinate point B and the first connecting line is not 0 but satisfies the removal condition.

[0089] In addition, in the process of determining the coordinate point to be removed, the server may first determine the candidate coordinate point, that is, the second coordinate point that may block the first coordinate point. Exemplarily, a reference point set corresponding to any first coordinate point is obtained, and the reference point set corresponding to any first coordinate point includes multiple reference points located on the first connecting line corresponding to any first coordinate point; based on the existence of a reference point in the reference point set corresponding to multiple first coordinate points whose distance from any second coordinate point is less than a distance threshold, any second coordinate point is determined to be a candidate coordinate point; the distance between the candidate coordinate point and the first connecting line corresponding to the candidate coordinate point is determined, and based on the distance satisfying the removal condition, the candidate coordinate point is determined to be a coordinate point to be removed.

[0090] The embodiment of the present application does not limit the manner of dividing reference points on the first line corresponding to any first coordinate point. It may be based on experience to determine the interval threshold, with the first coordinate point or the phase center of the first laser radar as the starting point, and a reference point is determined on the first line at each interval threshold, thereby obtaining a reference point set including multiple reference points. Of course, it is also possible to determine the number of reference points based on experience, determine the distance value between the reference points according to the length of the first line corresponding to the first coordinate point and the number of reference points, and then determine multiple reference points on the first line with the phase center of the first coordinate point or the first laser radar as the starting point. Based on the above examples, it can be understood that the number of reference points corresponding to different first coordinate points may be the same or different, and the distances between reference points corresponding to different first coordinate points may be the same or different.

[0091] In a possible implementation, after determining the reference point set corresponding to each first coordinate point, a candidate coordinate point may be determined from a plurality of second coordinate points based on the reference point set corresponding to the plurality of first coordinate points. The determination process includes: searching for a reference point closest to any second coordinate point from a plurality of reference points included in the reference point set corresponding to the plurality of first coordinate points as a target reference point; and determining any second coordinate point as a candidate coordinate point based on the distance between any second coordinate point and the target reference point corresponding to any second coordinate point being less than a distance threshold.

[0092] Optionally, in the process of determining the target reference point, the search range of the server can be a reference point set corresponding to multiple first coordinate points, or a reference point set corresponding to multiple first coordinate points and multiple first coordinate points. Exemplarily, the server searches for the target reference point through a kd tree (k-dimensional tree). Among them, a kd tree is a data structure used for fast nearest neighbor and approximate nearest neighbor search in a multidimensional space, and a node in the kd tree represents a multidimensional coordinate, that is, the reference point involved in the above embodiment. Through the kd tree, the server can quickly determine the target reference point that is closest to the second coordinate point.

[0093] Exemplarily, the server may choose to use the kd tree data structure to search for the target reference point, or may use other data structures or other search methods. After finding the target reference point, the server may further calculate the distance between the second coordinate point and the target reference point. When the distance between any second coordinate point and the target reference point is less than the distance threshold, any second coordinate point is determined to be a candidate coordinate point. The distance threshold may be set based on experience and the implementation environment, such as the size of the area occupied by the target area. By determining the candidate coordinate point from a plurality of second coordinate points, the second coordinate point is screened, thereby reducing the amount of data of the second coordinate point whose distance from the first line needs to be determined later.

[0094] Optionally, after the server obtains the candidate coordinate points from the second coordinate points through screening, the server may determine the coordinate points to be removed based on the candidate coordinate points. Since the process of determining the coordinate points to be removed through the candidate coordinate points needs to be based on the distance between the candidate coordinate points and the first connecting line corresponding to the candidate coordinate points, and the method of determining the distance between the candidate coordinate points and the first connecting line corresponding to the candidate coordinate points is related to the curvature of the candidate coordinate points, the server needs to determine the curvature of the candidate coordinate points among the multiple second coordinate points.

[0095] In a possible implementation, the process of determining the curvature of the candidate coordinate point includes: taking the candidate coordinate point as the center of the circle, setting the reference radius based on experience, determining the reference surface corresponding to the candidate coordinate point according to the center of the circle and the reference radius, calculating the curvature of the reference surface based on multiple second coordinate points included in the reference surface, and determining the curvature of the reference surface as the curvature of the candidate coordinate point. For example, based on experience, the reference radius is set to 10 cm to obtain a reference surface, determine multiple second coordinate points located in the reference surface, obtain multiple nearest neighbor points, and obtain the coefficients in the local parabola fitting formula by fitting the multiple second coordinate points included in the reference surface through the least squares method to obtain the curvature function, and calculate the curvature of the reference surface according to the curvature function. Among them, the curvature of the reference surface is used to indicate the degree of curvature of the reference surface. The greater the curvature of the reference surface, the greater its curvature. According to the curvature size of the candidate coordinate point, it is determined whether the distance between the candidate coordinate point and the first connecting line corresponding to the candidate coordinate point meets the removal condition, and then determine whether the candidate coordinate point is a coordinate to be removed by including but not limited to the following two methods.

[0096] Determination method 1: based on the curvature of the candidate coordinate point being greater than the first threshold, calculate the first distance between the candidate coordinate point and the first connecting line corresponding to the candidate coordinate point; based on the first distance being less than the second threshold, determine that the first distance satisfies the removal condition, and determine that the candidate coordinate point is the coordinate point to be removed.

[0097] Optionally, the curvature of the candidate coordinate point is greater than the first threshold, that is, the reference plane of the candidate coordinate point is a non-plane. In this case, the first distance between the candidate coordinate point and the first connecting line corresponding to the candidate coordinate point is calculated, that is, the vertical distance from the point to the line. The first connecting line corresponding to the candidate coordinate point refers to the first connecting line where the target reference point corresponding to the candidate coordinate point is located. By comparing the magnitude relationship between the first distance and the second threshold, when the first distance is less than the second threshold, it is determined that the candidate coordinate point corresponding to the first distance is a coordinate point to be removed that will block the first coordinate point.

[0098] For example, the second threshold value can be set based on experience. By setting the second threshold value, the aperture radius of the laser beam is also taken into account when determining the coordinate points to be removed, so as to avoid removing only the second coordinate points located on the first connecting line while ignoring the second coordinate points close to the first connecting line but blocking the laser beam, so that the determined coordinate points to be removed are more comprehensive and more accurate.

[0099] Determination method two: based on the curvature of the candidate coordinate point being less than the third threshold, calculate the second distance between the plane where the candidate coordinate point is located and the first coordinate point corresponding to the candidate coordinate point; based on the second distance being greater than the fourth threshold, determine that the second distance meets the removal condition, and determine that the candidate coordinate point is the coordinate point to be removed.

[0100] Optionally, the curvature of the candidate coordinate point is less than a third threshold, that is, the reference plane of the candidate coordinate point is a plane. The third threshold is an arbitrary value set based on experience. The third threshold may be the same as the first threshold or different from the first threshold. The embodiment of the present application does not limit this. In addition, the plane where the candidate coordinate point is located refers to the reference plane of the candidate coordinate point, and the first coordinate point corresponding to the candidate coordinate point refers to the first coordinate point connected to the first connecting line where the target reference point corresponding to the candidate coordinate point is located. Figure 4 Taking the second coordinate point B as the candidate coordinate point as an example, the first coordinate point corresponding to the second coordinate point B is also the first coordinate point B.

[0101] After calculating the second distance between the reference plane of the candidate coordinate point and the first coordinate point corresponding to the candidate coordinate point, the server compares the magnitude relationship between the second distance and the fourth threshold value. When the second distance is greater than the fourth distance, it is determined that the second distance meets the condition to be removed, that is, the candidate coordinate point is the coordinate point to be removed. The fourth threshold value is a value set based on experience, and the fourth threshold value may be the same as or different from the second threshold value, which is not limited in the embodiments of the present application.

[0102] In a possible implementation, in the process of determining the second coordinate point based on the second initial coordinate point shown in step 202, the second initial coordinate point that coincides with the first coordinate point is not screened out, and the second coordinate point that coincides with the first coordinate point will also be determined as a coordinate point to be removed in the process of determining the coordinate point to be removed. The coordinate point in the second coordinate point that coincides with the first coordinate point is called a coincident coordinate point. Since the coincident coordinate point coincides with the first coordinate point, for any coincident coordinate point, the target reference point found is also the first coordinate point corresponding to the coincident coordinate point, and its distance is 0, which is less than the distance threshold, and it will be determined as a candidate coordinate point. Since the above-mentioned coincident coordinate point coincides with the first coordinate point, it is located on the first connecting line corresponding to the target reference point. Therefore, the coincident coordinate point and the first connecting line corresponding to the coincident coordinate point meet the removal condition and will be determined as a coordinate point to be removed.

[0103] Through the above steps, the server can determine the coordinate points to be removed in the second coordinate points that block the first coordinate points, and remove the coordinate points to be removed from the initial point cloud to obtain a target point cloud that does not include the coordinate points to be removed. By removing the coordinate points to be removed, in the process of updating the second point cloud based on the first point cloud, in addition to adding the originally missing content to the second point cloud, the redundant content in the second point cloud will also be removed. The redundant content refers to the content that does not exist in the current target area. For example, when collecting the second point cloud, the target area includes street lights and bushes, and when collecting the first point cloud, a new building A is built in the target area. The first point cloud includes building A, street lights and bushes. By adding the second point cloud to the first point cloud, the point cloud data of building A is added, that is, the originally missing content is added. Since the bushes have fallen leaves during the collection of the first point cloud, the leaves of the bushes included in the second point cloud are redundant content. Through the method provided in the embodiment of the present application, the point cloud data of the leaves on the bushes can be determined as the coordinate points to be removed, and these coordinate points to be removed are removed to ensure that the updated target point cloud is closer to the current situation of the target area and has a high accuracy rate.

[0104] In a possible implementation, since the laser radar has a limited scanning range, when collecting point cloud data of the target area through the laser radar, the collected point cloud data is multi-frame point cloud data including the first point cloud. In this case, after the server updates the second point cloud based on the first point cloud to obtain the target point cloud, it also obtains the fifth point cloud of the target area, and the collection pose of the fifth point cloud is different from the collection pose of the first point cloud; and updates the target point cloud based on the fifth point cloud.

[0105] It should be noted that the difference between the fifth point cloud and the first point cloud lies in the acquisition posture of the laser radar, but the acquisition time of the fifth point cloud and the first point cloud can be the same or different. In addition, the laser radar used to collect the fifth point cloud and the first laser radar used to collect the first point cloud can be the same laser radar or different laser radars, which is not limited in the embodiment of the present application. The process of updating the target point cloud based on the fifth point cloud is similar to the process of updating the second point cloud based on the first point cloud in steps 201-204, and will not be repeated here.

[0106] In summary, the point cloud processing method provided by the embodiment of the present application, since the first line between any first coordinate point and the phase center of the laser radar reflects the emission trajectory of any first coordinate point, the emission trajectory of the first point cloud is reconstructed in the second coordinate system based on the first line. Because the first point cloud is a collected coordinate point, it will not be blocked during the collection process. The coordinate points to be removed that block the first point cloud are determined according to the first line, and the accuracy is high. In addition, in the process of determining the coordinate points to be removed from the candidate coordinate points, different determination methods are selected based on the curvature of the candidate coordinate points, which is highly flexible. By determining the coordinate points to be removed, in the process of updating the second point cloud based on the first point cloud, in addition to adding the first point cloud to the second point cloud, the coordinate points to be removed in the initial point cloud will also be removed, and the update range is more comprehensive, and the accuracy of the updated target point cloud is higher.

[0107] See also Figure 5 , the embodiment of the present application provides a point cloud processing device, the device comprising: an acquisition module 501, an adding module 502, a determination module 503 and a removal module 504;

[0108] An acquisition module 501 is used to acquire a first point cloud and a second point cloud of a target area, where the acquisition time of the first point cloud is later than the acquisition time of the second point cloud;

[0109] An adding module 502 is used to add the first point cloud to the second point cloud based on the posture transformation relationship between the first coordinate system of the first point cloud and the second coordinate system of the second point cloud, so as to obtain an initial point cloud including a plurality of first coordinate points and a plurality of second coordinate points, wherein the first coordinate point is a coordinate of the corresponding first point cloud in the second coordinate system, and the second coordinate point is a coordinate of the corresponding second point cloud in the second coordinate system;

[0110] A determination module 503 is used to determine, for any first coordinate point, a first connecting line between any first coordinate point and a phase center of a laser radar used to collect a first point cloud;

[0111] The removal module 504 is used to determine the coordinate point to be removed from the multiple second coordinate points included in the initial point cloud to obtain the target point cloud. The coordinate point to be removed is the second coordinate point whose distance from the first connecting line meets the removal condition.

[0112] Optionally, the removal module 504 is used to obtain a reference point set corresponding to any first coordinate point, the reference point set corresponding to any first coordinate point including multiple reference points located on a first connecting line corresponding to any first coordinate point; based on the existence of a reference point in the reference point set corresponding to multiple first coordinate points whose distance from any second coordinate point is less than a distance threshold, determine any second coordinate point as a candidate coordinate point; determine the distance between the candidate coordinate point and the first connecting line corresponding to the candidate coordinate point, and determine the candidate coordinate point as a coordinate point to be removed based on the distance satisfying the removal condition.

[0113] Optionally, the removal module 504 is used to determine the curvature of a candidate coordinate point among multiple second coordinate points, where the candidate coordinate point is determined based on the first connecting line; based on the curvature of the candidate coordinate point being greater than a first threshold, calculate the first distance between the candidate coordinate point and the first connecting line corresponding to the candidate coordinate point; based on the first distance being less than the second threshold, determine that the first distance satisfies the removal condition, and determine that the candidate coordinate point is the coordinate point to be removed; or, based on the curvature of the candidate coordinate point being less than a third threshold, calculate the second distance between the plane where the candidate coordinate point is located and the first coordinate point corresponding to the candidate coordinate point; based on the second distance being greater than a fourth threshold, determine that the second distance satisfies the removal condition, and determine that the candidate coordinate point is the coordinate point to be removed.

[0114] Optionally, the removal module 504 is used to search for a reference point that is closest to any second coordinate point from a plurality of reference points included in a set of reference points corresponding to a plurality of first coordinate points as a target reference point; and determine that any second coordinate point is a candidate coordinate point based on the distance between any second coordinate point and the target reference point corresponding to any second coordinate point being less than a distance threshold.

[0115] Optionally, the acquisition module 501 is used to acquire a third point cloud and a fourth point cloud of the target area, and the acquisition time of the third point cloud is later than the acquisition time of the fourth point cloud; the third point cloud is downsampled according to the voxel size to obtain a first point cloud with a uniform distribution, and the fourth point cloud is downsampled according to the voxel size to obtain a second point cloud with a uniform distribution.

[0116] Optionally, the acquisition module 501 is also used to search for a second initial coordinate point corresponding to any first initial coordinate point of the first point cloud in the first coordinate system among the second initial coordinate points included in the second point cloud, and determine the pose transformation relationship between the first coordinate system and the second coordinate system based on multiple first initial coordinate points and the second initial coordinate points corresponding to each of the multiple first initial coordinate points; or, obtain the pose transformation relationship between the first coordinate system and the second coordinate system from a pose acquisition device, and the pose acquisition device obtains the pose transformation relationship through measurement.

[0117] Optionally, the device also includes: an updating module, used to obtain a fifth point cloud of the target area, the acquisition posture of the fifth point cloud is different from the acquisition posture of the first point cloud; and update the target point cloud based on the fifth point cloud.

[0118] Since the first connecting line between any first coordinate point and the phase center of the laser radar reflects the emission trajectory of any first coordinate point, the device reconstructs the emission trajectory of the first point cloud in the second coordinate system based on the first connecting line. Since the first point cloud is a coordinate point successfully collected, it will not be blocked during the collection process. The coordinate points to be removed that block the first point cloud are determined based on the first connecting line with high accuracy. By determining the coordinate points to be removed, in the process of updating the second point cloud based on the first point cloud, in addition to adding the first point cloud to the second point cloud, the coordinate points to be removed in the initial point cloud will also be removed, so the update range is more comprehensive and the accuracy of the updated target point cloud is higher.

[0119] It should be noted that the device provided in the above embodiment only uses the division of the above functional modules as an example to implement its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and their specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0120] Figure 6 It is a structural diagram of a server provided in an embodiment of the present application. The server may have relatively large differences due to different configurations or performances, and may include one or more processors 601 and one or more memories 602, wherein the one or more memories 602 store at least one computer program, and the at least one computer program is loaded and executed by the one or more processors 601, so that the server implements the point cloud processing method provided by the above-mentioned various method embodiments. The processor is, for example, a CPU (Central Processing Unit). Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server may also include other components for implementing device functions, which will not be repeated here.

[0121] Figure 7: is a schematic diagram of the structure of a point cloud processing device provided in an embodiment of the present application. The device may be a terminal, such as a smart phone, a tablet computer, an MP3 (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Compression Standard Audio Layer 3) player, an MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) player, a laptop computer or a desktop computer. The terminal may also be referred to as a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0122] Typically, the terminal includes: a processor 701 and a memory 702 .

[0123] The processor 701 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 701 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 701 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 701 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 701 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0124] The memory 702 may include one or more computer-readable storage media, which may be non-transitory. The memory 702 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 702 is used to store at least one instruction, which is used to be executed by the processor 701 so that the terminal implements the point cloud processing method provided in the method embodiment of the present application.

[0125] In some embodiments, the terminal may further optionally include: a peripheral device interface 703 and at least one peripheral device. The processor 701, the memory 702 and the peripheral device interface 703 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 703 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 704, a display screen 705, a camera assembly 706, an audio circuit 707, a positioning assembly 708 and a power supply 709.

[0126] The peripheral device interface 703 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 701 and the memory 702. In some embodiments, the processor 701, the memory 702, and the peripheral device interface 703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 701, the memory 702, and the peripheral device interface 703 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0127] The radio frequency circuit 704 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 704 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 704 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 704 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 704 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 704 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0128] The display screen 705 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 705 is a touch display screen, the display screen 705 also has the ability to collect touch signals on the surface or above the surface of the display screen 705. The touch signal can be input to the processor 701 as a control signal for processing. At this time, the display screen 705 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 705 can be one, set on the front panel of the terminal; in other embodiments, the display screen 705 can be at least two, respectively set on different surfaces of the terminal or in a folding design; in other embodiments, the display screen 705 can be a flexible display screen, set on a curved surface or a folding surface of the terminal. Even, the display screen 705 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 705 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc.

[0129] The camera assembly 706 is used to capture images or videos. Optionally, the camera assembly 706 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 706 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0130] The audio circuit 707 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 701 for processing, or input them into the radio frequency circuit 704 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 701 or the radio frequency circuit 704 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 707 may also include a headphone jack.

[0131] Positioning component 708 is used to locate the current geographical location of the terminal to implement navigation or LBS (Location Based Service). Positioning component 708 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system, Russia's Grenas system or the European Union's Galileo system.

[0132] The power supply 709 is used to power various components in the terminal. The power supply 709 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 709 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0133] In some embodiments, the terminal further includes one or more sensors 710 , including but not limited to: an acceleration sensor 711 , a gyroscope sensor 712 , a pressure sensor 713 , a fingerprint sensor 714 , an optical sensor 715 , and a proximity sensor 716 .

[0134] The acceleration sensor 711 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal. For example, the acceleration sensor 711 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 701 can control the display screen 705 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 711. The acceleration sensor 711 can also be used for collecting game or user motion data.

[0135] The gyroscope sensor 712 can detect the body direction and rotation angle of the terminal, and the gyroscope sensor 712 can cooperate with the acceleration sensor 711 to collect the user's 3D actions on the terminal. The processor 701 can implement the following functions based on the data collected by the gyroscope sensor 712: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0136] The pressure sensor 713 can be set in the side frame of the terminal and / or the lower layer of the display screen 705. When the pressure sensor 713 is set in the side frame of the terminal, the user's holding signal of the terminal can be detected, and the processor 701 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 713. When the pressure sensor 713 is set in the lower layer of the display screen 705, the processor 701 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 705. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0137] The fingerprint sensor 714 is used to collect the user's fingerprint, and the processor 701 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 714, or the fingerprint sensor 714 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 701 authorizes the user to perform relevant sensitive operations, which include unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 714 can be set on the front, back or side of the terminal. When a physical button or a manufacturer logo is set on the terminal, the fingerprint sensor 714 can be integrated with the physical button or the manufacturer logo.

[0138] The optical sensor 715 is used to collect the ambient light intensity. In one embodiment, the processor 701 can control the display brightness of the display screen 705 according to the ambient light intensity collected by the optical sensor 715. Specifically, when the ambient light intensity is high, the display brightness of the display screen 705 is increased; when the ambient light intensity is low, the display brightness of the display screen 705 is reduced. In another embodiment, the processor 701 can also dynamically adjust the shooting parameters of the camera assembly 706 according to the ambient light intensity collected by the optical sensor 715.

[0139] The proximity sensor 716, also known as a distance sensor, is usually arranged on the front panel of the terminal. The proximity sensor 716 is used to collect the distance between the user and the front of the terminal. In one embodiment, when the proximity sensor 716 detects that the distance between the user and the front of the terminal is gradually decreasing, the processor 701 controls the display screen 705 to switch from the screen-on state to the screen-off state; when the proximity sensor 716 detects that the distance between the user and the front of the terminal is gradually increasing, the processor 701 controls the display screen 705 to switch from the screen-off state to the screen-on state.

[0140] Those skilled in the art will understand that Figure 7 The structure shown in the figure does not constitute a limitation on the point cloud processing device, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0141] In an exemplary embodiment, a computer device is also provided, the computer device comprising a processor and a memory, wherein at least one computer program is stored in the memory. The at least one computer program is loaded and executed by one or more processors, so that the computer device implements any of the above-mentioned point cloud processing methods.

[0142] In an exemplary embodiment, a computer-readable storage medium is also provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor of a computer device to enable the computer to implement any of the above-mentioned point cloud processing methods.

[0143] In one possible implementation, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0144] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs any of the above-mentioned point cloud processing methods.

[0145] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions. For example, the first point cloud and the second point cloud involved in this application are both obtained with full authorization.

[0146] It should be understood that the "plurality" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0147] The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A point cloud processing method, characterized in that: The method comprises: Acquire a first point cloud and a second point cloud of a target area, wherein the acquisition time of the first point cloud is later than the acquisition time of the second point cloud; Based on the posture transformation relationship between the first coordinate system of the first point cloud and the second coordinate system of the second point cloud, the first point cloud is added to the second point cloud to obtain an initial point cloud including a plurality of first coordinate points and a plurality of second coordinate points, wherein the first coordinate point is the coordinate of the corresponding first point cloud in the second coordinate system, and the second coordinate point is the coordinate of the corresponding second point cloud in the second coordinate system; for any first coordinate point, a first connecting line between the any first coordinate point and the phase center of the laser radar used to collect the first point cloud is determined, and the phase center of the first laser radar used to establish the connecting line also needs to be moved from the first coordinate system to the second coordinate system accordingly; Determine a coordinate point to be removed from a plurality of second coordinate points included in the initial point cloud to obtain a target point cloud, wherein the coordinate point to be removed is a second coordinate point whose distance from the first connecting line satisfies a removal condition; The step of determining the coordinate points to be removed from the plurality of second coordinate points included in the initial point cloud comprises: Acquire a reference point set corresponding to any one of the first coordinate points, where the reference point set corresponding to any one of the first coordinate points includes a plurality of reference points located on a first connecting line corresponding to any one of the first coordinate points; Based on the existence of a reference point in the reference point set corresponding to the plurality of first coordinate points, the distance from any second coordinate point being less than a distance threshold, determining the any second coordinate point as a candidate coordinate point; Determine a distance between the candidate coordinate point and a first connecting line corresponding to the candidate coordinate point, and determine the candidate coordinate point as the coordinate point to be removed based on the distance satisfying a removal condition; Or the determining of the coordinate points to be removed from the plurality of second coordinate points included in the initial point cloud comprises: Determining a curvature of a candidate coordinate point among the plurality of second coordinate points, wherein the candidate coordinate point is determined based on the first connecting line; Based on the curvature of the candidate coordinate point being greater than a first threshold, calculating a first distance between the candidate coordinate point and a first connecting line corresponding to the candidate coordinate point; based on the first distance being less than a second threshold, determining that the first distance satisfies the removal condition, and determining that the candidate coordinate point is a coordinate point to be removed; Alternatively, based on the curvature of the candidate coordinate point being less than a third threshold, a second distance between the plane where the candidate coordinate point is located and the first coordinate point corresponding to the candidate coordinate point is calculated; based on the second distance being greater than a fourth threshold, it is determined that the second distance satisfies the removal condition, and the candidate coordinate point is determined to be a coordinate point to be removed.

2. The method according to claim 1, characterized in that The determining that any second coordinate point is a candidate coordinate point comprises: Searching for a reference point closest to any second coordinate point from a plurality of reference points included in a reference point set corresponding to the plurality of first coordinate points as a target reference point; Based on the fact that the distance between the any second coordinate point and the target reference point corresponding to the any second coordinate point is less than the distance threshold, the any second coordinate point is determined to be the candidate coordinate point.

3. The method according to claim 1 or 2, characterized in that: The step of acquiring the first point cloud and the second point cloud of the target area includes: Acquire a third point cloud and a fourth point cloud of the target area, wherein the acquisition time of the third point cloud is later than the acquisition time of the fourth point cloud; The third point cloud is downsampled according to the voxel size to obtain the first point cloud with uniform distribution, and the fourth point cloud is downsampled according to the voxel size to obtain the second point cloud with uniform distribution.

4. The method according to claim 1 or 2, characterized in that: Based on the pose transformation relationship between the first coordinate system of the first point cloud and the second coordinate system of the second point cloud, before adding the first point cloud to the second point cloud, the method further includes: For any first initial coordinate point of the first point cloud in the first coordinate system, searching for a second initial coordinate point corresponding to the any first initial coordinate point among the second initial coordinate points included in the second point cloud, and determining a posture transformation relationship between the first coordinate system and the second coordinate system based on a plurality of first initial coordinate points and a second initial coordinate point corresponding to each of the plurality of first initial coordinate points; Alternatively, a posture transformation relationship between the first coordinate system and the second coordinate system is obtained from a posture acquisition device, and the posture acquisition device obtains the posture transformation relationship through measurement.

5. The method according to claim 1 or 2, characterized in that: After obtaining the target point cloud, the method further includes: Acquire a fifth point cloud of the target area, wherein a collection pose of the fifth point cloud is different from a collection pose of the first point cloud; The target point cloud is updated based on the fifth point cloud.

6. A point cloud processing device, characterized in that: The device comprises: An acquisition module, used for acquiring a first point cloud and a second point cloud of a target area, wherein the acquisition time of the first point cloud is later than the acquisition time of the second point cloud; An adding module, used for adding the first point cloud to the second point cloud based on the posture transformation relationship between the first coordinate system of the first point cloud and the second coordinate system of the second point cloud, to obtain an initial point cloud including a plurality of first coordinate points and a plurality of second coordinate points, wherein the first coordinate point is the coordinate of the corresponding first point cloud in the second coordinate system, the second coordinate point is the coordinate of the corresponding second point cloud in the second coordinate system, and the phase center of the first laser radar used to establish the connection line also needs to be moved from the first coordinate system to the second coordinate system accordingly; a determining module, used for determining, for any first coordinate point, a first connection line between the any first coordinate point and the phase center of the laser radar used to collect the first point cloud; A removal module, used to determine a coordinate point to be removed from a plurality of second coordinate points included in the initial point cloud, and obtain a target point cloud, wherein the coordinate point to be removed is a second coordinate point whose distance from the first connecting line satisfies a removal condition; The step of determining the coordinate points to be removed from the plurality of second coordinate points included in the initial point cloud comprises: Acquire a reference point set corresponding to any one of the first coordinate points, where the reference point set corresponding to any one of the first coordinate points includes a plurality of reference points located on a first connecting line corresponding to any one of the first coordinate points; Based on the existence of a reference point in the reference point set corresponding to the plurality of first coordinate points, the distance from any second coordinate point being less than a distance threshold, determining the any second coordinate point as a candidate coordinate point; Determine a distance between the candidate coordinate point and a first connecting line corresponding to the candidate coordinate point, and determine the candidate coordinate point as the coordinate point to be removed based on the distance satisfying a removal condition; Or the determining of the coordinate points to be removed from the plurality of second coordinate points included in the initial point cloud comprises: Determining a curvature of a candidate coordinate point among the plurality of second coordinate points, wherein the candidate coordinate point is determined based on the first connecting line; Based on the curvature of the candidate coordinate point being greater than a first threshold, calculating a first distance between the candidate coordinate point and a first connecting line corresponding to the candidate coordinate point; based on the first distance being less than a second threshold, determining that the first distance satisfies the removal condition, and determining that the candidate coordinate point is a coordinate point to be removed; Alternatively, based on the curvature of the candidate coordinate point being less than a third threshold, a second distance between the plane where the candidate coordinate point is located and the first coordinate point corresponding to the candidate coordinate point is calculated; based on the second distance being greater than a fourth threshold, it is determined that the second distance satisfies the removal condition, and the candidate coordinate point is determined to be a coordinate point to be removed.

7. A computer device, characterized in that: The computer device includes a processor and a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the computer device implements the point cloud processing method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that a computer implements the point cloud processing method according to any one of claims 1 to 5.

Images