Point cloud annotation method

Abstract

The present application discloses a point cloud annotation method, which is applied to computer equipment and belongs to the field of surveying and mapping science and technology. The method includes: obtaining a three-dimensional point cloud to be annotated; determining a frame of the annotated point cloud in the three-dimensional point cloud; determining the height difference between the frame and the reference surface where the point cloud is located; automatically adjusting the frame according to the height difference between the frame and the reference surface, and obtaining the annotation result of the point cloud according to the adjusted frame. The height of the reference surface where the point cloud is located is automatically determined according to the coordinates of the three-dimensional point cloud, and the frame is automatically adjusted according to the height difference between the frame and the reference surface to obtain the annotation result of the point cloud, which improves the annotation efficiency while also improving the accuracy of the annotation result.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of surveying and mapping science and technology, and in particular to a point cloud annotation method. Background Art

[0002] With the rapid development of surveying and mapping science and technology, in many scenarios, it is necessary to collect information about objects for computer learning and recognition. For example, in the scenario of autonomous driving, the vehicle needs to automatically identify and avoid objects encountered during driving, so it is necessary to use the 3D point cloud annotation method to annotate the objects in order to obtain relevant information about the objects.

[0003] In the related art, the radar sensor scans an object and obtains relevant data of the scanned object. After processing the relevant data obtained from the scan, a three-dimensional point cloud of the object is obtained. After manually marking the frame in the three-dimensional point cloud, the frame is manually adjusted to fit the frame to the object. In addition, since the object is generally on the ground, it is also necessary to manually confirm whether the z-axis of the frame fits the ground. If the z-axis of the frame does not fit the ground, it is necessary to manually adjust the z-axis of the frame to fit the ground.

[0004] Since it is necessary to manually confirm whether the z-axis of the frame is in contact with the ground, the labeling efficiency is low; in addition, even if the z-axis of the frame is manually adjusted to fit the ground, the z-axis cannot be completely in contact with the ground, resulting in inaccurate labeling results. Summary of the invention

[0005] The embodiments of the present application provide a point cloud annotation method, device, equipment and storage medium, which can be used to solve the problems existing in the related technologies. The technical solution is as follows:

[0006] On the one hand, an embodiment of the present application provides a point cloud annotation method, the method comprising:

[0007] Obtain the three-dimensional point cloud to be annotated;

[0008] Determine a frame of the annotated point cloud in the three-dimensional point cloud;

[0009] Determining a height difference between the frame and a reference surface where the point cloud is located;

[0010] The frame is automatically adjusted according to a height difference between the frame and the reference surface, and a labeling result of the point cloud is obtained according to the adjusted frame.

[0011] In a possible implementation manner, determining the height difference between the frame and the reference plane where the point cloud is located includes:

[0012] Determining the height of the reference surface where the point cloud is located according to the coordinates of the three-dimensional point cloud;

[0013] A height difference between the frame and the reference surface is determined based on the height of the reference surface.

[0014] In a possible implementation manner, determining the height of the reference plane where the point cloud is located according to the coordinates of the three-dimensional point cloud includes:

[0015] Determine points in the three-dimensional point cloud that are on a reference surface;

[0016] The height of the reference surface is determined according to the coordinates of the points on the reference surface.

[0017] In a possible implementation manner, determining the points in the three-dimensional point cloud that are on the reference surface includes:

[0018] Traversing the three-dimensional point cloud, for a first point traversed, calculating angles between the first point and a line connecting a point adjacent to the first point and a horizontal plane, the first point being any point traversed;

[0019] Based on the angles between the first point and the lines connecting the first point's adjacent points and a horizontal plane, it is determined whether the first point is a point on the reference plane.

[0020] In a possible implementation, calculating angles between the first point and a line connecting a point adjacent to the first point and a horizontal plane respectively includes:

[0021] Calculate a first angle between a line connecting the first point and the second point and a horizontal plane, and a second angle between a line connecting the first point and a third point and the horizontal plane, wherein the second point and the third point are adjacent points of the first point respectively;

[0022] The determining whether the first point is a point on the reference plane based on angles between the first point and lines connecting points adjacent to the first point and a horizontal plane comprises:

[0023] If the first angle and the second angle are both smaller than a first threshold, and the difference between the first angle and the second angle is smaller than a second threshold, the first point is a point on the reference plane.

[0024] In a possible implementation manner, after traversing the three-dimensional point cloud, the method further includes:

[0025] Calculating angles between a fourth point and a line connecting a point adjacent to the fourth point and a horizontal plane, wherein the fourth point refers to a point belonging to the same scanning radius as the first point;

[0026] Calculating a first distance between the first point and a coordinate origin, and a second distance between the fourth point and the coordinate origin;

[0027] The determining whether the first point is a point on the reference plane based on angles between the first point and lines connecting points adjacent to the first point and a horizontal plane comprises:

[0028] Based on the angles between the first point and the lines connecting the first point's adjacent points and the horizontal plane, and the angles between the fourth point and the lines connecting the fourth point's adjacent points and the horizontal plane, determine whether the first point belongs to a point on the reference plane.

[0029] In a possible implementation, calculating the angles between the first point and the lines connecting the first point and the adjacent points of the first point and the horizontal plane respectively includes: calculating a fourth angle between the line connecting the first point and a sixth point and the horizontal plane, the sixth point being the adjacent point of the first point;

[0030] The calculating the angles between the connecting line between the fourth point and the adjacent point of the fourth point and the horizontal plane respectively comprises: calculating a third angle between the connecting line between the fourth point and a fifth point and the horizontal plane, the fifth point being the adjacent point of the fourth point;

[0031] The determining whether the first point is a point on the reference plane based on angles respectively included between a line between the first point and an adjacent point of the first point and a horizontal plane, and an angle respectively included between a line between the fourth point and an adjacent point of the fourth point and the horizontal plane, comprises:

[0032] If the difference between the first distance and the second distance is smaller than a third threshold, and the difference between the third angle and the fourth angle is smaller than a fourth threshold, the first point is a point on the reference plane.

[0033] In a possible implementation manner, determining the height of the reference plane according to the coordinates of the point on the reference plane includes:

[0034] Dividing the three-dimensional point cloud into a plurality of grids according to the x-axis coordinates and the y-axis coordinates of the three-dimensional point cloud;

[0035] The height of the reference surface is determined according to the coordinates on the z-axis of the three-dimensional point cloud corresponding to the points on the reference surface in each grid of the multiple grids.

[0036] In a possible implementation, determining the height of the reference surface according to the coordinates on the z-axis of the three-dimensional point cloud corresponding to the point on the reference surface in each of the multiple grids includes:

[0037] averaging the values ​​of the three-dimensional point cloud corresponding to the points on the reference surface in each of the multiple grids on the z-axis coordinate;

[0038] The average value is taken as the height of the reference surface.

[0039] In a possible implementation manner, determining the height difference between the frame and the reference plane based on the height of the reference plane includes:

[0040] Determine the height of the center point of the frame according to the coordinates of the three-dimensional point cloud corresponding to the frame;

[0041] The height difference between the center point of the frame and the reference surface is used as the height difference between the frame and the reference surface.

[0042] On the other hand, a point cloud annotation device is provided, the device comprising:

[0043] An acquisition module, used to acquire a three-dimensional point cloud to be annotated;

[0044] A determination module, used to determine the frame of the marked point cloud in the three-dimensional point cloud;

[0045] The determination module is further used to determine the height difference between the frame and the reference surface where the point cloud is located;

[0046] The adjustment module is used to automatically adjust the frame according to the height difference between the frame and the reference surface, and obtain the annotation result of the point cloud according to the adjusted frame.

[0047] In a possible implementation, the determination module is used to determine the height of a reference plane where the point cloud is located according to the coordinates of the three-dimensional point cloud; and determine the height difference between the frame and the reference plane based on the height of the reference plane.

[0048] In a possible implementation, the determination module is used to determine points on the reference surface in the three-dimensional point cloud; and determine the height of the reference surface according to the coordinates of the points on the reference surface.

[0049] In a possible implementation, the determination module is used to traverse the three-dimensional point cloud, and for the first point traversed, calculate the angles between the first point and the lines connecting the first point's adjacent points and the horizontal plane, where the first point is any point traversed; based on the angles between the first point and the lines connecting the first point's adjacent points and the horizontal plane, determine whether the first point belongs to a point on the reference plane.

[0050] In a possible implementation, the determination module is used to calculate a first angle between a line connecting the first point and the second point and a horizontal plane, and a second angle between a line connecting the first point and a third point and the horizontal plane, wherein the second point and the third point are respectively adjacent points of the first point;

[0051] The determination module is configured to determine that the first point belongs to a point on the reference plane if both the first angle and the second angle are smaller than a first threshold, and the difference between the first angle and the second angle is smaller than a second threshold.

[0052] In a possible implementation, the determination module is further configured to calculate angles between a fourth point and a line connecting a point adjacent to the fourth point and a horizontal plane, the fourth point being a point belonging to the same scanning radius as the first point; calculate a first distance between the first point and a coordinate origin, and a second distance between the fourth point and the coordinate origin;

[0053] The determination module is used to determine whether the first point belongs to a point on the reference plane based on the angles between the lines connecting the first point and its adjacent points and the horizontal plane, and the angles between the lines connecting the fourth point and its adjacent points and the horizontal plane.

[0054] In a possible implementation, the determination module is configured to calculate a fourth angle between a line between the first point and a sixth point and a horizontal plane, where the sixth point is an adjacent point to the first point; and calculate a third angle between a line between the fourth point and a fifth point and the horizontal plane, where the fifth point is an adjacent point to the fourth point;

[0055] The determination module is configured to determine that the first point belongs to a point on the reference plane if the difference between the first distance and the second distance is smaller than a third threshold, and the difference between the third angle and the fourth angle is smaller than a fourth threshold.

[0056] In a possible implementation, the determination module is used to divide the three-dimensional point cloud into multiple grids according to the x-axis coordinates and y-axis coordinates of the three-dimensional point cloud; and determine the height of the reference surface according to the coordinates on the z-axis of the three-dimensional point cloud corresponding to the points on the reference surface in each of the multiple grids.

[0057] In a possible implementation, the determination module is used to average the values ​​of the three-dimensional point cloud corresponding to the points on the reference surface in each grid of the multiple grids on the z-axis coordinate; and use the average value as the height of the reference surface.

[0058] In a possible implementation, the determination module is used to determine the height of the center point of the frame according to the coordinates of the three-dimensional point cloud corresponding to the frame; and use the height difference between the center point of the frame and the reference surface as the height difference between the frame and the reference surface.

[0059] On the other hand, a computer device is provided, comprising a processor and a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the computer device implements any of the above-mentioned point cloud annotation methods.

[0060] On the other hand, a computer-readable storage medium is also provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor so that a computer implements any of the above-mentioned point cloud annotation methods.

[0061] On the other hand, a computer program product or a computer program is also provided, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs any of the above-mentioned point cloud annotation methods.

[0062] The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:

[0063] In this point cloud annotation method, the height difference between the frame and the reference surface where the point cloud is located is automatically determined according to the coordinates of the three-dimensional point cloud, and the frame and the reference surface are automatically adjusted according to the height difference between the frame and the reference surface to obtain the annotation result of the point cloud. Compared with manually adjusting the z-axis of the frame to fit the reference surface, this method improves the annotation efficiency and the accuracy of the annotation result by automatically adjusting the frame. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0065] Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0066] Figure 2 is a flow chart of a point cloud annotation method provided in an embodiment of the present application;

[0067] Figure 3 is a schematic diagram of vertically traversing a three-dimensional point cloud provided by an embodiment of the present application;

[0068] Figure 4 is a schematic diagram of horizontally traversing a three-dimensional point cloud provided by an embodiment of the present application;

[0069] Figure 5 is a schematic diagram of a ground height grid provided in an embodiment of the present application;

[0070] Figure 6 is a schematic diagram of a point cloud annotation device provided in an embodiment of the present application;

[0071] Figure 7 It is a structural diagram of a server provided in an embodiment of the present application;

[0072] Figure 8 It is a structural diagram of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0074] The present application embodiment provides a point cloud annotation method, which is executed by a computer device. The computer device is a terminal or a server. Figure 1 , which shows a schematic diagram of an implementation environment of the method provided in an embodiment of the present application. The implementation environment may include: a terminal 11 and a server 12.

[0075] Among them, the terminal 11 is installed with an application or webpage that can annotate the point cloud. When the application or webpage needs to annotate the point cloud, the method provided in the embodiment of the present application can be used to annotate. The server 12 can store the three-dimensional point cloud data scanned by the radar sensor, and the terminal 11 can obtain the three-dimensional point cloud data scanned by the radar sensor from the server 12. Of course, the terminal 11 can also store the obtained three-dimensional point cloud data.

[0076] Optionally, the terminal 11 may be a smart device such as a tablet computer, a personal computer, etc. The server 12 may be a single server, or a server cluster consisting of multiple servers, or a cloud computing service center. The terminal 11 establishes a communication connection with the server 12 via a wired or wireless network.

[0077] Optionally, the terminal 11 can be any electronic product that can interact with the user through one or more methods such as a keyboard, a touch pad, a touch screen, a remote control, voice interaction or a handwriting device, such as a PC (Personal Computer), a PPC (Pocket PC), a tablet computer, etc. The server 12 can be a server, a server cluster consisting of multiple servers, or a cloud computing service center. The terminal 11 establishes a communication connection with the server 12 through a wired or wireless network.

[0078] Those skilled in the art should understand that the above-mentioned terminal 11 and server 12 are only examples, and other existing or future terminals or servers that are applicable to the present application should also be included in the protection scope of the present application and are included here by reference.

[0079] The present application embodiment provides a point cloud annotation method, which can be applied to the above Figure 1 The implementation environment shown. Figure 2 As shown, taking the method applied to a computer device as an example, the method includes steps 201 to 204.

[0080] In step 201, a computer device obtains a three-dimensional point cloud to be labeled.

[0081] In the method provided in the embodiment of the present application, a three-dimensional point cloud can be obtained by a radar sensor. For example, after the radar sensor scans the area by emitting a laser beam, the three-dimensional point cloud data to be annotated can be obtained. The radar sensor uploads the obtained three-dimensional point cloud data to a computer device, and the computer device can receive the three-dimensional point cloud. The three-dimensional point cloud can be used to annotate the frame of the point cloud and determine the height of the reference surface. It should be noted that after the radar sensor scans the object, the three-dimensional point cloud to be annotated corresponding to the object can be obtained.

[0082] When scanning an area, the number of radar sensors is not limited, and can be one or more. The installation location of the radar sensor is not fixed, for example, it can be installed on a moving vehicle, as long as it can scan the area as required and collect relevant data of the point cloud to be annotated. The type of point cloud to be annotated is also not limited, and can be a car or a pedestrian.

[0083] In addition, the coordinates of the data obtained by scanning the radar sensor may be based on the coordinates of the radar sensor, that is, the position of the radar sensor is the origin of the coordinates.

[0084] In step 202, the computer device determines a bounding box of the annotated point cloud in the three-dimensional point cloud.

[0085] After the computer device receives the three-dimensional point cloud data uploaded by the radar sensor, it can mark the point cloud with a frame.

[0086] Among them, when annotating the point cloud frame, the methods that can be used include but are not limited to the following two:

[0087] The first method is manual labeling. In this method, the point clouds of different consecutive frames are manually labeled with boxes.

[0088] When the radar sensor scans the area, different frames of 3D point clouds can be obtained at different times. After the 3D point clouds of multiple consecutive frames are synthesized, a 3D point cloud is formed. When scanning an area, the number of radar sensors can be one or more. After a radar sensor scans the area, a 3D point cloud can be obtained. The 3D point cloud records the x, y, z coordinates of each point, the distance relative to the radar, and the angle.

[0089] The second method is differential labeling. In this method, after manually labeling the point clouds in two non-adjacent frames, the computer device can automatically predict the interval frame in a time difference manner, and then label the point clouds in the interval frame respectively. It should be noted that differential labeling is based on the principle of uniform motion, that is, it is assumed that the movement of the object corresponding to the labeled point cloud in a short period of time conforms to uniform motion.

[0090] When annotating a frame to a 3D point cloud, the frame needs to fit the point cloud, that is, the size of the frame cannot be too large or too small compared to the corresponding frame, to ensure that the frame can accurately reflect the size, position and other information of the object corresponding to the point cloud. Therefore, after the 3D point cloud is annotated with a frame, the xy axis of the frame can be manually adjusted to fit the point cloud. Regardless of which of the above annotation methods is used to annotate the point cloud frame, the xy axis of the frame can be manually adjusted to ensure that the frame can accurately reflect the size, position and other information of the object corresponding to the point cloud on the xy plane.

[0091] In step 203, the computer device determines the height difference between the frame and the reference surface where the point cloud is located.

[0092] In a possible implementation, determining the height difference between the frame and the reference plane where the point cloud is located includes: determining the height of the reference plane where the point cloud is located according to the coordinates of the three-dimensional point cloud; and determining the height difference between the frame and the reference plane based on the height of the reference plane.

[0093] In a possible implementation, determining the height of a reference surface on which the object is located according to the coordinates of a three-dimensional point cloud includes: determining points on the reference surface in the three-dimensional point cloud; and determining the height of the reference surface according to the coordinates of the points on the reference surface.

[0094] In one possible implementation, determining a point in a three-dimensional point cloud that is on a reference plane includes: traversing the three-dimensional point cloud, and for a first point traversed, calculating angles between the first point and a line connecting the first point's adjacent points and a horizontal plane, where the first point is any point traversed; and determining whether the first point is on the reference plane based on angles between the first point and a line connecting the first point's adjacent points and the horizontal plane.

[0095] In one possible implementation, the angles between a first point and a line between adjacent points of the first point and a horizontal plane are calculated, including: calculating a first angle between a line between a first point and a second point and the horizontal plane, and a second angle between a line between a first point and a third point and the horizontal plane, the second point and the third point being adjacent points of the first point respectively; based on the angles between the first point and the line between adjacent points of the first point and the horizontal plane, determining whether the first point belongs to a point on a reference plane includes: if the first angle and the second angle are both less than a first threshold, and the difference between the first angle and the second angle is less than a second threshold, then the first point belongs to a point on the reference plane.

[0096] Figure 3 It is a schematic diagram of longitudinal traversal of a three-dimensional point cloud. The figure has three points A, B, and C, a rear view of the car, and a schematic diagram of the roof radar. The radar sensor is located on the top of the car and scans the area in the figure by emitting a laser beam. As shown in the figure, points A, B, and C are any three adjacent points in the three-dimensional point cloud, and it is unknown whether points A, B, and C are points on the reference plane. In this case, if you want to calculate whether point B is a point on the reference plane, you need to calculate the angle α between the line between points A and B and the horizontal plane. AB , and calculate the angle α between the line connecting point B and point C and the horizontal plane BC .

[0097] If the coordinates of point A are known to be (x A ,y A ,z A ), the coordinates of point B are (x B ,y B ,z B ), the coordinates of point C are (x C ,y C ,z C ), then α can be calculated according to the following formula (1): AB :

[0098]

[0099] Calculate α according to the following formula (2): BC :

[0100]

[0101] If α AB and α BC are all less than the first threshold, and α BC -α ABIf the value of point B is less than the second threshold, point B can be determined to be a point on the reference surface. The first threshold and the second threshold can be empirical values ​​or values ​​determined according to the application scenario. Similarly, point C is determined to be a point on the reference surface. If the first point that does not belong to the reference surface is found according to the above method, the backward search is stopped, and the points before this point are all points on the reference surface.

[0102] In a possible implementation, after traversing the three-dimensional point cloud, it also includes: calculating the angles between the fourth point and the lines between the fourth point's adjacent points and the horizontal plane, the fourth point refers to a point belonging to the same scanning radius as the first point; calculating a first distance between the first point and the coordinate origin, and a second distance between the fourth point and the coordinate origin; based on the angles between the first point and the lines between the first point's adjacent points and the horizontal plane, determining whether the first point belongs to a point on the reference plane, including: based on the angles between the first point and the first point's adjacent points and the horizontal plane, and the angle between the fourth point and the fourth point's adjacent points and the horizontal plane, determining whether the first point belongs to a point on the reference plane.

[0103] In one possible implementation, the angles between the first point and the lines between the first point's adjacent points and the horizontal plane are calculated, including: calculating the fourth angle between the line between the first point and the sixth point and the horizontal plane, the sixth point being an adjacent point of the first point; the angles between the fourth point and the lines between the fourth point and the adjacent points of the fourth point and the horizontal plane are calculated, including: calculating the third angle between the line between the fourth point and the fifth point and the horizontal plane, the fifth point being an adjacent point of the fourth point; based on the angles between the first point and the first point's adjacent points and the horizontal plane, and the angle between the line between the fourth point and the fourth point's adjacent points and the horizontal plane, determining whether the first point belongs to a point on a reference plane includes: if the difference between the first distance and the second distance is less than a third threshold, and the difference between the third angle and the fourth angle is less than the fourth threshold, then the first point belongs to a point on the reference plane.

[0104] Figure 4 The figure is a schematic diagram of traversing a 3D point cloud horizontally. The figure contains four points A, B, D, and E, a top view of the car, and a schematic diagram of the roof radar. The radar sensor is located on the top of the car and scans the area in the figure by emitting a laser beam. Points A, B, and E are known points on the reference surface. To determine whether point D is a point on the reference surface, first calculate the distance L between point A and the origin of the coordinate system. A , and then calculate the distance L between point D and the origin of the coordinate system D Then calculate the angle α between the line connecting point A and point B and the horizontal plane AB , and calculate the angle α between the line connecting point D and point E and the horizontal plane DE . Point A and point D are two points located on the same scanning radius. If the coordinates of point A are known to be (xA ,y A ,z A ), the coordinates of point B are (x B ,y B ,z B ), the coordinates of point D are (x D ,y D ,z D ), the coordinates of point E are (x E ,y E ,z E ), then L A The calculation method is shown in formula (3):

[0105]

[0106] L D The calculation method is shown in formula (4):

[0107]

[0108] α AB The calculation method of is shown in formula (5):

[0109]

[0110] α DE The calculation method of is shown in formula (6):

[0111]

[0112] If L D and L A The difference is less than the third threshold, and α AB and α DE If the difference between is less than the fourth threshold, it can be determined that point D is a point on the reference surface. The third threshold and the fourth threshold can be empirical values ​​or values ​​determined according to the application scenario.

[0113] It is worth noting that compared with Figure 3 The method for determining points on a reference surface is shown. The premise for determining points on the reference surface by method 2 is that at least three points on the reference surface are known, and one of the three points on the reference surface must be located on the same scanning radius as the point to be determined.

[0114] Regardless of which of the above methods is used to determine the points on the reference surface, the height of the reference surface is determined based on the coordinates of the points on the reference surface, including but not limited to: dividing the three-dimensional point cloud into multiple grids based on the x-axis coordinates and y-axis coordinates of the three-dimensional point cloud; and determining the height of the reference surface based on the coordinates on the z-axis of the three-dimensional point cloud corresponding to the points on the reference surface in each of the multiple grids.

[0115] In a possible implementation, the height of the reference surface is determined based on the coordinates on the z-axis of the points on the reference surface in each of the multiple grids, including: averaging the values ​​on the z-axis of the three-dimensional point clouds corresponding to the points on the reference surface in each of the multiple grids; and using the average value as the height of the reference surface.

[0116] After the computer device obtains the three-dimensional point cloud to be annotated, it divides the three-dimensional point cloud into multiple grids on the xy plane, wherein the embodiment of the present application does not limit the size of the grid. According to the coordinates of the points on the reference surface in the three-dimensional point cloud determined in the above steps, the average value of the points on the reference surface falling into each grid on the z-axis is calculated, and the average value is the height of the reference surface of the area represented by each grid. The height grid of the reference surface can be output based on the grids divided on the xy plane and the height of the reference surface of each grid, and the terminal can automatically find the height of the reference surface where the point cloud is located through the height grid of the reference surface.

[0117] by Figure 5 As shown in the figure, the reference surface is set to the ground. It should be noted that the reference surface includes but is not limited to the ground. Point A, i.e., the origin of the coordinate system, represents the location of the radar sensor, and the size of the divided grid is 1.0*1.0 meters. Figure 5 Each grid in contains two parts of data, the coordinate value and the z-axis value. The coordinate value represents the coordinate of the upper left corner of each grid on the xy plane. For example, (1,0) represents the coordinate of point B on the xy plane, that is, the coordinate of point B on the x-axis is 1 and the coordinate on the y-axis is 0. The z-axis value represents the height of the ground in the grid. One grid corresponds to the height of a piece of ground. For example, for grid C, z=-1 means that the ground height of the area corresponding to grid C is -1.

[0118] It should be noted that the coordinates of the points on the ground are different when the radar sensor scans different areas. Therefore, after averaging the values ​​of the three-dimensional point cloud corresponding to the points on the ground in the grid on the z-axis coordinate, the output ground height grid is also different. The size of the grid contained in the ground height grid is not limited, and can be 1.0*1.0 meters or 0.5*0.5 meters, as long as the sizes of the grids are the same.

[0119] After obtaining the ground height grid, the computer device searches for the ground height grid according to the coordinates of any point in the 3D point cloud on the xy plane to obtain the ground height where the point cloud is located. For example, if the coordinates of point D in the 3D point cloud on the xy plane are (1.5, 1.5), then Figure 5Find the height z of the ground where the point cloud is located = -2.

[0120] It should be noted that, because the radar sensor is usually located on a moving vehicle, the ground points collected by the radar sensor are always located below the radar sensor in the z-axis direction. Therefore, the value of z in the ground height grid is always a negative number.

[0121] After the computer device obtains the height of the reference surface where the object is located, it can further calculate the height difference between the frame of the point cloud and the reference surface where the point cloud is located, and then automatically adjust the frame to fit the reference surface according to the height difference.

[0122] In a possible implementation, determining the height difference between the frame and the reference plane based on the height of the reference plane includes: determining the height of the center point of the frame according to the coordinates of the three-dimensional point cloud corresponding to the frame; and using the height difference between the center point of the frame and the reference plane as the height difference between the frame and the reference plane.

[0123] For example, after the computer device determines the height of the reference surface where the point cloud is located, it is also necessary to determine the height difference between the frame of the point cloud and the reference surface, and then adjust the frame to fit the reference surface. After the frame of the annotated point cloud is determined, the coordinates of the three-dimensional point cloud corresponding to the frame are also determined, so the coordinates of the center point of the frame on the z-axis can be known, and then the height difference between the center point of the frame and the reference surface can be obtained. Among them, the height difference between the frame of the point cloud and the reference surface where the point cloud is located can be the height difference between the center point of the frame and the reference surface.

[0124] In step 204, the computer device automatically adjusts the frame according to the height difference between the frame and the reference surface, and obtains the annotation result of the point cloud according to the adjusted frame.

[0125] For example, the reference surface is the ground, and the object corresponding to the point cloud is a vehicle traveling on the ground. Because the vehicle is in contact with the ground, if the frame of the vehicle is not in contact with the ground, then the frame of the vehicle cannot accurately represent the size, position and other information of the vehicle. Therefore, the frame needs to be adjusted to make it in contact with the ground, so as to obtain an accurate labeling result of the vehicle.

[0126] In the embodiment of the present application, the height difference between the frame and the reference surface where the point cloud is located is automatically determined according to the coordinates of the three-dimensional point cloud, and the frame and the reference surface are automatically adjusted according to the height difference between the frame and the reference surface to obtain the annotation result of the point cloud. Compared with manually adjusting the z-axis of the frame to fit the reference surface, this method improves the annotation efficiency and the accuracy of the annotation result by automatically adjusting the frame.

[0127] See also Figure 6 The present application embodiment provides a point cloud annotation device, which is used to perform the above Figure 2 The point cloud annotation method shown in Figure 2 is shown in Figure 2. Figure 6 As shown, the device comprises:

[0128] An acquisition module 601 is used to acquire a three-dimensional point cloud to be annotated;

[0129] A determination module 602 is used to determine a frame of a marked point cloud in a three-dimensional point cloud;

[0130] The determination module 602 is further used to determine the height difference between the frame and the reference surface where the point cloud is located;

[0131] The adjustment module 603 is used to automatically adjust the frame according to the height difference between the frame and the reference surface, and obtain the annotation result of the point cloud according to the adjusted frame.

[0132] In a possible implementation, the determination module 602 is used to determine the height of the reference surface where the point cloud is located according to the coordinates of the three-dimensional point cloud; and determine the height difference between the frame and the reference surface based on the height of the reference surface.

[0133] In a possible implementation, the determination module 602 is used to determine points on the reference surface in the three-dimensional point cloud; and determine the height of the reference surface according to the coordinates of the points on the reference surface.

[0134] In one possible implementation, the determination module 602 is used to traverse the three-dimensional point cloud, and for the first point traversed, calculate the angles between the first point and the lines connecting the first point's adjacent points and the horizontal plane, where the first point is any point traversed; based on the angles between the first point and the lines connecting the first point's adjacent points and the horizontal plane, determine whether the first point belongs to a point on the reference plane.

[0135] In a possible implementation, the determination module 602 is configured to calculate a first angle between a line connecting a first point and a second point and a horizontal plane, and a second angle between a line connecting the first point and a third point and the horizontal plane, wherein the second point and the third point are adjacent points of the first point respectively;

[0136] The determination module 602 is configured to determine that the first point belongs to a point on the reference plane if both the first angle and the second angle are smaller than a first threshold and the difference between the first angle and the second angle is smaller than a second threshold.

[0137] In a possible implementation, the determination module 602 is further configured to calculate angles between a fourth point and a line connecting points adjacent to the fourth point and a horizontal plane, where the fourth point refers to a point belonging to the same scanning radius as the first point; calculate a first distance between the first point and the coordinate origin, and a second distance between the fourth point and the coordinate origin;

[0138] The determination module 602 is used to determine whether the first point is a point on the reference plane based on the angles between the lines between the first point and its adjacent points and the horizontal plane, and the angles between the lines between the fourth point and its adjacent points and the horizontal plane.

[0139] In a possible implementation, the determination module 602 is configured to calculate a fourth angle between a line connecting the first point and a sixth point and a horizontal plane, where the sixth point is an adjacent point to the first point; calculate a third angle between a line connecting the fourth point and a fifth point and the horizontal plane, where the fifth point is an adjacent point to the fourth point;

[0140] The determination module 602 is configured to determine that the first point belongs to a point on the reference plane if the difference between the first distance and the second distance is less than a third threshold, and the difference between the third angle and the fourth angle is less than a fourth threshold.

[0141] In one possible implementation, the determination module 602 is used to divide the three-dimensional point cloud into multiple grids according to the x-axis coordinates and y-axis coordinates of the three-dimensional point cloud; and determine the height of the reference surface according to the coordinates on the z-axis of the three-dimensional point cloud corresponding to the points on the reference surface in each of the multiple grids.

[0142] In a possible implementation, the determination module 602 is used to average the values ​​of the three-dimensional point cloud corresponding to the points on the reference surface in each grid of the multiple grids on the z-axis coordinate; and use the average value as the height of the reference surface.

[0143] In a possible implementation, the determination module 602 is used to determine the height of the center point of the frame according to the coordinates of the three-dimensional point cloud corresponding to the frame; and use the height difference between the center point of the frame and the reference surface as the height difference between the frame and the reference surface.

[0144] In the embodiment of the present application, the height difference between the frame and the reference surface where the point cloud is located is automatically determined according to the coordinates of the three-dimensional point cloud, and the frame and the reference surface are automatically adjusted according to the height difference between the frame and the reference surface to obtain the annotation result of the point cloud. Compared with manually adjusting the z-axis of the frame to fit the reference surface, this method improves the annotation efficiency and the accuracy of the annotation result by automatically adjusting the frame.

[0145] It should be noted that the device provided in the above embodiment only uses the division of the above functional modules as an example to implement its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and their specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0146] Figure 7It is a structural diagram of a server provided in an embodiment of the present application. The server may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 701 and one or more memories 702, wherein at least one computer program is stored in the one or more memories 702, and the at least one computer program is loaded and executed by the one or more processors 701, so that the server implements the point cloud annotation method provided by the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server may also include other components for implementing device functions, which will not be repeated here.

[0147] Figure 8 1 is a schematic diagram of a terminal structure provided in an embodiment of the present application. For example, the terminal may be a tablet computer, a laptop computer, or a desktop computer. The terminal may also be referred to as a user device, a portable terminal, a laptop terminal, a desktop terminal, or other names.

[0148] Typically, the terminal includes: a processor 1501 and a memory 1502 .

[0149] The processor 1501 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1501 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1501 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1501 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1501 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0150] The memory 1502 may include one or more computer-readable storage media, which may be non-transitory. The memory 1502 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1502 is used to store at least one instruction, which is used to be executed by the processor 1501 so that the terminal implements the point cloud annotation method provided in the method embodiment of the present application.

[0151] In some embodiments, the terminal may also optionally include: a peripheral device interface 1503 and at least one peripheral device. The processor 1501, the memory 1502 and the peripheral device interface 1503 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 1503 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 1504, a display screen 1505, a camera assembly 1506, an audio circuit 1507, a positioning assembly 1508 and a power supply 1509.

[0152] The peripheral device interface 1503 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1501 and the memory 1502. In some embodiments, the processor 1501, the memory 1502, and the peripheral device interface 1503 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1501, the memory 1502, and the peripheral device interface 1503 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0153] The radio frequency circuit 1504 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1504 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 1504 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 1504 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 1504 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 1504 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0154] The display screen 1505 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1505 is a touch display screen, the display screen 1505 also has the ability to collect touch signals on the surface or above the surface of the display screen 1505. The touch signal can be input to the processor 1501 as a control signal for processing. At this time, the display screen 1505 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 1505 can be one, which is set on the front panel of the terminal; in other embodiments, the display screen 1505 can be at least two, which are respectively set on different surfaces of the terminal or are folded; in other embodiments, the display screen 1505 can be a flexible display screen, which is set on the curved surface or folded surface of the terminal. Even, the display screen 1505 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1505 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0155] The camera assembly 1506 is used to capture images or videos. Optionally, the camera assembly 1506 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1506 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0156] The audio circuit 1507 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 1501 for processing, or input them into the RF circuit 1504 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 1501 or the RF circuit 1504 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 1507 may also include a headphone jack.

[0157] The positioning component 1508 is used to locate the current geographical location of the terminal to implement navigation or LBS (Location Based Service). The positioning component 1508 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system, Russia's Grenas system or the European Union's Galileo system.

[0158] The power supply 1509 is used to power various components in the terminal. The power supply 1509 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1509 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0159] In some embodiments, the terminal further includes one or more sensors 1510 , including but not limited to: an acceleration sensor 1511 , a gyroscope sensor 1512 , a pressure sensor 1513 , a fingerprint sensor 1514 , an optical sensor 1515 , and a proximity sensor 1516 .

[0160] The acceleration sensor 1511 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal. For example, the acceleration sensor 1511 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 1501 can control the display screen 1505 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 1511. The acceleration sensor 1511 can also be used for collecting game or user motion data.

[0161] The gyro sensor 1512 can detect the body direction and rotation angle of the terminal, and the gyro sensor 1512 can cooperate with the acceleration sensor 1511 to collect the user's 3D actions on the terminal. The processor 1501 can implement the following functions based on the data collected by the gyro sensor 1512: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0162] The pressure sensor 1513 can be set in the side frame of the terminal and / or the lower layer of the display screen 1505. When the pressure sensor 1513 is set in the side frame of the terminal, the user's holding signal of the terminal can be detected, and the processor 1501 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 1513. When the pressure sensor 1513 is set in the lower layer of the display screen 1505, the processor 1501 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 1505. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0163] The fingerprint sensor 1514 is used to collect the user's fingerprint, and the processor 1501 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 1514, or the fingerprint sensor 1514 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 1501 authorizes the user to perform relevant sensitive operations, which include unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 1514 can be set on the front, back or side of the terminal. When a physical button or a manufacturer logo is set on the terminal, the fingerprint sensor 1514 can be integrated with the physical button or the manufacturer logo.

[0164] The optical sensor 1515 is used to collect the ambient light intensity. In one embodiment, the processor 1501 can control the display brightness of the display screen 1505 according to the ambient light intensity collected by the optical sensor 1515. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1505 is increased; when the ambient light intensity is low, the display brightness of the display screen 1505 is reduced. In another embodiment, the processor 1501 can also dynamically adjust the shooting parameters of the camera component 1506 according to the ambient light intensity collected by the optical sensor 1515.

[0165] The proximity sensor 1516, also known as a distance sensor, is usually arranged on the front panel of the terminal. The proximity sensor 1516 is used to collect the distance between the user and the front of the terminal. In one embodiment, when the proximity sensor 1516 detects that the distance between the user and the front of the terminal is gradually decreasing, the processor 1501 controls the display screen 1505 to switch from the screen-on state to the screen-off state; when the proximity sensor 1516 detects that the distance between the user and the front of the terminal is gradually increasing, the processor 1501 controls the display screen 1505 to switch from the screen-off state to the screen-on state.

[0166] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the terminal, and may include more or less components than those shown in the figure, or combine certain components, or adopt a different component arrangement.

[0167] In an exemplary embodiment, a computer device is also provided, the computer device comprising a processor and a memory, wherein at least one computer program is stored in the memory. The at least one computer program is loaded and executed by one or more processors, so that the computer device implements any of the above-mentioned point cloud annotation methods.

[0168] In an exemplary embodiment, a computer-readable storage medium is also provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor of a computer device to enable the computer to implement any of the above-mentioned point cloud annotation methods.

[0169] In one possible implementation, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0170] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs any of the above-mentioned point cloud annotation methods.

[0171] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the three-dimensional point cloud of the area containing the object to be annotated involved in this application is obtained with full authorization.

[0172] It should be understood that the "plurality" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0173] It should be noted that the terms "first", "second", etc. (if any) in the specification and claims of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0174] The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A point cloud annotation method, characterized in that: The method is applied to a computer device, and the method comprises: Obtain the three-dimensional point cloud to be annotated; Determine a frame of the annotated point cloud in the three-dimensional point cloud; Determining the height of the reference surface where the point cloud is located according to the coordinates of the three-dimensional point cloud, specifically comprising: determining points in the three-dimensional point cloud that are on the reference surface; dividing the three-dimensional point cloud into a plurality of grids according to the x-axis coordinates and the y-axis coordinates of the three-dimensional point cloud; Determine the height of the reference surface according to the coordinates on the z-axis of the three-dimensional point cloud corresponding to the points on the reference surface in each of the multiple grids; Determining a height difference between the frame and the reference surface based on the height of the reference surface; The frame is automatically adjusted according to a height difference between the frame and the reference surface, and a labeling result of the point cloud is obtained according to the adjusted frame.

2. The method according to claim 1, characterized in that The step of determining points in the three-dimensional point cloud that are on the reference surface includes: Traversing the three-dimensional point cloud, for a first point traversed, calculating angles between the first point and a line connecting a point adjacent to the first point and a horizontal plane, the first point being any point traversed; Based on the angles between the first point and the lines connecting the first point's adjacent points and a horizontal plane, it is determined whether the first point is a point on the reference plane.

3. The method according to claim 2, characterized in that The calculating the angles between the first point and the lines connecting the first point's adjacent points and the horizontal plane respectively includes: Calculate a first angle between a line connecting the first point and the second point and a horizontal plane, and a second angle between a line connecting the first point and a third point and the horizontal plane, wherein the second point and the third point are adjacent points of the first point respectively; The determining whether the first point is a point on the reference plane based on angles between the first point and lines connecting points adjacent to the first point and a horizontal plane comprises: If the first angle and the second angle are both smaller than a first threshold, and the difference between the first angle and the second angle is smaller than a second threshold, the first point is a point on the reference plane.

4. The method according to claim 2, characterized in that: After traversing the three-dimensional point cloud, the method further includes: Calculating angles between a fourth point and a line connecting a point adjacent to the fourth point and a horizontal plane, wherein the fourth point refers to a point belonging to the same scanning radius as the first point; Calculating a first distance between the first point and a coordinate origin, and a second distance between the fourth point and the coordinate origin; The determining whether the first point is a point on the reference plane based on angles between the first point and lines connecting points adjacent to the first point and a horizontal plane comprises: Based on the angles between the first point and the lines connecting the first point's adjacent points and the horizontal plane, and the angles between the fourth point and the lines connecting the fourth point's adjacent points and the horizontal plane, determine whether the first point belongs to a point on the reference plane.

5. The method according to claim 4, characterized in that The calculating the angles between the first point and the line connecting the first point and the adjacent point and the horizontal plane respectively includes: calculating a fourth angle between the line connecting the first point and a sixth point and the horizontal plane, the sixth point being the adjacent point of the first point; The calculating the angles between the connecting line between the fourth point and the adjacent point of the fourth point and the horizontal plane respectively comprises: calculating a third angle between the connecting line between the fourth point and a fifth point and the horizontal plane, the fifth point being the adjacent point of the fourth point; The determining whether the first point is a point on the reference plane based on angles respectively included between a line between the first point and an adjacent point of the first point and a horizontal plane, and an angle respectively included between a line between the fourth point and an adjacent point of the fourth point and the horizontal plane, comprises: If the difference between the first distance and the second distance is smaller than a third threshold, and the difference between the third angle and the fourth angle is smaller than a fourth threshold, the first point is a point on the reference plane.

6. The method according to claim 1, characterized in that The determining the height of the reference surface according to the coordinates on the z-axis of the three-dimensional point cloud corresponding to the points on the reference surface in each of the multiple grids comprises: averaging the values ​​of the three-dimensional point cloud corresponding to the points on the reference surface in each of the plurality of grids on the z-axis coordinate; The average value is taken as the height of the reference surface.

7. The method according to claim 1, characterized in that The determining the height difference between the frame and the reference surface based on the height of the reference surface comprises: Determine the height of the center point of the frame according to the coordinates of the three-dimensional point cloud corresponding to the frame; The height difference between the center point of the frame and the reference surface is used as the height difference between the frame and the reference surface.

Images