Point cloud data processing method and apparatus, and computer-readable storage medium

Abstract

The application discloses a point cloud data processing method, device and computer readable storage medium. The method is applied to a mobile robot and comprises the following steps: acquiring point cloud data and robot movement data; wherein the sampling frame rate of the point cloud data is lower than the sampling frame rate of the movement data; performing a correction operation on previous frame point cloud data by using the movement data to obtain corrected point cloud data; generating supplementary point cloud data corresponding to current frame movement data of the robot according to the global coordinate positions of each sampling point in the corrected point cloud data and the current frame movement data of the robot; after the current frame point cloud is collected, the next frame movement data is taken as the current frame movement data, the current frame point cloud data acquired is taken as the previous frame point cloud data, and the correction operation on the previous frame point cloud data by using the movement data and the subsequent supplementing step are returned. Through the above method, the accuracy and generation frequency of the point cloud data can be improved.

Description

Technical Field

[0001] The present application relates to the field of point cloud processing, and in particular to a point cloud data processing method, device, and computer-readable storage medium. Background Art

[0002] In order to complete complex tasks, robots often need to be equipped with a variety of sensors, the most common of which are lidar and odometer.

[0003] Planar LiDAR detects its surroundings by rotating to emit and receive laser light at regular angular intervals. This mechanical rotational acquisition method is limited by rotational speed and cannot instantly capture point cloud information about the surrounding environment. Robots, on the other hand, need to move within the environment to complete certain tasks, driving the LiDAR's movement. Therefore, the robot is not fixed in the same position during acquisition. Summary of the Invention

[0004] This application mainly provides a point cloud data processing method, device and computer-readable storage medium, which solves the problems of low point cloud sampling accuracy and low point cloud data output frequency in the prior art.

[0005] In order to solve the above technical problems, the first aspect of the present application provides a point cloud data processing method, including: acquiring point cloud data and robot movement data; wherein the sampling frame rate of the point cloud data is lower than the sampling frame rate of the movement data; using the movement data to perform a correction operation on the previous frame point cloud data to obtain corrected point cloud data; generating supplementary point cloud data corresponding to the current frame movement data according to the global coordinate position of each sampling point in the corrected point cloud data and the current frame movement data of the robot; using the next frame movement data as the current frame movement data, and the current frame point cloud data collected by the lidar sensor as the previous frame point cloud data, returning the correction operation on the previous frame point cloud data using the movement data and subsequent steps.

[0006] In order to solve the above technical problems, the second aspect of the present application provides a point cloud data processing device, comprising a processor and a memory coupled to each other; a computer program is stored in the memory, and the processor is used to execute the computer program to implement the point cloud data processing method provided in the first aspect above.

[0007] In order to solve the above technical problems, the third aspect of the present application provides a computer-readable storage medium, which stores program data. When the program data is executed by a processor, the point cloud data processing method provided by the first aspect is implemented.

[0008] The beneficial effects of the present application are as follows: Different from the prior art, the present application first obtains point cloud data and robot movement data; wherein, the sampling frame rate of the point cloud data is lower than the sampling frame rate of the movement data, and then uses the movement data to perform a correction operation on the previous frame point cloud data to obtain corrected point cloud data; based on the global coordinate position of each sampling point in the corrected point cloud data and the current frame movement data of the robot, generates supplementary point cloud data corresponding to the current frame movement data; uses the next frame movement data as the current frame movement data, and the current frame point cloud data collected by the lidar sensor as the previous frame point cloud data, and returns the correction operation on the previous frame point cloud data using the movement data and subsequent steps. The above method uses the posture information of the robot during movement in the movement data to correct the point cloud data, improves the accuracy of the point cloud data, and generates supplementary point cloud data corresponding to the current frame movement data on the basis of the corrected point cloud, thereby increasing the output frequency of the point cloud results and thereby improving the availability of the point cloud data. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0010] Figure 1 This is a schematic flow chart of an embodiment of the point cloud data processing method of the present application;

[0011] Figure 2 This is a schematic diagram of the mobile data and point cloud data collected by this application;

[0012] Figure 3 This is a schematic flow chart of an embodiment of step S12 of the present application;

[0013] Figure 4 This is a schematic block diagram of the process of determining the initial global posture of the robot according to an embodiment of the present application;

[0014] Figure 5 This is a schematic flow chart of an embodiment of step S122 of the present application;

[0015] Figure 6 This is a schematic flow chart of an embodiment of step S231 of the present application;

[0016] Figure 7 This is a schematic block diagram of a process for determining the global coordinate position of a sampling point after correction according to an embodiment of the present application;

[0017] Figure 8This is a schematic structural block diagram of an embodiment of a point cloud data processing device of the present application;

[0018] Figure 9 This is a schematic structural block diagram of another embodiment of the point cloud data processing device of the present application;

[0019] Figure 10 This is a schematic structural block diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0021] The terms "first" and "second" in this application are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0022] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor do they constitute independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0023] The mobile robot is equipped with a lidar sensor and a motion sensor. The lidar sensor is used to obtain point cloud data around the mobile robot. A frame of point cloud data includes at least the start sampling time, total sampling time, point cloud data volume, distance, and angle information for that frame. The motion sensor is used to obtain the robot's movement information, which includes at least information such as the mobile robot's position, angular orientation, and sampling time. The motion sensor is, for example, an odometer.

[0024] After extensive research, the inventors of this patent discovered that when a mobile robot collects point cloud data while moving, the LiDAR performs environmental detection at its natural frequency, requiring a full acquisition cycle to output a point cloud result. The robot's surroundings remain obscured between sampling intervals. To enhance the robot's perception capabilities between LiDAR sampling intervals, this patent proposes a method for supplementing and generating LiDAR data for use with robots.

[0025] See also Figure 1 , Figure 1 This is a flowchart of an embodiment of the point cloud data processing method of the present application. It should be noted that if there is substantially the same result, this embodiment does not Figure 1 The process sequence shown is limited. This embodiment includes the following steps:

[0026] Step S11: Acquire point cloud data and robot movement data.

[0027] During the movement of the mobile robot, movement data and point cloud data are continuously collected through lidar sensors and motion sensors.

[0028] Among them, the sampling frame rate of point cloud data is lower than the sampling frame rate of mobile data. For example, the sampling frequency of the lidar sensor is 5 Hz, and the sampling frequency of the mobile sensor is 25 Hz. Within the same sampling time, the frame rate of point cloud data is lower than the frame rate of mobile data.

[0029] A frame of point cloud data includes the acquisition time of the point cloud and the distance and direction of each point cloud. A frame of movement data includes the acquisition time of the movement data and the robot posture.

[0030] See also Figure 2 , Figure 2 This is a schematic diagram of the mobile data and point cloud data collected by this application. LDS0, ..., LDSm, LDSm+1 represent point cloud data. m 0.L m 1. ..., L m+1 k、L m+1 n represents the motion data. Multiple frames of motion data can be collected within the time of collecting one frame of point cloud data.

[0031] Step S12: Using the movement data, a correction operation is performed on the previous frame of point cloud data to obtain corrected point cloud data.

[0032] Since the acquisition time span of a frame of point cloud data is relatively long, the mobile robot is in a moving state during the process of collecting point cloud data. Therefore, when collecting different points of a frame of point cloud, the mobile robot may be located at different positions. This step can use the robot movement information in the mobile data to correct the point cloud data.

[0033] The previous frame of point cloud refers to a frame of point cloud data collected in the most recent acquisition cycle in which all points have been collected.

[0034] See also Figure 3 , Figure 3 This is a flowchart of an embodiment of step S12 of the present application. It should be noted that if there is substantially the same result, this embodiment does not necessarily Figure 3 The process sequence shown is limited. This embodiment includes the following steps:

[0035] Step S121: Determine the initial global pose of the robot at the start sampling time according to the start sampling time of the previous frame of point cloud data and the two frames of movement data adjacent to the start sampling time.

[0036] The sampling start time is the time when the sampling of the previous frame of point cloud data begins.

[0037] The two frames of motion data whose sampling moments are adjacent to the start sampling moment, that is, the previous frame of motion data whose sampling moment is adjacent to the start sampling moment, and the next frame of motion data whose sampling moment is adjacent to the sampling moment.

[0038] The initial global pose is the pose of the robot in the global coordinate system at the start of sampling. The global coordinate system is determined according to the pose of the mobile sensor when it is turned on this time. That is, the coordinate of the mobile sensor this time is set to the zero coordinate, and the global coordinate system is set with this zero coordinate as the zero point.

[0039] See also Figure 4 , Figure 4 This is a schematic diagram of the process of determining the initial global posture of the robot according to an embodiment of the present application. This embodiment may specifically include the following steps:

[0040] Step S211: According to the linear change law of the robot posture between the sampling moments of the two adjacent frames of movement data, determine the first position change of the robot from the sampling moment of the previous adjacent frame of movement data to the start sampling moment.

[0041] This step determines the first pose change amount according to the linear law of the robot's pose change within the acquisition interval from the previous adjacent frame movement data to the next adjacent frame movement data.

[0042] Specifically, we can first determine the ratio of the interval between the sampling time of the previous adjacent frame movement data and the start sampling time to the interval between the two adjacent frames of movement data; then, the product of the robot's second posture change between the sampling time of the previous adjacent frame movement data and the next adjacent frame movement data and this ratio is used as the first posture change. The first posture change can be calculated according to the following formula:

[0043] K1=(LDSm_timestamp-L m 0_timestamp) / (L m 1_timestamp-

[0044] L m 0_timestamp)

[0045] ΔL m 1_Pose.x=K1*(L m 1_Pose.xL m 0_Pose.x)

[0046] ΔL m 1_Pose.y=K1*(L m 1_Pose.yL m 0_Pose.y)

[0047] ΔL m 1_Pose.th=K1*(L m 1_Pose.th-L m 0_Pose.th)

[0048] Among them, LDSm_timestamp indicates the sampling start time, L m 0_timestamp represents the sampling time of the previous adjacent frame motion data, L m 1_timestamp indicates the sampling time of the next adjacent frame of mobile data, L m 0_Pose.x, L m 0_Pose.y, L m 0_Pose.th represents the components of the robot's posture in the x, y, and θ directions at the sampling moment of the previous adjacent frame movement data, L m 1_Pose.x, L m 1_Pose.y, L m 1_Pose.th represents the components of the robot's posture in the x, y, and θ directions at the moment of sampling the next adjacent frame of motion data.

[0049] Step S212: The robot posture corresponding to the previous adjacent frame movement data plus the first posture change amount is used as the initial global posture.

[0050] Right now:

[0051] LDSm_0_basePose.x = L m 0_Pose.x+ΔL m 1_Pose.x

[0052] LDSm_0_basePose.y=L m 0_Pose.y+ΔL m 1_Pose.y

[0053] LDSm_0_basePose.th=L m 0_Pose.th+ΔL m 1_Pose.th

[0054] Among them, LDSm_0_basePose.x, LDSm_0_basePose.y, and LDSm_0_basePose.th represent the components of the initial global pose in the x, y, and θ directions, respectively, and K1 represents the intermediate coefficient.

[0055] Step S122: converting the position of the sampling point to be corrected in the previous frame of point cloud data into the local coordinate position of the second robot.

[0056] The sampling points of the directly acquired point cloud data are located in polar coordinates, containing angle and distance information. This step uses each sampling point in the previous frame of point cloud data as the sampling point to be corrected, converting the sampling point to be corrected from the LiDAR data format to the robot's local coordinate system.

[0057] Specifically, the position of the sampling point to be corrected can be converted into the second rectangular coordinate position by the following formula:

[0058] Point_i_raw.x=rangei*cos(anglei)

[0059] Point_i_raw.y=rangei*sin(anglei)

[0060] Then, the second rectangular coordinate position is converted into the local coordinate position of the second robot through the external parameters between the lidar sensor and the robot, which can be specifically expressed as follows:

[0061] Point_i_L m .x=Point_i_raw.x*cos(LDSth)-

[0062] Point_i_raw.y*sin(LDSth)+LDSx

[0063] Point_i_L m .y=Point_i_raw.x*sin(LDSth)+Point_i_raw.y*cos(LDSth)+L

[0064] DSy

[0065] Among them, Point_i_raw.x and Point_i_raw.y are the components of the second rectangular coordinate position in the rectangular coordinate system x and y axis respectively, rangei is the distance value of the sampling point position to be corrected, anglei is the angle value of the sampling point position to be corrected, Point_i_L m .x、Point_i_L m .y is the component of the local coordinate position of the second robot in the x and y axes of the robot local coordinate system, LDSx, LDSy, and LDSth are the components of the external parameters in x, y, and θ, respectively.

[0066] S123: Convert the second robot local coordinate position into a second global coordinate position.

[0067] See also Figure 5 , Figure 5 This is a flowchart of an embodiment of step S122 of the present application. It should be noted that if there is substantially the same result, this embodiment does not Figure 5 The process sequence shown is limited. This embodiment includes the following steps:

[0068] Step S231: Determine the intermediate posture of the robot at the sampling moment of the sampling point to be corrected by using the two frames of movement data adjacent to the sampling point to be corrected in the movement data, the number of sampling points in the previous frame of point cloud data, the sampling order of the sampling point to be corrected in the previous frame of point cloud data, the starting sampling time and the total sampling time of the previous frame of point cloud data.

[0069] The two frames of motion data adjacent to the sampling point to be corrected refer to the previous frame of motion data whose sampling time is adjacent to the sampling time of the sampling point to be corrected, and the next frame of motion data whose sampling time is adjacent to the sampling time of the sampling point to be corrected.

[0070] Similarly, this step can also determine the posture change from the sampling moment of the previous adjacent frame movement data to the sampling moment of the sampling point to be corrected based on the linear law of the robot posture change within the collection interval of the previous adjacent frame movement data and the next adjacent frame movement data. The intermediate posture can be obtained by adding the robot posture corresponding to the previous adjacent frame movement data and the posture change.

[0071] Specifically, see Figure 6 , Figure 6 This is a flowchart of an embodiment of step S231 of the present application. It should be noted that if there is substantially the same result, this embodiment does not Figure 6 The process sequence shown is limited. This embodiment includes the following steps:

[0072] Step S2311: Determine the sampling time of the sampling point to be corrected according to the sampling start time, the sampling order of the sampling point to be corrected in the previous frame of point cloud data, the total sampling time of the previous frame of point cloud data, and the number of sampling points in the previous frame of point cloud data.

[0073] In this embodiment, the previous frame of point cloud data is represented as the m-th frame of point cloud data LDSm. The sampling point to be corrected in this frame of point cloud data is the i-th sampling point LDSm_i. The sampling time of the sampling point to be corrected is calculated according to the following formula:

[0074] LDSm_i_timestamp=LDSm_timestamp+i*totalSampleTime / totalPointSize

[0075] Among them, LDSm_i_timestamp is the sampling time of the sampling point to be corrected in the m-th frame point cloud data, LDSm_timestamp is the start sampling time of the m-th frame point cloud data, i is the sampling order of the point to be corrected LDSm_i in the previous frame point cloud data LDSm, totalSampleTime is the total sampling time of the previous frame point cloud data, and totalPointSize is the number of sampling points in the previous frame point cloud data.

[0076] Step S2312: According to the linear change law of the robot posture between the sampling moments of the two frames of movement data adjacent to the sampling point to be corrected, determine the third posture change of the robot from the sampling moment of the previous adjacent frame movement data to the sampling moment of the sampling point to be corrected.

[0077] The third pose change can be calculated according to the following formula:

[0078] K2=(LDSm_i_timestamp-L m j_timestamp) / (L m j+1_timestamp-

[0079] L m j_timestamp)

[0080] ΔPose.x=K2*(L m j+1_Pose.xL m j_Pose.x)

[0081] ΔPose.y=K2*(L m j+1_Pose.yL m j_Pose.y)

[0082] ΔPose.th=K2*(L mj+1_Pose.th-L m j_Pose.th)

[0083] Among them, K2 represents the intermediate coefficient, L m j_timestamp represents the sampling time of the previous adjacent frame movement data when the i-th point in the m-th frame point cloud is collected. m j+1_timestamp represents the sampling time of the motion data of the next adjacent frame, ΔPose.x, ΔPose.y, and ΔPose.th represent the components of the third pose change in the x, y, and θ directions, respectively.

[0084] Step S2313: The robot's posture at the moment of sampling the previous adjacent frame movement data plus the third posture change amount is used as the intermediate posture.

[0085] The intermediate posture can be calculated by the following formula:

[0086] LDSm_i_basePose.x = L m j_Pose.x+ΔPose.x

[0087] LDSm_i_basePose.y = L m j_Pose.y+ΔPose.y

[0088] LDSm_i_basePose.th=L m j_Pose.th+ΔPose.th

[0089] Among them, LDSm_i_basePose.x, LDSm_i_basePose.y, and LDSm_i_basePose.th represent the components of the intermediate pose in the x, y, and θ directions, respectively.

[0090] Step S232: The product of the angle cosine value of the intermediate posture and the horizontal coordinate of the second robot local coordinate position is subtracted from the product of the angle sine value of the intermediate posture and the vertical coordinate of the second robot local coordinate position, and the result is added to the horizontal coordinate of the intermediate posture as the horizontal coordinate of the second global coordinate position.

[0091] Step S233: The product of the sine value of the angle of the intermediate posture and the horizontal coordinate of the second robot local coordinate position, the product of the cosine value of the angle of the intermediate posture and the vertical coordinate of the second robot local coordinate position, and the vertical coordinate of the intermediate posture are used as the vertical coordinate of the second global coordinate position.

[0092] The horizontal coordinate of the second global coordinate position can be calculated by the following formula:

[0093] LDSm_i_pointPose.x=Point_i_L m .x*cos(LDSm_i_basePose.th)-

[0094] Point_i_L m .y*sin(LDSm_i_basePose.th)+LDSm_i_basePose.x

[0095] The ordinate of the second global coordinate position can be calculated by the following formula:

[0096] LDSm_i_pointPose.y=Point_i_L m .x*sin(LDSm_i_basePose.th)+Point_i

[0097] _L m .y*cos(LDSm_i_basePose.th)+LDSm_i_basePose.y

[0098] Among them, LDSm_i_pointPose.x and LDSm_i_pointPose.y represent the horizontal coordinate and vertical coordinate of the second global coordinate position respectively.

[0099] The above embodiment uses the movement data adjacent to the sampling time of the sampling point to be corrected to correct the position of the sampling point in the global coordinate system, so as to reduce the posture deviation caused by the movement of the robot during the sampling period.

[0100] Step S124: taking the square root of the difference between the ordinate of the second global coordinate position and the ordinate of the initial global pose, and the square root of the difference between the abscissa of the second global coordinate position and the abscissa of the initial global pose as the distance information of the corrected sampling point.

[0101] Step S125: The inverse tangent azimuth between the difference between the ordinate of the second global coordinate position and the ordinate of the initial global pose and the difference between the abscissa of the second global coordinate position and the abscissa of the initial global pose is subtracted from the angle value of the initial global pose as the angle information of the corrected sampling point.

[0102] Steps S124 to S125 convert the position of the sampling point to be corrected from the global coordinate system to the point cloud data format represented by the polar coordinate system. Specifically, the following formula can be used to convert the global coordinate position to the polar coordinate position:

[0103] calibration_rangei = sqrt(LDSm_i_pointPose.y - LDSm_0_basePose.y, LDSm_i_pointPose.x - LDSm_0_basePose.x)

[0104] calibration_anglei = atan2(LDSm_i_pointPose.y - LDSm_0_basePose.y, LDSm_i_pointPose.x - LDSm_0_basePose.x) - LDSm_0_basePose.th

[0105] Among them, calibration_rangei and calibration_anglei respectively represent the distance information and angle information in the polar coordinate position of the sampled points after calibration. sqrt represents the square root function, and atan2 represents the arctangent azimuth function.

[0106] This embodiment uses the motion information of the mobile robot reflected by the mobile data when collecting a frame of point cloud data to correct the positions of each sampled point. All the point cloud information takes into account the motion state of the robot body, and reflects the environmental information more accurately.

[0107] Step S13: Generate supplementary point cloud data corresponding to the current frame of mobile data according to the global coordinate positions of each sampled point in the corrected point cloud data and the current frame of mobile data of the robot.

[0108] Since the acquisition frame rate of the mobile data is higher than that of the point cloud data, the mobile data is continuously updated, and the update frequency of the point cloud data is lower than that of the mobile data. The purpose of this step is to infer the point cloud distribution of the current pose through the current pose of the robot reflected by the mobile data and the corrected point cloud data, as supplementary data for the lidar sensor, in order to Figure 2 For example, assume that the point cloud data LDSm has completed point cloud correction, and the newly arrived mobile data L m+1 0 to L m+1 k. When the current frame of point cloud data LDSm+1 is being collected but not completed, the point cloud data after the correction of LDSm is the point cloud distribution at the pose of the robot at the start sampling moment of this frame. It is not accurate enough to directly use it as the point cloud distribution of the current pose of the robot. At this time, the current pose of the robot has been updated to L m+1 j (j < n). At this time, the more accurate point cloud information at the current pose L m+1 j can be inferred by using the mobile data L m+1 j and the corrected point cloud of the previous frame of point cloud data LDSm. For details, please refer to the following embodiments.

[0109] Optionally, please refer to Figure 7 The global coordinate position of the corrected sampling point can be converted through steps S31 to S33 to obtain:

[0110] Step S31: Convert the laser radar coordinate position of the sampling point into the first rectangular coordinate position.

[0111] The following formula is used to convert the laser radar coordinate position of the sampling point in the corrected point cloud data into the rectangular coordinate position:

[0112] Point_i_calibration.x=calibration_rangei*cos(calibration_anglei)

[0113] Point_i_calibration.y=calibration_rangei*sin(calibration_anglei)

[0114] Among them, Point_i_calibration.x and Point_i_calibration.y represent the components of the first rectangular coordinate position of the sampling point Point_i_calibration of the point cloud data after correction in the x and y directions respectively.

[0115] Step S32: using the external parameters between the laser radar sensor and the robot, convert the first rectangular coordinate position into the first robot local coordinate position.

[0116] Point_i_L m+1 .x=Point_i_calibration.x*cos(LDSth)-

[0117] Point_i_calibration.y*sin(LDSth)+LDSx

[0118] Point_i_L m+1 .y=Point_i_calibration.x*sin(LDSth)+Point_i_calibration.y

[0119] *cos(LDSth)+LDSy

[0120] Among them, Point_i_L m+1 .x、Point_i_L m+1 .y is the x- and y-components of the first robot local coordinate position of the sampling point of the corrected point cloud data.

[0121] Step S33: Convert the first robot local coordinate position into a first global coordinate position to obtain the global coordinate position of the sampling point.

[0122] Optionally, before step S33, the initial global pose of the robot at the moment when the sampling of the previous frame of point cloud data starts can be determined according to the method of steps S211 to S212.

[0123] Then step S33 can determine the first global coordinate position of the corrected sampling point according to the following method: the product of the angle cosine value of the initial global pose and the horizontal coordinate of the first robot local coordinate position, minus the product of the angle sine value of the initial global pose and the vertical coordinate of the first robot local coordinate position, plus the horizontal coordinate of the initial global pose, as the horizontal coordinate of the first global coordinate position; the product of the angle sine value of the initial global pose and the horizontal coordinate of the first robot local coordinate position, plus the product of the angle cosine value of the initial global pose and the vertical coordinate of the first robot local coordinate position, plus the vertical coordinate of the initial global pose, as the vertical coordinate of the first global coordinate position.

[0124] That is, the first global coordinate position is determined according to the following formula:

[0125] LDSm_i_pointPose.x=Point_i_L m+1 .x*cos(LDSm_0_basePose.th)-

[0126] Point_i_L m+1 .y*sin(LDSm_0_basePose.th)+LDSm_0_basePose.x

[0127] LDSm_i_pointPose.y=Point_i_L m+1 .x*sin(LDSm_0_basePose.th)+Poin

[0128] t_i_L m+1 .y*cos(LDSm_0_basePose.th)+LDSm_0_basePose.y

[0129] Among them, LDSm_0_basePose.x, LDSm_0_basePose.y, LDSm_0_basePose.th represent the components of the initial global pose in the x, y, and θ directions respectively, Point_i_L m+1 .x、Point_i_L m+1.y represents the horizontal and vertical components of the local coordinate position of the first robot, respectively. LDSm_i_pointPose.x and LDSm_i_pointPose.y are the components of the global coordinate position of the sampling point of the corrected point cloud data in the x and y directions, respectively.

[0130] Step S33 can specifically generate supplementary point cloud data in the following manner: the square root of the difference between the ordinate of the global coordinate position and the ordinate of the current pose, and the difference between the abscissa of the global coordinate position and the abscissa of the current pose is used as the distance information in the supplementary point cloud data; the azimuth between the difference between the ordinate of the global coordinate position and the ordinate of the current pose, and the difference between the abscissa of the global coordinate position and the abscissa of the current pose, minus the angular coordinate of the current pose, is used as the angle information in the supplementary point cloud data. Specifically, the position information of the supplementary sampling point can be calculated using the following formula:

[0131] prediction_rangei=sqrt(LDSm_i_pointPose.yL m+1 jPose.y,

[0132] LDSm_i_pointPose.xL m+1 jPose.x)

[0133] prediction_anglei=atan2(LDSm_i_pointPose.yL m+1 jPose.y,

[0134] LDSm_i_pointPose.xL m+1 jPose.x)-L m+1 jPose.th

[0135] Among them, prediction_rangei and prediction_anglei respectively represent the distance component and angle component of the polar coordinates of the supplementary sampling point corresponding to the corrected point cloud data sampling point Point_i_calibration, L m+1 jPose.x, L m+1 jPose.y、L m+1 jPose.th represents the components of the current pose in the x, y, and θ directions respectively.

[0136] The current pose is determined directly from the pose information in the current frame motion data.

[0137] The point cloud is supplemented for all sampling points in the previous frame, generating supplementary points corresponding to each sampling point. This allows for a reasonable prediction of the point cloud data during movement based on the robot's posture changes, fully utilizing the previous point cloud data. The resulting supplementary point cloud data is accurate and reasonable.

[0138] Step S14: The next frame of movement data is used as the current frame of movement data, the obtained current frame of point cloud data is used as the previous frame of point cloud data, and the process returns to step S12.

[0139] The current frame movement data refers to the movement data from the completion of the previous frame point cloud data acquisition to the completion of the current frame point cloud data acquisition; the next frame movement data refers to the movement data from the completion of the current frame point cloud data acquisition to the completion of the next frame point cloud data acquisition. Figure 2 For example, after the current frame point cloud data LDSm+1 is collected, the current frame point cloud data LDSm+1 is used as the previous frame point cloud data, and the movement data collected at the next sampling moment is used as the current movement data, and the point cloud correction and supplement operations are continued according to the method of steps S12 to S13.

[0140] In this way, starting from the second frame of point cloud data, a corresponding frame of point cloud data can be supplemented for each frame of motion data, so that the frame rate of the point cloud data is increased to the same level as the acquisition frame rate of the motion data.

[0141] The point cloud correction and point cloud supplement operations are stopped until no new motion data or point cloud data is detected, or a stop processing command is received. Those skilled in the art may perform specific subsequent operations based on the point cloud data obtained through the above processing as needed, such as target recognition or target tracking operations, and the above description is not intended to be limiting.

[0142] Different from the existing technology, this embodiment uses the robot motion information in the mobile data to correct the point cloud data. The corrected point cloud data is combined with the corresponding mobile data to generate supplementary point cloud data. The point cloud supplementation within the time sampling interval improves the accuracy and frame rate of the point cloud information. The supplemented point cloud data can be used subsequently for target recognition or target tracking and other operations, thereby improving the availability of the point cloud data.

[0143] See also Figure 8 , Figure 8This is a schematic block diagram of the structure of an embodiment of the point cloud data processing device of the present application. The point cloud data processing device 100 includes an acquisition module 110, a point cloud correction module 120, and a supplementary generation module 130. The point cloud correction module 120 is connected to the acquisition module 110, and the supplementary generation module 130 is connected to the point cloud correction module 120. The acquisition module 110 is used to acquire point cloud data and robot movement data; the point cloud correction module 120 uses the movement data to perform a correction operation on the previous frame of point cloud data to obtain corrected point cloud data; the supplementary generation module 130 generates supplementary point cloud data corresponding to the current frame movement data based on the global coordinate position of each sampling point in the corrected point cloud data and the current frame movement data of the robot.

[0144] For the specific methods of each processing module regarding each step of processing execution, please refer to the description of each step of the embodiment of the point cloud data processing method of the present application above, which will not be repeated here.

[0145] The point cloud data processing device 100 may also include a laser radar sensor and a motion sensor, neither of which is shown in the figure. The laser radar sensor is used to collect data about the surrounding environment of the point cloud data processing device 100, and the motion sensor is used to collect motion data of the point cloud data processing device 100 itself. The laser radar sensor has a lower acquisition frame rate than the motion sensor. The outputs of the laser radar sensor and the motion sensor are connected to the inputs of the acquisition module 110, so that the collected point cloud data and motion data can be uploaded to the acquisition module 110 for data processing.

[0146] See also Figure 9 , Figure 9 2 is a schematic block diagram of another embodiment of a point cloud data processing device of the present application. The point cloud data processing device 200 includes a processor 210 and a memory 220 coupled to each other. The memory 220 stores a computer program, and the processor 210 is configured to execute the computer program to implement the point cloud data processing method described in the above embodiments.

[0147] For the description of each step of the processing execution, please refer to the description of each step of the embodiment of the point cloud data processing method of the present application above, and no further details will be given here.

[0148] The memory 220 can be used to store program data and modules. The processor 210 executes various functional applications and data processing by running the program data and modules stored in the memory 220. The memory 220 may mainly include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a point cloud data correction function or a point cloud data supplementation function). The data storage area may store data created based on the use of the point cloud data processing device 200 (such as corrected point cloud data, external reference data, supplemented point cloud data, etc.). In addition, the memory 220 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 220 may also include a memory controller to provide the processor 210 with access to the memory 220.

[0149] In the various embodiments of the present application, the disclosed methods and devices can be implemented in other ways. For example, the various embodiments of the point cloud data processing device 200 described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0150] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of this embodiment.

[0151] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0152] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, or the part that contributes to the existing technology, or all or part of the technical solution, can be embodied in the form of a software product, and the computer software product can be stored in a storage medium.

[0153] See Figure 10 , Figure 10 This is a schematic structural block diagram of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 300 stores program data 310. When the program data 310 is executed, the steps of each embodiment of the point cloud data processing method described above are implemented.

[0154] For the description of each step of the processing execution, please refer to the description of each step of the embodiment of the point cloud data processing method of the present application above, and no further details will be given here.

[0155] The computer-readable storage medium 300 may be any medium that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0156] The above description is merely an embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A point cloud data processing method, applied to a mobile robot, characterized in that: The method comprises: Acquire point cloud data and robot movement data; wherein the sampling frame rate of the point cloud data is lower than the sampling frame rate of the movement data; Performing a correction operation on the previous frame of point cloud data using the movement data to obtain corrected point cloud data; Generate supplementary point cloud data corresponding to the current frame movement data according to the global coordinate position of each sampling point in the corrected point cloud data and the current frame movement data of the robot, including: using the square root of the difference between the ordinate of the global coordinate position and the ordinate of the current posture and the difference between the abscissa of the global coordinate position and the abscissa of the current posture as distance information in the supplementary point cloud data; the current posture is determined according to the current frame movement data; using the arc tangent azimuth between the difference between the ordinate of the global coordinate position and the ordinate of the current posture and the difference between the abscissa of the global coordinate position and the abscissa of the current posture, minus the angular coordinate of the current posture, as the angle information in the supplementary point cloud data; The next frame of movement data is used as the current frame of movement data, the obtained current frame of point cloud data is used as the previous frame of point cloud data, and the correction operation and subsequent supplementary steps of the previous frame of point cloud data using the movement data are returned.

2. The method according to claim 1, characterized in that The point cloud data is collected by a laser radar sensor provided on the robot, and the movement data is collected by a movement sensor provided on the robot; After performing a correction operation on the previous frame of point cloud data using the movement data to obtain corrected point cloud data, the method further includes: Converting the laser radar coordinate position of the sampling point into a first rectangular coordinate position; Using external parameters between the laser radar sensor and the robot, converting the first rectangular coordinate position into a local coordinate position of the first robot; The local coordinate position of the first robot is converted into a first global coordinate position to obtain the global coordinate position of the sampling point.

3. The method according to claim 2, characterized in that Before converting the first robot local coordinate position into a first global coordinate position, the method further includes: Determine the initial global pose of the robot at the start sampling moment according to the start sampling moment of the previous frame of point cloud data and the two frames of movement data before and after the start sampling moment; The converting the first robot local coordinate position into a first global coordinate position comprises: The product of the sine value of the angle of the initial global pose and the ordinate of the first robot local coordinate position is subtracted from the product of the cosine value of the angle of the initial global pose and the abscissa value of the first robot local coordinate position, and the result is added to the abscissa value of the initial global pose, and the result is used as the abscissa value of the first global coordinate position; The ordinate of the first global coordinate position is obtained by multiplying the sine of the angle of the initial global pose by the horizontal coordinate of the first robot local coordinate position, adding the cosine of the angle of the initial global pose by the vertical coordinate of the first robot local coordinate position, and adding the vertical coordinate of the initial global pose.

4. The method according to claim 3, characterized in that Determining the initial global pose of the robot at the start sampling moment based on the start sampling moment of the previous frame of point cloud data and the two frames of movement data before and after the start sampling moment, including: Determine the first position change of the robot from the sampling moment of the previous adjacent frame of motion data to the starting sampling moment according to the linear change law of the robot's position and posture between the sampling moments of the two adjacent frames of motion data; The robot posture corresponding to the movement data of the previous adjacent frame plus the first posture change amount is used as the initial global posture.

5. The method according to claim 4, characterized in that The determining, based on the linear change law of the robot posture between the sampling moments of the two adjacent frames of movement data, the first posture change amount of the robot from the sampling moment of the previous adjacent frame of movement data to the starting sampling moment, comprises: Determine the ratio of the interval time between the sampling moment of the previous adjacent frame motion data and the starting sampling moment to the interval time corresponding to the two adjacent frames of motion data; The product of the second posture change of the robot between the sampling moments of the previous adjacent frame movement data and the next adjacent frame movement data and the ratio is used as the first posture change.

6. The method according to claim 1, characterized in that Performing a correction operation on the previous frame of point cloud data using the movement data to obtain corrected point cloud data, including: Determine the initial global pose of the robot at the start sampling moment according to the start sampling moment of the previous frame of point cloud data and the two frames of movement data before and after the start sampling moment; Converting the position of the sampling point to be corrected in the previous frame of point cloud data into the local coordinate position of the second robot; Converting the second robot's local coordinate position into a second global coordinate position; Taking the square root of the difference between the ordinate of the second global coordinate position and the ordinate of the initial global pose and the difference between the abscissa of the second global coordinate position and the abscissa of the initial global pose as the distance information of the corrected sampling point; The inverse tangent azimuth between the difference between the ordinate of the second global coordinate position and the ordinate of the initial global pose and the difference between the abscissa of the second global coordinate position and the abscissa of the initial global pose, minus the angle value of the initial global pose, is used as the angle information of the corrected sampling point.

7. The method according to claim 6, characterized in that The converting the second robot local coordinate position into a second global coordinate position comprises: Determine the intermediate posture of the robot at the sampling moment of the sampling point to be corrected by using the two frames of movement data before and after the sampling point to be corrected in the movement data, the number of sampling points in the previous frame of point cloud data, the sampling order of the sampling point to be corrected in the previous frame of point cloud data, the starting sampling moment and the total sampling time of the previous frame of point cloud data; The product of the sine value of the angle of the intermediate posture and the ordinate of the second robot local coordinate position is subtracted from the product of the cosine value of the angle of the intermediate posture and the abscissa value of the second robot local coordinate position, and the result is added to the abscissa value of the intermediate posture as the abscissa value of the second global coordinate position; The ordinate of the second global coordinate position is obtained by multiplying the sine value of the angle of the intermediate posture by the horizontal coordinate of the second robot local coordinate position, adding the cosine value of the angle of the intermediate posture by the vertical coordinate of the second robot local coordinate position, and adding the vertical coordinate of the intermediate posture.

8. The method according to claim 7, characterized in that The determining of the intermediate posture of the robot at the sampling moment of the sampling point to be corrected by using the two frames of movement data before and after the sampling point to be corrected in the movement data, the number of sampling points in the previous frame of point cloud data, the sampling order of the sampling point to be corrected in the previous frame of point cloud data, the starting sampling moment and the total sampling time of the previous frame of point cloud data, includes: Determining the sampling time of the sampling point to be corrected according to the sampling start time, the sampling order of the sampling point to be corrected in the previous frame of point cloud data, the total sampling time of the previous frame of point cloud data, and the number of sampling points in the previous frame of point cloud data; Determine a third posture change of the robot from the sampling moment of the previous adjacent frame of movement data to the sampling moment of the sampling point to be corrected based on a linear change law of the robot posture between the sampling moments of the two frames of movement data adjacent to the sampling point to be corrected; The intermediate posture is obtained by adding the third posture change amount to the posture of the robot at the sampling moment of the movement data of the previous adjacent frame.

9. A point cloud data processing device, characterized in that: The device includes a processor and a memory coupled to each other; a computer program is stored in the memory, and the processor is configured to execute the computer program to implement the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program data, and when the program data is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images