A method, device, robot and storage medium for heave correction

Abstract

The present invention discloses a method, device, robot and storage medium for heave correction, wherein the method comprises: when the robot is heaving, obtaining point cloud data detected by a binocular sensor; judging whether the point cloud data satisfies a first preset condition; if so, judging whether the point cloud data satisfies a second preset condition; if so, judging whether the point cloud data satisfies a third preset condition; if so, determining the heave parameters of the robot according to the point cloud data, and correcting the point cloud data according to the heave parameters. The present invention can reduce the impact of heave on obstacle detection and improve detection accuracy by obtaining and using point cloud data to correct the heave of the robot.

Description

Technical Field

[0001] The present invention relates to the field of robot technology, and in particular to a heave correction method, device, robot and storage medium. Background Art

[0002] In recent years, due to the increase in income levels and changes in consumption concepts, people are willing to spend more money in exchange for a better experience of using smart home products. Therefore, the concept of smart home, which aims to provide intelligent services, has been widely recognized as soon as it was proposed. At the same time, a large number of smart home products that meet the concept of smart home have sprung up on the market, prompting more and more people to replace traditional home appliances with smart home products in order to enjoy the convenience of life brought by smart home products.

[0003] Among the many smart home products, cleaning robots are particularly popular among the working class. Cleaning robots, also known as sweeping robots or smart sweepers, can automatically complete a series of cleaning tasks such as sweeping, vacuuming, and mopping the floor in the room with certain artificial intelligence technology. At present, although most of the existing cleaning robots on the market have replaced the monocular sensors in the technical solution with binocular sensors in order to achieve the purpose of improving obstacle detection capabilities, this approach still encounters some problems in practical applications. For example, in the process of detecting obstacles, the movement of the cleaning robot will be affected by the rugged ground, thresholds, etc., causing ups and downs, resulting in errors in the height of the detected obstacles, which may cause collisions with obstacles due to misjudgment during the movement, or cause errors in the grid content during the process of establishing or updating the grid map, or even the grid that should not be occupied is occupied.

[0004] Therefore, how to reduce the impact of the cleaning robot's ups and downs on obstacle detection or mapping has become one of the technical problems that technical personnel in this field urgently need to solve, and it is of great significance to the development of cleaning robots.

[0005] The above information is presented as background information only to assist with understanding the present disclosure and no determination or admission is made as to whether any of the above may be used as prior art with respect to the present disclosure. Summary of the invention

[0006] The present invention provides a method and device for correcting fluctuations, a robot and a storage medium, so as to correct the collected point cloud data when the robot fluctuates, thereby reducing or avoiding the influence of the robot fluctuations on the point cloud data.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A first aspect of an embodiment of the present invention discloses a heave correction method; the method is applied to a robot, wherein the robot includes a binocular sensor and an inertial measurement unit.

[0009] As an optional implementation manner, in a first aspect of the embodiment of the present invention, the method includes:

[0010] When the robot is up and down, acquiring point cloud data detected by the binocular sensor;

[0011] Determining whether the point cloud data meets a first preset condition;

[0012] If so, determining whether the point cloud data satisfies a second preset condition;

[0013] If so, determining whether the point cloud data satisfies a third preset condition;

[0014] If yes, determine the ups and downs parameters of the robot according to the point cloud data, and correct the point cloud data according to the ups and downs parameters.

[0015] A second aspect of an embodiment of the present invention discloses a undulation correction device; applied to a robot, the robot comprising a binocular sensor and an inertial measurement unit,

[0016] As an optional implementation manner, in a second aspect of the embodiment of the present invention, the device includes:

[0017] A data acquisition module, used for acquiring point cloud data detected by the binocular sensor when the robot is in ups and downs;

[0018] An undulation correction module, used for determining whether the point cloud data meets a first preset condition;

[0019] If so, determining whether the point cloud data satisfies a second preset condition;

[0020] If so, determining whether the point cloud data satisfies a third preset condition;

[0021] If yes, determine the ups and downs parameters of the robot according to the point cloud data, and correct the point cloud data according to the ups and downs parameters.

[0022] A third aspect of an embodiment of the present invention discloses a robot;

[0023] As an optional implementation, in a third aspect of the embodiments of the present invention, the robot includes a memory and a processor, the memory stores a computer program, and the processor implements the undulation correction method as described in any of the above aspects when executing the computer program.

[0024] A fourth aspect of an embodiment of the present invention discloses a storage medium containing computer executable instructions;

[0025] As an optional implementation, in a fourth aspect of the embodiments of the present invention, the computer executable instructions are executed by a computer processor to implement the undulation correction method as described in any of the above aspects.

[0026] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0027] The embodiments of the present invention provide a method, device, robot and storage medium for heave correction, which performs heave correction on the robot by acquiring and using point cloud data detected by a binocular sensor. Specifically, when the robot heaves, the point cloud data of the current frame of the robot is corrected to obtain corrected point cloud data. When obstacle detection is performed, the corrected point cloud data is used for obstacle detection, which can reduce the impact of the robot heaving on obstacle detection and improve the accuracy of obstacle detection. Alternatively, when the robot builds a map, the corrected point cloud data is used for map construction, which can improve the accuracy of the robot's map construction and has a high value for promotion and application. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0029] Figure 1 It is a flow chart of a heave correction method provided in the first embodiment of the present invention;

[0030] Figure 2 is a flow chart of a heave correction method provided in Embodiment 2 of the present invention;

[0031] Figure 3 is a flow chart of a heave correction method provided in Embodiment 3 of the present invention;

[0032] Figure 4 is a flow chart of a heave correction method provided in Embodiment 4 of the present invention;

[0033] Figure 5 It is a functional module diagram of a heave correction device provided by Embodiment 5 of the present invention;

[0034] Figure 6 It is a structural schematic diagram of a robot provided in Example 6 of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] In the description of the present invention, it is to be understood that when a component is considered to be "connected" to another component, it may be directly connected to the other component or there may be a centrally disposed component. When a component is considered to be "disposed on" another component, it may be directly disposed on the other component or there may be a centrally disposed component.

[0037] In addition, terms such as "long", "short", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings and are only used to facilitate the description of the present invention. They do not indicate or imply that the device or component referred to must have this specific direction or operate with a specific direction structure, and should not be understood as a limitation of the present invention.

[0038] The technical solution of the present invention is further described below with reference to the accompanying drawings and through specific implementation methods.

[0039] Embodiment 1

[0040] As mentioned in the background technology, if the robot (mainly binocular robot) is up and down during the movement, there will be errors in the obstacle height information in the collected image. The main reason is that in the process of obstacle detection or global grid mapping in the binocular algorithm, there is a step to project the 3D point cloud into a 2.5D map. When performing this projection, we directly project it along the Z axis to the XOY plane. The assumption for this projection is that the XOY plane is the ground, that is, the robot is horizontal, because the 3D point cloud is in the robot coordinate system. However, in actual situations, the movement of the robot is bumpy and undulating. In this case, if the 3D point cloud is directly projected to the XOY plane, if the robot is performing obstacle detection on the way to perform the task, then the height of the obstacle detected will be wrong. If the robot is mapping the global grid, then the grid content will be wrong. Of course, it is mainly due to errors in height information, and in the case of large angles, grids that should not be occupied will be occupied.

[0041] In view of the defects of the existing technology, the inventor, based on his rich practical experience and professional knowledge in the industry for many years, and in conjunction with the application of theory, actively conducts research and innovation in order to create a practical obstacle detection technology to make it more practical. After continuous research, design, repeated trials and improvements, the present invention with real practical value was finally created.

[0042] See also Figure 1 , Figure 1 The present invention discloses a flow chart of a method for heave correction, which is applicable to the scene of obstacle detection when heave occurs during the robot's movement. The method is performed by a heave correction device, which can be implemented by software and / or hardware and integrated into the robot. The robot includes a binocular sensor and an inertial measurement unit. Figure 1 As shown, the fluctuation correction method may include the following steps:

[0043] S101. When the robot is up and down, obtain point cloud data detected by the binocular sensor.

[0044] It should be noted that the specific type of the robot in this embodiment is not limited, for example, it can be a sweeping robot, a food delivery robot, a welcoming robot, etc., as long as it is equipped with a binocular sensor and an inertial measurement unit. During the movement of the robot, the point cloud data of obstacles in the direction of movement can be obtained by using the binocular sensor of the robot.

[0045] S102, determining whether the point cloud data satisfies a first preset condition; if not, executing step S103; if so, executing step S104.

[0046] It should be noted that this step is to confirm whether the amount of the acquired point cloud data is sufficient. Then, according to the two results of whether it is sufficient or not, this embodiment will subsequently adopt different undulation correction strategies to correct the point cloud data.

[0047] On the one hand, optionally, in this embodiment, the collected point cloud data may be directly compared with a preset first preset threshold, that is, the step S102 may further include the following steps:

[0048] It is determined whether the amount of the point cloud data is greater than or equal to a first preset threshold.

[0049] The first preset threshold value may be arbitrarily set by technicians based on experience and actual application scenarios, and is not specifically limited in this embodiment.

[0050] On the other hand, optionally, in this embodiment, before step S102, the method further includes:

[0051] Downsampling the point cloud data by voxels;

[0052] Accordingly, the step S102 may further include the following steps:

[0053] It is determined whether the amount of the point cloud data after voxel downsampling is greater than or equal to a second preset threshold.

[0054] Among them, the second preset threshold can be arbitrarily set by technical personnel based on experience and actual application scenarios, and this embodiment does not make specific limitations here. The first preset threshold is greater than the second preset threshold.

[0055] It should be noted that the reason why the point cloud data is downsampled in this embodiment is that, generally, there are relatively large amounts of point cloud data, and the specific number may be as high as 30,000 to 40,000. In order to reduce CPU consumption and improve computing efficiency, this embodiment needs to downsample the point cloud data to reduce the amount of point cloud data.

[0056] If the number of point cloud data that has not been downsampled is greater than or equal to the first preset threshold, or the number of point cloud data after downsampling is greater than or equal to the second preset threshold, it means that the acquired point cloud data is sufficient and the point cloud data can be used to correct the ups and downs of the robot; similarly, if the number of point cloud data that has not been downsampled is less than the first preset threshold, or the number of point cloud data after downsampling is less than the second preset threshold, it means that the acquired point cloud data is too little and it is not suitable to use the point cloud data to correct the ups and downs of the robot.

[0057] S103, acquiring inertial measurement unit data detected by an inertial measurement unit; and correcting the point cloud data according to the inertial measurement unit data.

[0058] It should be noted that when the number of point cloud data without downsampling is less than the first preset threshold, or the number of point cloud data after downsampling is less than the second preset threshold, it is not suitable to use the point cloud data to correct the ups and downs of the robot. In this case, the inertial measurement unit data detected by the inertial measurement unit can be obtained and used to correct the point cloud data, so that no matter what situation the robot encounters, there is a corresponding ups and down correction strategy to correct the ups and downs, so as to reduce the impact of the robot's ups and downs on obstacle detection or mapping, thereby improving the accuracy of obstacle detection or the accuracy of robot mapping.

[0059] S104, determining whether the point cloud data satisfies a second preset condition; if not, returning to step S103; if so, executing step S105.

[0060] It should be noted that, in this embodiment, after determining that the acquired point cloud data is sufficient, it can be further determined whether the point cloud data meets the second preset condition.

[0061] Optionally, in this embodiment, step S104 may further include the following steps:

[0062] Randomly select three acquisition points from the point cloud data;

[0063] Determine a first plane according to the three acquisition points, wherein the first plane corresponds to a first plane equation;

[0064] Calculate the interior points and root mean square error of the first plane according to the first plane equation;

[0065] determining a first target plane according to the interior points and the root mean square error of the first plane, the first target plane corresponding to the first target plane equation;

[0066] Calculating the interior points and the root mean square error of the first target plane according to the first target plane equation;

[0067] Determine whether the number of interior points of the first target plane is greater than a third preset threshold, and whether the root mean square error of the first target plane is less than a fourth preset threshold; if not, iteratively calculate a new plane equation, interior points, and root mean square error until the number of interior points corresponding to the new plane equation is greater than the third preset threshold and the root mean square error corresponding to the new plane equation is less than the fourth preset threshold, or until the number of iterations reaches a preset number;

[0068] If so, it is determined that the point cloud data meets the second preset condition.

[0069] The preset number of times may be arbitrarily set by a technician based on experience and actual application scenarios, or may be set by system default. This embodiment does not make a unique limitation here, for example, it may be 50 times.

[0070] The third preset threshold and the fourth preset threshold can be arbitrarily set by the technician according to the experience value and the actual application scenario, or can be set by the system default. For example, the third preset threshold can be 0.7, and the fourth preset threshold can be 0.002. This embodiment does not make a unique limitation here.

[0071] It should be noted that this step is to fit a plane through point cloud data, and to determine the relationship between the plane and the horizontal plane through the number of interior points corresponding to the plane equation and the root mean square error corresponding to the plane equation.

[0072] S105, determining whether the point cloud data satisfies a third preset condition; if not, returning to step S103; if so, executing step S106.

[0073] It should be noted that, in this embodiment, after determining that the acquired point cloud data is sufficient and satisfies the second preset condition, it is necessary to further determine whether the point cloud data satisfies the third preset condition.

[0074] Optionally, in this embodiment, step S105 may further include the following steps:

[0075] Calculate a first flip angle, a first pitch angle and a first height according to the point cloud data;

[0076] Determining whether the first flip angle is less than a first angle threshold, whether the first pitch angle is less than a second angle threshold, and whether the first height is less than a first height threshold;

[0077] If so, it is determined that the point cloud data meets the third preset condition.

[0078] Among them, the first angle threshold, the second angle threshold, and the first height threshold can be arbitrarily set by technical personnel based on experience and actual application scenarios. This embodiment does not make a sole limitation here. For example, the first angle threshold and the second angle threshold can be 1.145°×1.5, and the first height threshold can be 0.02.

[0079] S106: Determine the ups and downs parameters of the robot according to the point cloud data, and correct the point cloud data according to the ups and downs parameters.

[0080] Wherein, the undulation parameters include the first flip angle, the first pitch angle and the first height.

[0081] It should be noted that correcting the point cloud data according to the undulation parameters or correcting the point cloud data according to the inertial measurement unit data is for correcting the undulations that occur during the movement of the robot. This solution can execute different undulation correction strategies in different situations.

[0082] An embodiment of the present invention provides a method for correcting undulations. By acquiring and utilizing point cloud data detected by a binocular sensor or inertial measurement unit data detected by an inertial measurement unit, when the robot undulates, the point cloud data of the current frame of the robot is corrected to obtain corrected point cloud data. Thus, when obstacle detection is performed, the corrected point cloud data is used for obstacle detection, which can reduce the impact of the robot's undulations on obstacle detection and improve the accuracy of obstacle detection. Alternatively, when the robot is building a map, the corrected point cloud data is used for building the map, which can improve the accuracy of the robot's mapping and has a high value for promotion and application.

[0083] Embodiment 2

[0084] See also Figure 2 , Figure 2 : is a flowchart of a method for heave correction disclosed in an embodiment of the present invention. Based on the technical solution provided in the first embodiment, this embodiment further optimizes step S101 "obtaining point cloud data detected by the binocular sensor when the robot is heaving". The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here, namely:

[0085] When the robot is up and down, obtain the depth data detected by the binocular sensor;

[0086] generating a depth map according to the depth data;

[0087] Extracting depth data in the depth map according to a preset depth threshold;

[0088] The depth data is projected in the robot coordinate system to obtain point cloud data.

[0089] Based on the above optimization, Figure 2 As shown, a heave correction method provided in this embodiment may specifically include the following steps:

[0090] S201. When the robot is up and down, obtain depth data detected by the binocular sensor.

[0091] S202: Generate a depth map according to the depth data.

[0092] S203: Extract depth data in the depth map according to a preset depth threshold.

[0093] The preset depth threshold may be arbitrarily set by technicians based on experience and actual application scenarios, and is not specifically limited in this embodiment.

[0094] It should be noted that we generally believe that the factors that affect point cloud projection are the pitch angle and the roll angle, that is, the greater the angle of fluctuation, the greater the vertical projection error and height error of the detected obstacle. However, in fact, in addition to the angle, the distance of the obstacle will also cause errors in the vertical projection and height of the obstacle. Specifically, even if the angle of the robot is not large, the vertical projection error and height error of the obstacle will still increase with the increase of the distance. Therefore, considering that the error of obstacle detection at a longer distance is greater, and considering that the robot's obstacle avoidance does not actually need to detect obstacles too far away in advance, in order to save unnecessary resources and time waste, the acquired point cloud data can be filtered to remove the error interference of the point cloud at a long distance.

[0095] Specifically, since the condition for filtering point cloud data is distance, and the depth map (also called the distance image) contains information about the distance from each point of the obstacle to the camera (i.e., the depth value), the point cloud data that is farther away can be filtered through the depth value of the pixel point corresponding to the point cloud data in the depth map.

[0096] S204: Project the depth data in the robot coordinate system to obtain point cloud data.

[0097] It should be noted that since the point cloud data originally obtained by the robot through the binocular sensor is in the world coordinate system, this step can convert the depth data to the robot coordinate system before projecting it. The specific conversion requires the use of the camera's intrinsic and extrinsic parameters. Among them, the intrinsic parameters refer to some projection parameters of the pinhole camera model, and the extrinsic parameters refer to the conversion relationship from the camera coordinate system to the robot coordinate system.

[0098] S205, determining whether the point cloud data satisfies a first preset condition; if not, executing step S206; if so, executing step S207.

[0099] S206, acquiring inertial measurement unit data detected by the inertial measurement unit; and correcting the point cloud data according to the inertial measurement unit data.

[0100] S207, determining whether the point cloud data satisfies a second preset condition; if not, returning to step S206; if so, executing step S208.

[0101] S208, determining whether the point cloud data satisfies a third preset condition; if not, returning to step S206; if so, executing step S209.

[0102] S209: Determine the ups and downs parameters of the robot according to the point cloud data, and correct the point cloud data according to the ups and downs parameters.

[0103] In addition to the beneficial effects of the first embodiment, the embodiment of the present invention further refines the process of the robot acquiring point cloud data, and can obtain point cloud data including height, thereby making the detected obstacle information more accurate, or, in the process of mapping, making the map information more complete and accurate.

[0104] Embodiment 3

[0105] See also Figure 3 , Figure 3: is a flow chart of a heave correction method disclosed in an embodiment of the present invention. Based on the technical solution provided in the first embodiment, this embodiment further optimizes step S106 "determining the heave parameters of the robot according to the point cloud data, and correcting the point cloud data according to the heave parameters". The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here, namely:

[0106] The point cloud data is corrected according to a first flip angle, a first pitch angle and a first height.

[0107] Based on the above optimization, Figure 3 As shown, a heave correction method provided in this embodiment may specifically include the following steps:

[0108] S301. When the robot is up and down, obtain point cloud data detected by a binocular sensor.

[0109] S302, determine whether the point cloud data meets the first preset condition; if not, execute step S303; if so, execute step S304.

[0110] S303, acquiring inertial measurement unit data detected by an inertial measurement unit; and correcting the point cloud data according to the inertial measurement unit data.

[0111] S304, determining whether the point cloud data satisfies a second preset condition; if not, returning to step S303; if so, executing step S305.

[0112] S305, determining whether the point cloud data satisfies a third preset condition; if not, returning to step S303; if so, executing step S306.

[0113] S306: Correct the point cloud data according to the first flip angle, the first pitch angle, and the first height.

[0114] It should be noted that in Example 1, when judging whether the point cloud data meets the third preset condition, the first flip angle, the first pitch angle and the first height have been calculated based on the point cloud data. When it is further judged that the first flip angle is less than the first angle threshold, and the first pitch angle is less than the second angle threshold, and the first height is less than the first height threshold, this embodiment can correct the point cloud data according to the calculated first flip angle, the first pitch angle and the first height.

[0115] In addition to the beneficial effects of embodiment one, the embodiment of the present invention further refines the process of correcting the normal ups and downs of the robot using point cloud data, so that the robot can automatically use ground point cloud data to correct the ups and downs of the robot when the amount of point cloud data is sufficient, thereby improving the intelligence of the robot and improving people's user experience.

[0116] Embodiment 4

[0117] See also Figure 4 , Figure 4 : is a flowchart of a heave correction method disclosed in an embodiment of the present invention. Based on the technical solution provided in the first embodiment, this embodiment further optimizes step S103 "obtaining inertial measurement unit data detected by the inertial measurement unit; correcting the point cloud data according to the inertial measurement unit data". The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here, namely:

[0118] S401. When the robot is in ups and downs, obtain point cloud data detected by a binocular sensor.

[0119] S402, determine whether the point cloud data meets the first preset condition; if not, execute step S403; if so, execute step S408.

[0120] S403: Acquire inertial measurement unit data detected by the inertial measurement unit, where the inertial measurement unit data includes angular velocity and acceleration.

[0121] It should be noted that the inertial measurement unit data (also called inertial measurement data) is the data collected by the inertial measurement unit (IMU) during the movement of the robot, usually including angular velocity and acceleration. The angular velocity and acceleration data can be used to calculate information such as the roll angle and pitch angle.

[0122] Optionally, in this embodiment, before step S403, the method may further include the following steps:

[0123] Obtaining the position and posture of the robot and a global obstacle map of the location of the robot;

[0124] Determining whether the robot is picked up according to the posture and the global obstacle map;

[0125] If not, proceed to step S403; if so, return to step S401.

[0126] It should be noted that, in order to improve the efficiency of obstacle detection, this step is set to determine the position of the robot in the global obstacle map from the robot's posture, so as to determine whether there are factors that will cause ups and downs near the robot's position. If there are factors that will cause ups and downs near the robot's position, the posture is considered to be credible, and then in the following time, when the robot rises and falls, this embodiment will perform ups and downs correction on the robot. On the contrary, if there are no factors that will cause ups and downs near the robot's position, the posture is considered to be unreliable, and the robot may be picked up at this time, so there is no need to perform ups and downs correction on the robot.

[0127] S404, calculating a second flip angle and a second pitch angle, and a flip angle change value and a pitch angle change value between the current frame point cloud data and the previous frame point cloud data according to the angular velocity and the acceleration.

[0128] S405, determining whether the roll angle change value and the pitch angle change value are both less than a fifth preset threshold; if not, executing step S406; if so, executing step S407.

[0129] The fifth preset threshold value may be arbitrarily set by technicians based on experience and actual application scenarios, and is not specifically limited in this embodiment.

[0130] S406: Filter the point cloud data of the current frame.

[0131] It should be noted that by comparing the flip angle or pitch angle corresponding to two adjacent frames of point cloud data, the flip angle change value and the pitch angle change value of the two adjacent frames of point cloud data can be obtained, which can be used to determine whether the robot has fluctuated. Specifically, if the robot does not fluctuate during the march, the flip angle change value and the pitch angle change value are basically close to 0. On the contrary, if the flip angle change value and the pitch angle change value are greater than zero, it means that the robot has fluctuated. However, since the fluctuations that need to be corrected in this embodiment are small fluctuations (i.e., normal fluctuations), after determining that the flip angle change value and the pitch angle change value are greater than zero, it can also be determined whether the flip angle change value and the pitch angle change value are both less than the fifth preset threshold value. In the case of large fluctuations of the robot, such as the large fluctuations (i.e., abnormal fluctuations) such as the robot being picked up, the current frame of point cloud data is filtered to avoid the inaccurate point cloud data detected under large fluctuations affecting the obstacle detection results or the accuracy of the map.

[0132] S407: Correct the point cloud data according to the second flip angle and the second pitch angle.

[0133] It should be noted that, after removing the abnormal fluctuations as described above, this embodiment defines the remaining fluctuations as normal fluctuations, such as the robot fluctuations caused by cleaning (mopping mode, sweeping mode) when operating on normal ground, the robot fluctuations caused by operating on uneven ground, the robot fluctuations caused by crossing low obstacles, the robot fluctuations caused by crossing thresholds (depressions), or the robot fluctuations caused by being kicked during cleaning. These fluctuations are all within the scope of this embodiment, and corrections can be made for these smaller fluctuations to ensure the accuracy of obstacle detection results or the accuracy of mapping.

[0134] Optionally, in this embodiment, after step S407, the method may further include the following steps:

[0135] The grid data of the global obstacle map is updated according to the point cloud data.

[0136] S408, determining whether the point cloud data satisfies a second preset condition; if not, returning to step S403; if so, executing step S409.

[0137] S305, determining whether the point cloud data satisfies a third preset condition; if not, returning to step S403; if so, executing step S410.

[0138] S410: Determine the ups and downs parameters of the robot according to the point cloud data, and correct the point cloud data according to the ups and downs parameters.

[0139] In addition to the beneficial effects of embodiment one, the embodiment of the present invention further refines the process of correcting the normal ups and downs of the robot using the inertial measurement unit data detected by the inertial measurement unit, so that when the amount of ground point cloud data is insufficient, the robot can use the inertial measurement unit data to correct the ups and downs of the robot, thereby improving the intelligence of the robot and people's user experience.

[0140] Embodiment 5

[0141] Please see attached Figure 5 , is a functional module diagram of a heave correction device provided in Embodiment 5 of the present invention, and the device is suitable for executing the heave correction method provided in Embodiment 5 of the present invention. The device specifically comprises the following modules:

[0142] A data acquisition module 501 is used to acquire point cloud data detected by a binocular sensor when the robot is up and down;

[0143] The undulation correction module 502 is used to determine whether the point cloud data meets a first preset condition;

[0144] If not, obtaining inertial measurement unit data detected by the inertial measurement unit; correcting the point cloud data according to the inertial measurement unit data; and if so, determining whether the point cloud data meets a second preset condition;

[0145] If not, obtaining inertial measurement unit data detected by the inertial measurement unit; correcting the point cloud data according to the inertial measurement unit data; and if so, determining whether the point cloud data meets a third preset condition;

[0146] If not, obtain inertial measurement unit data detected by the inertial measurement unit; correct the point cloud data according to the inertial measurement unit data; if yes, determine the ups and downs parameters of the robot according to the point cloud data, and correct the point cloud data according to the ups and downs parameters.

[0147] Optionally, in the undulation correction device, the data acquisition module 501 is specifically used for:

[0148] Acquire depth data detected by the binocular sensor;

[0149] generating a depth map according to the depth data;

[0150] Extracting depth data in the depth map according to a preset depth threshold;

[0151] The depth data is projected in the robot coordinate system to obtain point cloud data.

[0152] Optionally, in the undulation correction device, the step of determining whether the point cloud data meets a first preset condition performed by the undulation correction module 502 includes:

[0153] It is determined whether the amount of the point cloud data is greater than or equal to a first preset threshold.

[0154] Optionally, the undulation correction device further comprises:

[0155] A downsampling module, used for performing voxel downsampling on the point cloud data before the step of determining whether the point cloud data satisfies the first preset condition;

[0156] Accordingly, the step of determining whether the point cloud data meets the first preset condition performed by the undulation correction module 502 includes:

[0157] It is determined whether the amount of the point cloud data after voxel downsampling is greater than or equal to a second preset threshold.

[0158] Optionally, in the undulation correction device, the step of determining whether the point cloud data meets the second preset condition performed by the undulation correction module 502 includes:

[0159] Randomly select three acquisition points from the point cloud data;

[0160] Determine a first plane according to the three acquisition points, wherein the first plane corresponds to a first plane equation;

[0161] Calculate the interior points and root mean square error of the first plane according to the first plane equation;

[0162] determining a first target plane according to the interior points and the root mean square error of the first plane, the first target plane corresponding to the first target plane equation;

[0163] Calculating the interior points and the root mean square error of the first target plane according to the first target plane equation;

[0164] Determine whether the number of interior points of the first target plane is greater than a third preset threshold, and whether the root mean square error of the first target plane is less than a fourth preset threshold; if not, iteratively calculate a new plane equation, interior points, and root mean square error until the number of interior points corresponding to the new plane equation is greater than the third preset threshold and the root mean square error corresponding to the new plane equation is less than the fourth preset threshold, or until the number of iterations reaches a preset number;

[0165] If so, it is determined that the point cloud data meets the second preset condition.

[0166] Optionally, in the undulation correction device, the step of determining whether the point cloud data meets a third preset condition performed by the undulation correction module 502 includes:

[0167] Calculate a first flip angle, a first pitch angle and a first height according to the point cloud data;

[0168] Determining whether the first flip angle is less than a first angle threshold, whether the first pitch angle is less than a second angle threshold, and whether the first height is less than a first height threshold;

[0169] If so, it is determined that the point cloud data meets the third preset condition.

[0170] Optionally, in the undulation correction device, the undulation parameters include the first flip angle, the first pitch angle, and the first height, and the step of correcting the point cloud data according to the point cloud data performed by the undulation correction module 502 includes:

[0171] The point cloud data is corrected according to the first flip angle, the first pitch angle, and the first height.

[0172] Optionally, in the heave correction device, the inertial measurement unit data includes angular velocity and acceleration;

[0173] Accordingly, the step of correcting the point cloud data according to the inertial measurement unit data performed by the undulation correction module 502 includes:

[0174] Calculate a second flip angle and a second pitch angle, as well as a flip angle change value and a pitch angle change value between a current frame of point cloud data and a previous frame of point cloud data according to the angular velocity and the acceleration;

[0175] Determine whether the flip angle change value and the pitch angle change value are both less than a fifth preset threshold;

[0176] If not, filter the point cloud data of the current frame;

[0177] If yes, the point cloud data is corrected according to the second flip angle and the second pitch angle.

[0178] Optionally, the undulation correction device further comprises:

[0179] A judging and lifting module is used to obtain the position and posture of the robot and a global obstacle map of the place where the robot is located before the step of obtaining the inertial measurement unit data detected by the inertial measurement unit;

[0180] Determining whether the robot is picked up according to the posture and the global obstacle map;

[0181] If not, the step of acquiring the inertial measurement unit data detected by the inertial measurement unit is performed.

[0182] Optionally, the undulation correction device further comprises:

[0183] A map updating module is used to update the grid data of the global obstacle map according to the point cloud data after the step of correcting the point cloud data according to the second flip angle and the second pitch angle.

[0184] An embodiment of the present invention provides an ups and downs correction device, which obtains and utilizes point cloud data detected by a binocular sensor or inertial measurement unit data detected by an inertial measurement unit, and corrects the point cloud data of the current frame of the robot when the robot ups and downs, so as to obtain corrected point cloud data. When obstacle detection is performed, the corrected point cloud data is used for obstacle detection, which can reduce the impact of the robot's ups and downs on obstacle detection and improve the accuracy of obstacle detection. Alternatively, when the robot is building a map, the corrected point cloud data is used for building the map, which can improve the accuracy of the robot's mapping and has a high value for promotion and application.

[0185] The above-mentioned device can execute the method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0186] Embodiment 6

[0187] Figure 6 This is a schematic diagram of the structure of a robot provided in Example 6 of the present invention. Figure 6 A block diagram of an exemplary robot 12 suitable for use in implementing embodiments of the present invention is shown. Figure 6 The robot 12 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0188] like Figure 6 As shown, the robot 12 is embodied in the form of a general purpose computing device. The components of the robot 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting various system components (including the system memory 28 and the processing unit 16).

[0189] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0190] The robot 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the robot 12, including volatile and non-volatile media, removable and non-removable media.

[0191] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The robot 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 6 not shown, usually called a "hard drive"). Although Figure 6 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0192] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0193] The robot 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the robot 12, and / or any device that enables the robot 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the robot 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the robot 12 via the bus 18. It should be understood that although Figure 6 Not shown, other hardware and / or software modules may be used in conjunction with the robot 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0194] The processing unit 16 executes the programs stored in the system memory 28 to perform various functional applications and data processing, such as implementing the heave correction method provided in the embodiment of the present invention.

[0195] That is, when the processing unit 16 executes the program, it achieves: when the robot is up and down, obtaining point cloud data detected by the binocular sensor; judging whether the point cloud data meets the first preset condition; if so, judging whether the point cloud data meets the second preset condition; if so, judging whether the point cloud data meets the third preset condition; if so, determining the ups and down parameters of the robot according to the point cloud data, and correcting the point cloud data according to the ups and down parameters.

[0196] Embodiment 7

[0197] Embodiment 7 of the present invention provides a computer-readable storage medium on which computer-executable instructions are stored. When the instructions are executed by a processor, the heave correction method provided in all the embodiments of the present invention is implemented:

[0198] That is, when the processor executes the computer-executable instructions, it achieves: when the robot is up and down, obtaining point cloud data detected by the binocular sensor; judging whether the point cloud data meets the first preset condition; if so, judging whether the point cloud data meets the second preset condition; if so, judging whether the point cloud data meets the third preset condition; if so, determining the ups and down parameters of the robot according to the point cloud data, and correcting the point cloud data according to the ups and down parameters.

[0199] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0200] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0201] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0202] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0203] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for undulation correction, characterized in that: Applied to a robot, the robot includes a binocular sensor, and the method includes: When the robot is up and down, acquiring point cloud data detected by the binocular sensor; Determining whether the point cloud data meets a first preset condition; If so, determining whether the point cloud data satisfies a second preset condition; If so, determining whether the point cloud data satisfies a third preset condition; If so, determining the ups and downs parameters of the robot according to the point cloud data, and correcting the point cloud data according to the ups and downs parameters; The step of determining whether the point cloud data meets the first preset condition comprises: Determine whether the amount of the point cloud data is greater than or equal to a first preset threshold; The step of determining whether the point cloud data meets the second preset condition comprises: Fitting a first target plane through the point cloud data, wherein the first target plane corresponds to a first target plane equation; Determining whether the first target plane satisfies the condition of a horizontal plane by the number of interior points and the root mean square error corresponding to the first target plane equation; The step of determining whether the point cloud data meets the third preset condition comprises: Calculate a first flip angle, a first pitch angle and a first height according to the point cloud data; Determining whether the first flip angle is less than a first angle threshold, whether the first pitch angle is less than a second angle threshold, and whether the first height is less than a first height threshold; If so, it is determined that the point cloud data meets the third preset condition.

2. The method for undulation correction according to claim 1, characterized in that: The step of acquiring the point cloud data detected by the binocular sensor comprises: Acquire depth data detected by the binocular sensor; generating a depth map according to the depth data; Extracting depth data in the depth map according to a preset depth threshold; The depth data is projected in the robot coordinate system to obtain point cloud data.

3. The method for undulation correction according to claim 1, characterized in that: Before the step of determining whether the point cloud data meets the first preset condition, the method further includes: Downsampling the point cloud data by voxels; The step of determining whether the point cloud data meets the first preset condition comprises: It is determined whether the amount of the point cloud data after voxel downsampling is greater than or equal to a second preset threshold.

4. The method for undulation correction according to claim 1, characterized in that: The first target plane is fitted by the point cloud data, and the first target plane corresponds to a first target plane equation; The step of judging whether the first target plane satisfies the condition of a horizontal plane by the number of interior points and the root mean square error corresponding to the first target plane equation comprises: Randomly select three acquisition points from the point cloud data; Determine a first plane according to the three acquisition points, wherein the first plane corresponds to a first plane equation; Calculate the interior points and root mean square error of the first plane according to the first plane equation; determining a first target plane according to the interior points and the root mean square error of the first plane, the first target plane corresponding to the first target plane equation; Calculating the interior points and the root mean square error of the first target plane according to the first target plane equation; Determine whether the number of interior points of the first target plane is greater than a third preset threshold, and whether the root mean square error of the first target plane is less than a fourth preset threshold; if not, iteratively calculate a new plane equation, interior points, and root mean square error until the number of interior points corresponding to the new plane equation is greater than the third preset threshold and the root mean square error corresponding to the new plane equation is less than the fourth preset threshold, or until the number of iterations reaches a preset number; If so, it is determined that the point cloud data meets the second preset condition.

5. The undulation correction method according to claim 1, characterized in that: The relief parameters include the first flip angle, the first pitch angle and the first height, and the step of correcting the point cloud data according to the relief parameters includes: The point cloud data is corrected according to the first flip angle, the first pitch angle, and the first height.

6. The method for undulation correction according to claim 1, characterized in that: The robot further comprises an inertial measurement unit, and the method further comprises: If the point cloud data does not satisfy the first preset condition; or, the point cloud data does not satisfy the second preset condition; or, the point cloud data does not satisfy the third preset condition, then the inertial measurement unit data detected by the inertial measurement unit is acquired; and the point cloud data is corrected according to the inertial measurement unit data.

7. The method for undulation correction according to claim 6, characterized in that: The inertial measurement unit data includes angular velocity and acceleration; The step of correcting the point cloud data according to the inertial measurement unit data comprises: Calculate a second flip angle and a second pitch angle, and a flip angle change value and a pitch angle change value between a current frame of point cloud data and a previous frame of point cloud data according to the angular velocity and the acceleration; Determining whether the flip angle change value and the pitch angle change value are both less than a fifth preset threshold; If not, filter the point cloud data of the current frame; If yes, the point cloud data is corrected according to the second flip angle and the second pitch angle.

8. The method for undulation correction according to claim 7, characterized in that: Before the step of acquiring the inertial measurement unit data detected by the inertial measurement unit, the method further includes: Obtaining the position and posture of the robot and a global obstacle map of the location of the robot; Determining whether the robot is picked up according to the posture and the global obstacle map; If not, the step of acquiring the inertial measurement unit data detected by the inertial measurement unit is performed.

9. The method for undulation correction according to claim 8, characterized in that: After the step of correcting the point cloud data according to the second flip angle and the second pitch angle, the method further includes: The grid data of the global obstacle map is updated according to the point cloud data.

10. A ripple correction device, characterized in that: Applied to a robot, the robot includes a binocular sensor and an inertial measurement unit, and the device includes: A data acquisition module, used for acquiring point cloud data detected by the binocular sensor when the robot is in ups and downs; An undulation correction module, used for determining whether the point cloud data meets a first preset condition; If so, determining whether the point cloud data satisfies a second preset condition; If so, determining whether the point cloud data satisfies a third preset condition; If so, determining the ups and downs parameters of the robot according to the point cloud data, and correcting the point cloud data according to the ups and downs parameters; The step of determining whether the point cloud data meets the first preset condition comprises: Determine whether the amount of the point cloud data is greater than or equal to a first preset threshold; The step of determining whether the point cloud data meets the second preset condition comprises: Fitting a first target plane through the point cloud data, wherein the first target plane corresponds to a first target plane equation; Determining whether the first target plane satisfies the condition of a horizontal plane by the number of interior points and the root mean square error corresponding to the first target plane equation; The step of determining whether the point cloud data meets the third preset condition comprises: Calculate a first flip angle, a first pitch angle and a first height according to the point cloud data; Determining whether the first flip angle is less than a first angle threshold, whether the first pitch angle is less than a second angle threshold, and whether the first height is less than a first height threshold; If so, it is determined that the point cloud data meets the third preset condition.

11. A robot comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the undulation correction method according to any one of claims 1 to 9 is implemented.

12. A storage medium comprising computer executable instructions, wherein the computer executable instructions are executed by a computer processor to implement the heave correction method according to any one of claims 1 to 9.

Images