Trajectory planning method, device, robot and storage medium

Abstract

The embodiments of the present application provide a trajectory planning method, device, robot and storage medium. In some embodiments of the present application, multiple frames of images collected by the self-moving robot when it moves are obtained; the vibration amplitude of the self-moving robot when it moves is determined according to the pixel coordinates of the corresponding feature points in the multiple frames of images; the path direction of the self-moving robot is determined according to the vibration amplitude; and the moving trajectory of the self-moving robot is planned according to the path direction. A moving trajectory with a small vibration amplitude can be planned for the self-moving robot. By detecting the vibration amplitude in each moving direction in the ground medium area, a moving trajectory in which the ground texture has the least effect on the self-moving robot can be planned, which can effectively improve the working effect of the self-moving robot on different ground textures.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more particularly to a trajectory planning method, device, robot and storage medium. Background Art

[0002] With the continuous development of artificial intelligence technology, various intelligent robots are increasingly entering people's lives, such as logistics robots, sweeping robots, welcoming robots and so on.

[0003] Taking a sweeping robot as an example, the sweeping robot needs to be able to clean a variety of objects to be cleaned, such as carpets, wooden floors, tiles, etc. In actual applications, the cleaning effects on different cleaning objects are not exactly the same. For example, when a sweeping robot cleans a carpet, the sweeping robot has a poor cleaning effect due to the fluff on the carpet surface. Summary of the invention

[0004] Multiple aspects of the present application provide a trajectory planning method, device, robot and storage medium, which are used to plan a moving trajectory with a low vibration amplitude during the movement of the mobile robot.

[0005] The present application provides a trajectory planning method, which is applied to a self-propelled robot. The method includes:

[0006] Acquire multiple frames of images collected by the self-moving robot while it is moving;

[0007] Determining the vibration amplitude of the self-propelled robot in different moving directions according to the pixel coordinates of the corresponding feature points in the multiple frames of images;

[0008] determining a path direction of the self-moving robot according to the vibration amplitude;

[0009] The moving trajectory of the self-moving robot is planned according to the path direction.

[0010] The present application provides a trajectory planning device, which is applied to a self-propelled robot. The device includes:

[0011] An acquisition module, used to acquire multiple frames of images collected by the self-moving robot while it is moving;

[0012] an amplitude determination module, for determining the vibration amplitude of the self-moving robot when it moves according to the pixel coordinates of the corresponding feature points in the multiple frames of images;

[0013] A path determination module, used to determine the path direction of the self-moving robot according to the vibration amplitude;

[0014] The trajectory planning module is used to plan the moving trajectory of the self-moving robot according to the path direction.

[0015] An embodiment of the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by one or more processors, the one or more processors are caused to perform the following actions:

[0016] Acquire multiple frames of images collected by the mobile robot while it is moving;

[0017] Determining the vibration amplitude of the self-moving robot when it moves according to the pixel coordinates of the corresponding feature points in the multiple frames of images;

[0018] determining a path direction of the self-moving robot according to the vibration amplitude;

[0019] The moving trajectory of the self-moving robot is planned according to the path direction.

[0020] An embodiment of the present application provides a self-moving robot, comprising: a mechanical body, on which one or more processors and one or more memories storing computer programs are provided;

[0021] The one or more processors are configured to execute the computer program to:

[0022] Acquire multiple frames of images collected by the self-moving robot while it is moving;

[0023] Determining the vibration amplitude of the self-moving robot when it moves according to the pixel coordinates of the corresponding feature points in the multiple frames of images;

[0024] determining a path direction of the self-moving robot according to the vibration amplitude;

[0025] The moving trajectory of the self-moving robot is planned according to the path direction.

[0026] In some embodiments of the present application, before the self-moving robot performs a mobile operation in a ground medium area, it is necessary to plan the moving trajectory. Specifically, after the self-moving robot arrives in the ground medium area, the self-moving robot rotates according to a preset radius and continuously collects multiple frames of images corresponding to the rotation process. Then, the pixel coordinates of the corresponding feature points in the multiple frames of images are calculated to determine the vibration amplitude in different moving directions. By detecting the vibration amplitude in each moving direction in the ground medium area, it is possible to plan a moving trajectory in which the ground texture has the least effect on the vibration amplitude of the self-moving robot, which can effectively improve the working effect of the self-moving robot on different ground textures, such as improving the cleaning effect of the sweeping self-moving robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0028] Figure 1 A schematic diagram of the cleaning trajectory of an existing sweeping robot provided in an embodiment of the present application;

[0029] Figure 2 A schematic diagram of a flow chart of a trajectory planning method provided in an embodiment of the present application;

[0030] Figure 3 A schematic diagram of a method for detecting texture direction provided in an embodiment of the present application;

[0031] Figure 4 A schematic diagram of determining a path direction provided in an embodiment of the present application;

[0032] Figure 5 A schematic diagram of movement trajectory planning provided in an embodiment of the present application;

[0033] Figure 6a , 6b A schematic diagram of moving trajectory planning for different starting positions provided in an embodiment of the present application;

[0034] Figure 7 A schematic diagram of an exemplary movement trajectory planning based on texture direction provided in an embodiment of the present application;

[0035] Figure 8 A schematic diagram of the structure of a trajectory planning device is provided for an embodiment of the present application;

[0036] Fig. 9 A schematic diagram of the structure of a self-moving robot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0038] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two, but does not exclude the situation of including at least one.

[0039] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0040] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a product or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such a product or system. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the product or system including the elements.

[0041] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0042] In the present application, in addition to being able to walk autonomously and perform corresponding service functions, the self-moving robot may also have functions such as computing, communication, and surfing the Internet. The self-moving robot in the embodiment of the present application may be a drone, an unmanned vehicle, and the like. Depending on the application scenario, the basic service functions of the self-moving robot may also be different. The self-moving robot may be a sweeping robot, a following robot, a welcoming robot, and the like. For example, for a sweeping robot used in scenes such as homes, office buildings, and shopping malls, its basic service function is to clean the ground in the scene; for a glass-wiping robot used in scenes such as homes, office buildings, and shopping malls, its basic service function is to clean the glass in the scene; for a following robot, its basic service function is to follow the target object; for a welcoming robot, its basic service function is to welcome customers and guide them to their destination.

[0043] In order to facilitate understanding of the applicable scenarios of the trajectory planning method provided in the present application, a sweeping robot is taken as an example for explanation below, assuming that the sweeping robot is currently cleaning the carpet area. Figure 1 A schematic diagram of the cleaning trajectory of an existing cleaning robot provided in an embodiment of the present application. Figure 1It can be seen that the weaving direction of the carpet is from left to right, which is indicated by the solid line with arrows in the figure. In the prior art, the sweeping robot may sweep back and forth on the carpet in a left-right reciprocating trajectory. It is easy to understand that the fiber fluff on the carpet is woven in a certain order, that is, the weaving direction of the same carpet is certain. When the sweeping robot moves in the same direction as the weaving direction, it will be relatively stable; when the sweeping robot moves in the opposite direction to the weaving direction, the sweeping robot will be bumpy during the movement due to the erection of the carpet fiber fluff, and the vibration amplitude will be large. This bump will seriously affect the cleaning effect of the sweeping robot.

[0044] Therefore, the technical solution of this application proposes a cleaning method for areas to be cleaned where there is an obvious texture direction such as carpets, and the texture direction causes the robot to vibrate significantly. Before the sweeping robot cleans the carpet area, it needs to first determine the weaving direction of the carpet, and then determine a moving trajectory that can meet the cleaning needs of the sweeping robot based on the weaving direction. When the sweeping robot cleans according to the moving trajectory, it can effectively avoid the adverse effects of the carpet weaving direction on the cleaning work.

[0045] Figure 2 A flow chart of a trajectory planning method provided in an embodiment of the present application. The method comprises the following steps:

[0046] 201: Acquire multiple frames of images collected by the mobile robot while it is moving.

[0047] 202: Determine the vibration amplitude of the self-moving robot when it moves according to the pixel coordinates of the corresponding feature points in the multiple frames of images.

[0048] 203: Determine the path direction of the self-moving robot according to the vibration amplitude.

[0049] 204: Plan a moving trajectory of the self-moving robot according to the path direction.

[0050] When the self-propelled robot needs to move on a ground with obvious textures (such as wooden floors) or on a ground with regular tracks, the self-propelled robot needs to plan a trajectory that allows for more stable movement based on the actual ground conditions. When planning the trajectory, the user can assist in planning, or the self-propelled robot can plan it completely autonomously.

[0051] In the application scenario where the self-moving robot plans the trajectory completely by itself, the self-moving robot needs to first detect the ground medium area. If the self-moving robot is detected to be in a preset ground medium area (for example, a carpet area), the texture direction of the ground medium area is determined; then the path direction is determined based on the texture direction; and then the movement trajectory is planned based on the path direction. If the self-moving robot detects the ground medium area and finds that it is not in the preset ground medium area, there is no need to detect the texture direction, nor to determine the path direction based on the texture direction.

[0052] like Figure 3 This is a schematic diagram of a method for detecting texture direction provided in an embodiment of the present application. Figure 3 How to determine the texture direction in the ground medium area is specifically described. It is assumed that the ground medium is a carpet and the texture direction is the weaving direction of the carpet. That is, a sweeping robot is used to determine the weaving direction of the carpet as an example for description.

[0053] At least one monocular camera is provided on the sweeping robot. Figure 3 In the sweeping robot shown in , the monocular camera is usually set at the front end of the robot, so that the monocular camera has a better field of view, which is conducive to the clearer and more accurate acquisition of multiple frames of images. When acquiring the multiple frames of images acquired by the sweeping robot while moving, the sweeping robot moves in different directions, which can be non-straight lines such as circles and ellipses. For example, it rotates in a circular shape according to a preset radius (it should be noted that the circular rotation mentioned here is not the self-rotation of the sweeping robot). During the rotation of the sweeping robot, multiple frames of images in different moving directions are continuously collected by at least one monocular camera. It should be noted that the reason why circular rotation is used here is to be able to detect all directions of the ground medium area to avoid the problem of missing detection in individual directions.

[0054] The sweeping robot will analyze the collected images to determine the magnitude of the vibration amplitude of the sweeping robot in different moving directions. During the movement of the robot, there are many ways to represent the magnitude of the vibration amplitude. The technical solution of this application is to represent the magnitude of the vibration amplitude by the pixel ordinate difference of the feature point in the collected multiple frames of images.

[0055] It is easy to understand that when the vibration amplitude of the sweeping robot is large, the change in the vertical coordinate of the pixel corresponding to the same feature point in the collected multiple frames of images is larger; when the vibration amplitude of the sweeping robot is small (for example, there is no vibration), the change in the vertical coordinate of the pixel corresponding to the same feature point in the collected multiple frames of images is smaller.

[0056] The robot vacuum cleaner stores a map of the current environment, and the map has a corresponding coordinate system, so that the robot vacuum cleaner can accurately identify its position and direction in the current environment through the laser radar during operation, and accurately know the position of the marked obstacles in the map, which can assist the robot in planning the movement trajectory and automatically avoid obstacles. During the image acquisition process of the robot vacuum cleaner, the image acquisition frequency is usually relatively high, and multiple frames of images are continuously acquired in each moving direction, so as to determine the vibration amplitude corresponding to each moving direction by comparing the changes in the coordinates of the feature points in the multiple frames of images.

[0057] Assuming that two adjacent frames of images are captured by a monocular camera, the captured images need to be simply processed first. Because the shooting angle of the sweeping robot is constantly changing during vibration, the relative positions of the boundary feature points of the captured image are significantly distorted. In order to eliminate the influence of these feature points on the coordinate comparison results, the captured images can be cropped, and only the middle part of the image (the part with better perspective consistency) is retained as the first image and the second image. After obtaining the first image and the second image, the optical flow algorithm can be used to calculate the vibration amplitude of the sweeping robot.

[0058] Assume that the pixel coordinates of the valid feature points in the first image are Pk = {(x1, y1), (x2, y2), (x3, y3), ... (xn, yn)}, and the pixel coordinates of the valid feature points in the second image are Pk-1 = {(x'1, y'1), (x'2, y'2), (x'3, y'3), ... (x'n, y'n)}.

[0059] After obtaining the first image and the second image, before calculating the vibration amplitude based on the pixel coordinates of the above-mentioned feature points, the influence of some interfering feature points must be eliminated first. In practical applications, since the sweeping robot is always moving forward during the image acquisition process, it may be moving at a constant speed or at a variable speed. Since the sweeping robot itself moves forward, there may be a relatively obvious difference in the pixel coordinates of some feature points in the two adjacent frames of the image. When the sweeping robot moves forward too fast, there may be a relatively obvious difference in the pixel coordinates of all feature points in the first image and the second image. Therefore, in order to eliminate the influence of the sweeping robot's own movement on the feature point coordinate comparison results, the following filtering measures can be specifically adopted:

[0060] First, the pixel coordinate values ​​of each feature point in the first image and the second image are obtained, and then the pixel horizontal coordinates of the feature points at the specified position in the first image are compared with the pixel horizontal coordinates of the feature points corresponding to the second image to obtain the pixel horizontal coordinate difference. If the pixel horizontal coordinate difference is greater than the threshold, it is considered that the pixel coordinates of the feature point are significantly affected by the robot's own motion, and the feature point can no longer be used to calculate the vibration amplitude. The sweeping robot can use the above-mentioned pixel horizontal coordinate comparison method to compare and screen all feature points in the first image and the second image to obtain qualified feature points. Then determine the vibration amplitude based on the qualified feature points.

[0061] Assume that all feature points in the first image and the second image are compared and screened once, and after obtaining qualified feature points, the vibration amplitude can be calculated using the valid feature points in the first image and the second image. Specifically: obtain n valid feature points in the first image, and the corresponding pixel ordinate values ​​y1~yn, obtain n valid feature points in the second image, and the corresponding pixel ordinate values ​​y'1~y'n. Then, calculate the absolute value of the difference between the pixel ordinate y and the pixel ordinate y' of the n valid feature points one by one, and then calculate the average of the absolute values ​​of the n differences as the pixel ordinate difference between the first image and the second image. This pixel ordinate difference represents the relative vibration amplitude generated by the adjacent first image and second image.

[0062] Then, the maximum amplitude and the minimum amplitude are determined according to the pixel ordinate difference. Generally speaking, the larger the pixel ordinate difference, the larger the corresponding vibration amplitude, and the smaller the pixel ordinate difference, the smaller the corresponding vibration amplitude. Among them, the moving direction corresponding to the maximum vibration amplitude is opposite to the weaving direction of the carpet, and the moving direction corresponding to the minimum vibration amplitude is the same as the carpet weaving. Therefore, in practical applications, if you want to determine the texture direction, you can first find the direction of the maximum vibration amplitude or the direction of the minimum vibration amplitude. For example, the sweeping robot wants to find the texture direction of the carpet. After experiments, it is known that when the moving direction of the sweeping robot is opposite to the texture direction, the vibration amplitude generated is also the largest; when the moving direction of the sweeping robot is the same as or close to the texture direction, the vibration amplitude is not very large, and it is not easy to find the minimum vibration amplitude, but it is relatively easy to find the maximum vibration amplitude.

[0063] As mentioned above, during the rotation of the self-propelled robot, images and corresponding directions are collected in real time. However, the image collection frequency of the robot's monocular camera is higher than the direction collection frequency of the robot's laser radar. In order to accurately find the direction corresponding to the maximum amplitude and the minimum amplitude, a list can be established. For example, in the list, every 10 vibration amplitudes of V1 to V10 correspond to a moving direction D1, and vibration amplitudes of V11 to V20 correspond to a moving direction D2. By analogy, the corresponding relationship between all vibration amplitudes and moving directions can be obtained.

[0064] After obtaining the above list, in order to more accurately find the maximum vibration amplitude or the minimum vibration amplitude, a sliding window algorithm can be used to determine the texture direction (that is, the weaving direction of the carpet). For example, assume that each sliding window contains 10 vibration amplitude sampling data, and then calculate the vibration amplitude average value V for the absolute values ​​of these 10 data; then calculate the average value of the vibration amplitude of the next sliding window, and the difference between the two adjacent sliding windows is at least ten sampling periods corresponding to the sampling data. Each vibration amplitude average value V corresponds to a moving direction D, and then, the vibration amplitude average values ​​V corresponding to all sliding windows during one rotation are compared to determine the maximum amplitude and the minimum amplitude.

[0065] In order to accurately determine the maximum vibration amplitude or the minimum vibration amplitude, the vibration amplitude of the effective feature points in two adjacent frames of images is first calculated, and the average value of the vibration amplitude of all effective feature points is used as the vibration amplitude between the two frames of images. Then, based on the vibration amplitudes corresponding to the multiple frames of images, the average value of the vibration amplitude of the multiple frames of images is calculated as the vibration amplitude in a certain moving direction. The vibration amplitudes in each moving direction are compared to confirm the maximum amplitude or the minimum amplitude in one rotation.

[0066] In practical applications, if multiple identical large vibration amplitudes are obtained through calculation, the middle direction among the corresponding directions of the multiple large vibration amplitudes is selected as the direction corresponding to the maximum amplitude. For example, if the vibration amplitudes V1, V2, and V3 are the same, and the corresponding moving directions are D1, D2, and D3, respectively, and D2 is greater than D3 and D2 is less than D1, then the moving direction corresponding to the maximum vibration amplitude is selected as D2. This moving direction D2 is determined by the robot rotating one circle.

[0067] In practical applications, when the robot rotates one circle, due to the influence of the robot's position positioning accuracy, or the calculation accuracy of the maximum vibration amplitude and the minimum vibration amplitude, the corresponding relationship between the direction corresponding to the maximum vibration amplitude or the minimum vibration amplitude and the moving direction may be inaccurate. To this end, the robot can be rotated several times, and an alternative texture direction can be determined through each circle (as mentioned above, the moving direction corresponding to the maximum vibration amplitude is opposite to the texture direction, and the moving direction corresponding to the minimum vibration amplitude is in the same direction as the texture direction). Then, the average value of multiple alternative texture directions is obtained as the actual texture direction of the ground medium (for example, the weaving direction of the carpet). For example, assuming that in the rectangular coordinate system of the current room, the alternative texture direction obtained in the first circle is 30 degrees clockwise deflected relative to the Y axis, the alternative texture direction obtained in the second circle is 40 degrees clockwise deflected relative to the Y axis, and the alternative texture direction obtained in the third circle is 22 degrees clockwise deflected relative to the Y axis. The final texture direction is 31 degrees clockwise deflected relative to the Y axis.

[0068] In practical applications, after the robot completes the texture direction detection, it will further determine the path direction corresponding to the ground medium area. It is easy to understand that when the robot walks, the direction is not single, and some may have to go back and forth many times. Usually, the round trip means that the direction of going is completely opposite to the direction of returning, for example Figure 1 As shown in the figure, the robot goes from left to right, and the return direction is from right to left. Since the texture direction is from left to right, the robot will not vibrate when going, but will vibrate greatly when returning. Therefore, the direction that is the same as or opposite to the texture direction cannot be selected as the path direction. In order to minimize the influence of the texture direction during multiple round trips, the direction perpendicular to the texture direction can be selected as the path direction. This path direction can prevent the robot from vibrating greatly during the round trip.

[0069] For ease of understanding, the following will combine Figure 4 Explain the process of determining the path direction. Figure 4 A schematic diagram of determining the path direction provided by an embodiment of the present application. Assuming that the carpet is woven from left to right, that is, when the sweeping robot moves from right to left, the sweeping robot will generate a large vibration amplitude. Then, in order to ensure that the round-trip movement is as little affected by the weaving direction as possible, a direction perpendicular to the weaving direction can be selected, for example, Figure 4The upward or downward direction is used as the path direction (the up, down, left, and right mentioned here refer to the directions in the plane view). Both path directions can meet the needs of the robot to plan the moving trajectory. If the upward direction is used as the robot path direction, that is, the direction the robot is going, then the downward direction is the corresponding return direction. It can be seen that during the process of the robot moving back and forth in the path direction shown in the figure, the robot will not produce excessive vibration amplitude due to the texture direction. It is easy to understand that the robot's starting position relative to the carpet is different, so the path direction is not unique, and the moving trajectory determined based on the path direction is also not unique.

[0070] After determining the path direction, the robot's moving trajectory is planned based on the path direction. Figure 5 A schematic diagram of the movement trajectory planning provided in the embodiment of the present application. Specifically, Figure 5 As shown in the figure, assuming that the starting path direction is upward, the robot works back and forth, and a "bow"-shaped moving trajectory can be obtained. It should be noted that even if the path direction is determined, the influence of the texture direction should be excluded as much as possible when planning the moving trajectory. That is, the extension direction of the moving trajectory should be the same as the texture direction. For example, it should be extended in the order of the track numbers ①, ②, ③, and ④ in the figure to ensure that the moving direction of the "bow"-shaped connecting the head and tail is the same or similar to the texture direction. The entire cleaning process of the sweeping robot will clean the carpet area according to the moving trajectory.

[0071] In actual applications, after the sweeping robot completes the cleaning of the carpet area, it will mark the location of the carpet and the path direction or texture direction suitable for the carpet in its stored map. Furthermore, the moving trajectory is planned in combination with the starting position of the sweeping robot in the carpet. For example, Figure 6a , 6b The schematic diagram of the moving trajectory planning for different starting positions provided in the embodiment of the present application, if the starting position of the robot is A, the following can be obtained: Figure 6a The moving trajectory shown in FIG. 1 is as follows; if the robot's starting position is B, the following can be obtained: Figure 6b The moving trajectory shown; if the robot's starting position is C or D, due to the round-trip connection position in the moving trajectory (that is, Figure 6b The moving direction of the carpet (the middle semicircular arc connection position) is opposite to the texture direction, which will produce a relatively large vibration amplitude. The sweeping robot needs to move to position A or B and re-plan the moving trajectory. Through the above solution, the robot does not need to rotate a circle to re-find the carpet weaving direction. Under the premise of ensuring the cleaning effect of the carpet area, the cleaning efficiency of the sweeping robot can be effectively improved.

[0072] It should be noted that before the robot vacuum cleaner starts working in different working environments, it first determines whether the current room is carpeted. If not, it will first clean non-carpet areas such as the floor area. After completing the cleaning of the non-carpet area, it will clean the carpet area. The robot vacuum cleaner will determine whether the current robot vacuum cleaner has moved to the carpet area based on the marked position of the carpet area in the previous map.

[0073] If the robot vacuum cleaner moves to the carpet area, it will Figure 3 In the method shown, the sweeping robot rotates in a circle on the carpet, so as to determine the texture direction of the carpet (that is, the weaving direction of the carpet pile), and then determine the path direction of the sweeping robot during cleaning based on the texture direction.

[0074] If the whole house is carpeted, the robot vacuum cleaner will rotate around the carpet in a circle before starting to clean. Figure 3 In the method shown, the sweeping robot rotates in a circle on the carpet, so as to determine the texture direction of the carpet (that is, the weaving direction of the carpet pile), and then determine the path direction of the sweeping robot during cleaning based on the texture direction.

[0075] As an optional embodiment, Figure 7 A schematic diagram of an exemplary trajectory planning based on texture direction provided for an embodiment of the present application. As shown in the figure, the texture direction in the carpet area (that is, the weaving direction of the carpet pile) is along the diagonal direction of the rectangle. After the sweeping robot moves to the carpet area, it rotates on the carpet according to the rotation trajectory shown in the figure, and rotates around a 360° circle, so that the sweeping robot can determine that the texture direction of the carpet is toward the diagonal direction of the rectangle as shown in the figure. Based on the texture direction, the path direction perpendicular to the texture direction is further determined, and a "bow"-shaped moving trajectory as shown in the figure can be planned.

[0076] Figure 8 A structural schematic diagram of a trajectory planning device is provided for an embodiment of the present application, which can be applied to a robot. The device includes: an acquisition module 81, an amplitude determination module 82, a path determination module 83 and a trajectory planning module 84.

[0077] The acquisition module 81 is used to acquire multiple frames of images collected by the mobile robot when it moves.

[0078] The amplitude determination module 82 is used to determine the vibration amplitude of the self-moving robot when it moves according to the pixel coordinates of the corresponding feature points in the multiple frames of images.

[0079] The path determination module 83 is used to determine the path direction of the self-moving robot according to the vibration amplitude.

[0080] The trajectory planning module 84 is used to plan the moving trajectory of the self-moving robot according to the path direction.

[0081] Optionally, the acquisition module 81 is used to acquire multiple frames of images collected by the self-moving robot when it moves if it is determined that the self-moving robot is located in a preset ground medium area.

[0082] Optionally, the preset ground medium area includes a carpet area.

[0083] Optionally, the acquisition module 81 is used to control the self-moving robot to move in different directions; and acquire multiple frames of images collected in different moving directions during the movement of the self-moving robot.

[0084] Optionally, the amplitude determination module 82 is used to determine the vibration amplitude of the self-moving robot in different moving directions according to the average value of the pixel ordinate difference of the corresponding feature points in the multiple frames of images.

[0085] Optionally, the amplitude determination module 82 is used to determine the difference in pixel horizontal coordinates of corresponding feature points in the multiple frame images; filter out corresponding feature points whose pixel horizontal coordinate difference is greater than a threshold; and determine the vibration amplitude of the self-moving robot in different moving directions based on the average value of the pixel vertical coordinate difference of the remaining corresponding feature points in the multiple frame images.

[0086] Optionally, the path determination module 83 is used to determine the maximum or minimum vibration amplitude; determine the texture direction of the ground medium according to the maximum or minimum vibration amplitude; and determine the path direction of the self-moving robot according to the texture direction.

[0087] Optionally, the path determination module 83 is used to determine the moving direction corresponding to the minimum vibration amplitude as an alternative texture direction, or to determine the direction opposite to the moving direction corresponding to the maximum vibration amplitude as an alternative texture direction; and to determine the texture direction of the ground medium based on the average direction of multiple alternative texture directions.

[0088] Optionally, the path determination module 83 is used to determine a direction perpendicular to the texture direction as the path direction of the self-propelled robot.

[0089] Fig. 9 A schematic diagram of the structure of a self-moving robot provided for an exemplary embodiment of the present application. The self-moving robot includes a mechanical body, one or more processors 901, and one or more memories 902 storing computer programs. In addition, the self-moving robot may also include necessary components such as a power supply component 903.

[0090] One or more processors 901, configured to execute a computer program for:

[0091] Acquire multiple frames of images collected by the self-moving robot when it moves; determine the vibration amplitude of the self-moving robot when it moves according to the pixel coordinates of the corresponding feature points in the multiple frames of images; determine the path direction of the self-moving robot according to the vibration amplitude; and plan the moving trajectory of the self-moving robot according to the path direction.

[0092] Optionally, the one or more processors 901 are used to obtain multiple frames of images collected by the self-moving robot while it is moving if it is determined that the self-moving robot is located in a preset ground medium area.

[0093] Optionally, the preset ground medium area includes a carpet area.

[0094] Optionally, one or more processors 901 are used to control the self-moving robot to move in different directions; and obtain multiple frames of images collected in different moving directions during the movement of the self-moving robot.

[0095] Optionally, the one or more processors 901 are used to determine the vibration amplitude of the self-moving robot in different moving directions according to the average value of the pixel vertical coordinate difference of the corresponding feature points in the multiple frames of images.

[0096] Optionally, one or more processors 901 are used to determine the difference in pixel horizontal coordinates of corresponding feature points in the multiple frame images; filter out corresponding feature points whose pixel horizontal coordinate difference is greater than a threshold; and determine the vibration amplitude of the self-moving robot in different moving directions based on the average value of the pixel vertical coordinate difference of the remaining corresponding feature points in the multiple frame images.

[0097] Optionally, one or more processors 901 are used to determine the maximum or minimum vibration amplitude; determine the texture direction of the ground medium based on the maximum or minimum vibration amplitude; and determine the path direction of the self-moving robot based on the texture direction.

[0098] Optionally, one or more processors 901 are used to determine the moving direction corresponding to the minimum vibration amplitude as an alternative texture direction, or to determine the direction opposite to the moving direction corresponding to the maximum vibration amplitude as an alternative texture direction; and to determine the texture direction of the ground medium based on the average direction of multiple alternative texture directions.

[0099] Optionally, one or more processors 901 are used to determine a direction perpendicular to the texture direction as the path direction of the self-moving robot.

[0100] In the embodiment of the self-moving robot, a corresponding computer-readable storage medium may also be provided for the self-moving robot, which will not be described in detail here.

[0101] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0102] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0103] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0104] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0105] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0106] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0107] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0108] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0109] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A trajectory planning method, characterized in that: Applied to a self-moving robot, the method comprises: Acquire multiple frames of images collected by the self-moving robot while it is moving; Determining the vibration amplitude of the self-moving robot when it moves according to the pixel coordinates of the corresponding feature points in the multiple frames of images; Determining the path direction of the self-propelled robot according to the vibration amplitude specifically includes: determining the maximum or minimum vibration amplitude; determining the texture direction of the ground medium according to the maximum or minimum vibration amplitude; specifically includes: determining the moving direction corresponding to the minimum vibration amplitude as the candidate texture direction, or determining the opposite direction of the moving direction corresponding to the maximum vibration amplitude as the candidate texture direction; determining the texture direction of the ground medium according to the average direction of multiple candidate texture directions; Determining the path direction of the self-moving robot according to the texture direction; specifically comprising: determining a direction perpendicular to the texture direction as the path direction of the self-moving robot; The moving trajectory of the self-moving robot is planned according to the path direction.

2. The method according to claim 1, characterized in that Acquiring multiple frames of images collected by the self-moving robot while it is moving, including: If it is determined that the self-moving robot is located in a preset ground medium area, a plurality of frames of images collected by the self-moving robot while moving are acquired.

3. The method according to claim 2, characterized in that The preset ground medium area includes a carpet area.

4. The method according to claim 1 or 2, characterized in that: Acquiring multiple frames of images collected by the self-moving robot while it is moving, including: Controlling the self-propelled robot to move in different directions; Acquire multiple frames of images collected in different moving directions during the movement of the self-moving robot.

5. The method according to claim 4, characterized in that The step of determining the vibration amplitude of the self-moving robot when it moves according to the pixel coordinates of the corresponding feature points in the multiple frames of images includes: The vibration amplitudes of the self-moving robot in different moving directions are determined according to the average values ​​of the pixel ordinate differences of the corresponding feature points in the multiple frames of images.

6. The method according to claim 5, characterized in that The step of determining the vibration amplitude of the self-moving robot in different moving directions according to the average value of the difference of the pixel ordinates of the corresponding feature points in the multiple frames of images comprises: Determine the difference of the horizontal coordinates of the pixels of the corresponding feature points in the multiple frames of images; Filter out the corresponding feature points whose pixel horizontal coordinate difference is greater than the threshold; The vibration amplitude of the self-moving robot in different moving directions is determined according to the average value of the pixel ordinate difference of the corresponding feature points remaining in the multiple frames of images.

7. A trajectory planning device, characterized in that: Applied to a self-moving robot, the device comprises: An acquisition module, used to acquire multiple frames of images collected by the self-moving robot while it is moving; an amplitude determination module, for determining the vibration amplitude of the self-moving robot when it moves according to the pixel coordinates of the corresponding feature points in the multiple frames of images; A path determination module is used to determine the path direction of the self-propelled robot according to the vibration amplitude, specifically including: determining the maximum or minimum vibration amplitude; determining the texture direction of the ground medium according to the maximum or minimum vibration amplitude; determining the path direction of the self-propelled robot according to the texture direction; The trajectory planning module is used to plan the moving trajectory of the self-moving robot according to the path direction.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by one or more processors, the one or more processors are caused to perform the following actions: Acquire multiple frames of images collected by the mobile robot while it is moving; Determining the vibration amplitude of the self-moving robot when it moves according to the pixel coordinates of the corresponding feature points in the multiple frames of images; Determining the path direction of the self-moving robot according to the vibration amplitude specifically includes: determining the maximum or minimum vibration amplitude; determining the texture direction of the ground medium according to the maximum or minimum vibration amplitude; specifically including: determining the moving direction corresponding to the minimum vibration amplitude as the alternative texture direction, or determining the opposite direction of the moving direction corresponding to the maximum vibration amplitude as the alternative texture direction; determining the texture direction of the ground medium according to the average direction of multiple alternative texture directions; determining the path direction of the self-moving robot according to the texture direction; specifically including: determining the direction perpendicular to the texture direction as the path direction of the self-moving robot; The moving trajectory of the self-moving robot is planned according to the path direction.

9. A self-propelled robot, characterized in that: include: A mechanical body, wherein the mechanical body is provided with one or more processors and one or more memories storing computer programs; The one or more processors execute the computer program to: Acquire multiple frames of images collected by the self-moving robot while it is moving; Determining the vibration amplitude of the self-moving robot when it moves according to the pixel coordinates of the corresponding feature points in the multiple frames of images; Determining the path direction of the self-moving robot according to the vibration amplitude specifically includes: determining the maximum or minimum vibration amplitude; determining the texture direction of the ground medium according to the maximum or minimum vibration amplitude; specifically including: determining the moving direction corresponding to the minimum vibration amplitude as the alternative texture direction, or determining the opposite direction of the moving direction corresponding to the maximum vibration amplitude as the alternative texture direction; determining the texture direction of the ground medium according to the average direction of multiple alternative texture directions; determining the path direction of the self-moving robot according to the texture direction; specifically including: determining the direction perpendicular to the texture direction as the path direction of the self-moving robot; The moving trajectory of the self-moving robot is planned according to the path direction.

Images