Intelligent path planning method and device

By acquiring and constructing obstacle information and adjusting the perspective range, the problem of low path planning efficiency in the prior art is solved, and more efficient obstacle detection and path planning are achieved.

CN119043348APending Publication Date: 2024-11-29VICORETEK NANJING CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310594458.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-24
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

Existing path planning methods cannot construct more detailed target objects or obstacle information, and cannot perform path planning based on the real-time situation of obstacles, resulting in low detection efficiency.

Method used

By acquiring visual images and/or point cloud data, we judge and construct obstacle information. If it is not fully constructed, adjust the viewing angle range until the obstacle information is fully constructed, and then the optimal path is planned and executed.

Benefits of technology

It realizes real-time acquisition of more comprehensive obstacle models, improves path planning and detection efficiency, replaces fixed distance detection methods, and improves obstacle avoidance effects.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119043348A_ABST
    Figure CN119043348A_ABST
Patent Text Reader

Abstract

The invention relates to the field of artificial intelligence, and discloses an intelligent path planning method and equipment based on obstacle target detection, which can acquire more comprehensive model / information of a target object or an obstacle in real time and plan a detection path in real time according to the condition of the obstacle. The method comprises the following steps: the equipment advances according to a set route, acquires a visual image and / or point cloud data, judges whether an obstacle exists or not according to the visual image and / or point cloud data, and continues to advance according to the set route if the obstacle does not exist; if the obstacle exists, obstacle information is constructed according to the image and / or the point cloud data, whether the obstacle information is completely constructed or not is judged, and if the obstacle information is completely constructed, the equipment continues to advance according to the set route; and if the construction is not complete, the equipment moves to the barrier edge point of the current detected area and then retreats until the barrier edge point appears in the visible area of the equipment, and the equipment continues to construct the barrier information until the barrier information is completely constructed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and specifically to an intelligent path planning method and device. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the present application as recited in the claims. The description herein is not admitted to be prior art as disclosed by inclusion in this section.

[0003] Robot motion path planning is the arrangement of motion sequences designed for the robot to complete set tasks and actions under certain physical constraints. It is divided into global robot planning and local robot planning.

[0004] The current path planning methods are mainly:

[0005] 1. Plan a collision-avoiding route;

[0006] 2. Plan an efficient route to reach the destination.

[0007] The planned route is in the form of a route from a starting point to an end point.

[0008] These path planning methods cannot construct more detailed target or obstacle information. Even if there are some path planning methods that aim to detect obstacles, they simply detect at a fixed distance, and do not plan the next detection path in real time based on the detected obstacles. Summary of the invention

[0009] The purpose of this application is to provide an intelligent path planning method and device, which can obtain a more comprehensive model / information of a target object or obstacle in real time and plan the detection path in real time according to the situation of the obstacle.

[0010] The present application discloses an intelligent path planning method, comprising:

[0011] The device moves forward along a predetermined route, obtains visual images and / or point cloud data, and determines whether there are obstacles based on the visual images and / or point cloud data. If there are no obstacles, the device continues to move forward along the predetermined route;

[0012] If there is an obstacle, construct obstacle information according to the image and / or point cloud data and determine whether the obstacle information is completely constructed. If it is completely constructed, the device continues to move along the predetermined route;

[0013] If it is not fully constructed, the device moves to the obstacle edge point of the currently detected area and then retreats until the obstacle edge point appears in the visible area of ​​the device, and the device continues to construct the obstacle information until the obstacle information is fully constructed.

[0014] In a preferred example, the device continues to construct obstacle information until the obstacle information is completely constructed, further comprising:

[0015] The device plans a route for continuing to move forward and exploring the obstacle;

[0016] The device moves to the obstacle edge point of the currently detected area according to the planned route;

[0017] The device rotates and plans the shortest route for the edge point of the obstacle to enter the visual range of the device;

[0018] The device continues to move along the edge of the obstacle until the obstacle information is completely constructed.

[0019] In a preferred embodiment, it includes:

[0020] The device avoids the obstacle and performs a predetermined action of the device during the forward movement;

[0021] The device constructs the obstacle model while detecting the obstacle from different directions when executing the predetermined action of the device.

[0022] In a preferred embodiment, it includes:

[0023] The device moves a fixed distance along the edge of the obstacle;

[0024] The device rotates backward to make the obstacle enter the visual range of the device to construct the obstacle information;

[0025] The device continues to walk a fixed distance along the edge of the obstacle until the obstacle information is completely constructed.

[0026] In a preferred example, the point cloud data are points on the surface of the obstacle observed by a camera.

[0027] In a preferred example, the visual image detection range of the device is 15cm-100cm.

[0028] In a preferred example, the device detects the visual image through binocular vision.

[0029] In a preferred example, the device detects the visual image through a depth camera.

[0030] The present application also discloses an intelligent path planning device, comprising:

[0031] a memory for storing computer executable instructions; and,

[0032] A processor, coupled to the memory, is configured to implement the steps in the method described above when executing the computer executable instructions.

[0033] The present application also discloses a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the steps in the method described above are implemented.

[0034] In the implementation manner of the present application, a path planning method is provided for detecting obstacles or target objects, which can enable the device to obtain a more comprehensive model / information of the target object or obstacle in real time, and plan the detection path in real time according to the situation of the obstacle. If the model / information is not fully constructed, the device will adjust the viewing angle range only after detecting the edge point of the target object or obstacle so that the target object or obstacle reappears in the visible range and continues to detect obstacles, replacing the conventional method of detecting at a fixed distance, thereby improving the detection and planning efficiency.

[0035] Each technical feature disclosed in the above invention content, each technical feature disclosed in each implementation mode and example below, and each technical feature disclosed in the accompanying drawings can be freely combined with each other to form various new technical solutions (these technical solutions should be deemed to have been recorded in this specification), unless such combination of technical features is technically infeasible. For example, in one example, feature A+B+C is disclosed, and in another example, feature A+B+D+E is disclosed, and features C and D are equivalent technical means that play the same role. Technically, only one of them can be used, and it is impossible to use them at the same time. Feature E can be combined with feature C technically. Then, the solution of A+B+C+D should not be deemed to have been recorded because it is technically infeasible, while the solution of A+B+C+E should be deemed to have been recorded. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flow chart according to an embodiment of the present application;

[0037] Figure 2 is a flow chart according to an embodiment of the present application;

[0038] Figure 3 is a flow chart according to an embodiment of the present application;

[0039] Figure 4 is a schematic diagram of the device according to one embodiment of the present application;

[0040] Figure 5 is a schematic diagram of the device according to one embodiment of the present application;

[0041] Figure 6 is a schematic diagram of the device according to one embodiment of the present application;

[0042] Figure 7 is a schematic diagram of the device according to one embodiment of the present application;

[0043] Figure 8 is a schematic diagram of the device according to one embodiment of the present application;

[0044] Fig. 9 is a schematic diagram of the device according to one embodiment of the present application;

[0045] Fig.10 is a schematic diagram of the device according to one embodiment of the present application;

[0046] Fig.11 is a schematic diagram of a device route according to one embodiment of the present application;

[0047] Fig.12 is a schematic diagram of a device route according to one embodiment of the present application;

[0048] Fig.13 is a schematic diagram of the device according to one embodiment of the present application;

[0049] Fig.14 is a schematic diagram of the device according to one embodiment of the present application;

[0050] Fig.15 is a schematic diagram of a device route according to one embodiment of the present application;

[0051] Fig.16 It is a flowchart according to one embodiment of the present application. DETAILED DESCRIPTION

[0052] In the following description, many technical details are provided to help readers better understand the present application. However, those skilled in the art can understand that the technical solution claimed in the present application can be implemented even without these technical details and various changes and modifications based on the following embodiments.

[0053] Explanation of terms:

[0054] Point cloud: a collection of points in three-dimensional space; composed of N points of D dimension, when D = 3, it can be represented as a three-dimensional coordinate point (x, y, z), each point is determined by a certain (xyz) position, and other attributes (such as RGB color) can be assigned to it. They are the original form of LiDAR data when it is acquired. Stereo vision systems and RGB-D data (containing images marked with depth values ​​for each pixel) are usually converted into point clouds before further processing.

[0055] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0056] The first embodiment of the present application relates to an intelligent path planning method based on obstacle target detection, the process of which is as follows: Figure 1 As shown, the method comprises the following steps:

[0057] In step S101, the device moves along a predetermined route, obtains visual images and / or point cloud data, and determines whether there are obstacles based on the visual images and / or point cloud data. If there are no obstacles, the device continues to move along the predetermined route.

[0058] In step S102, if there is an obstacle, the obstacle information is constructed based on the image and / or point cloud data and it is determined whether the obstacle information is completely constructed. If it is completely constructed, the device continues to move along the established route.

[0059] In step S103, if the obstacle information is not fully constructed, the device moves to the obstacle edge point of the currently detected area and then moves back until the obstacle edge point appears in the visible area of ​​the device, and the device continues to construct the obstacle information until the obstacle information is fully constructed.

[0060] In an optional embodiment, step S103 may further include:

[0061] In step S1031, the device plans a route to continue moving forward and explore obstacles.

[0062] In step S1032, the device moves to the obstacle edge point of the currently detected area according to the planned route.

[0063] In step S1033, the device rotates and plans the shortest route to bring the edge point of the obstacle into the visual range of the device.

[0064] In step S1034, the device continues to move along the edge of the obstacle until the obstacle information is completely constructed.

[0065] In order to better understand the technical solution of the present application, a specific example is provided below for illustration. The details listed in the example are mainly for ease of understanding and are not intended to limit the scope of protection of the present application.

[0066] like Figure 3 As shown, when the device moves forward, the following operations are continuously performed:

[0067] In step S301, the device acquires visual images and / or point cloud data.

[0068] In step S302, the device determines whether there is an obstacle based on the visual image and / or point cloud data. If there is an obstacle, the process proceeds to step S303; if there is no obstacle, the process proceeds to step S307.

[0069] In step S303, the device constructs obstacle information based on the point cloud data (if the obstacle has not been detected before, the point cloud information is the point on the surface of the obstacle observed by the camera, generally the edge of the obstacle facing the camera).

[0070] In step S304, it is determined whether the obstacle detection is complete or the obstacle information is complete. If not, the process proceeds to step S305; if yes, the process proceeds to step S307.

[0071] In step S305 , a plan is made to move the device to an area where the detection area includes the front edge of the undetected object.

[0072] In step S306, the device moves along the newly planned route and returns to step S301 to continue acquiring visual images and / or point cloud data.

[0073] In step S307, the device continues to move forward and determines whether it has reached the target point. If it has not reached the target point, it returns to step S301 to continue acquiring visual images and / or point cloud data; if it has reached the target point, the process ends.

[0074] In an optional embodiment, step S305 may include two situations:

[0075] A) The device performs path planning when coverage is required;

[0076] B) The device performs path planning without requiring coverage.

[0077] A) When the device is planning a path with coverage requirements, such as Figure 4As shown in the figure, it is necessary to consider making the device move along the edge of the obstacle as much as possible. The device moves forward to see the obstacle, and the dark gray narrow band part of the object to be detected represents the obstacle model or map position that has been detected and constructed. Due to the perspective problem, only one side of the blue area can be seen. For cleaning efficiency, the device plans a route to continue moving forward and explore obstacles. Figure 5 As shown in , the device moves to the side of the unexplored obstacle (or the nearest position), and then knows that the obstacle has not been explored, and rotates. Figure 6 As shown in the figure, after the device rotates, the obstacle is in the blind spot of the device. Figure 7 As shown, the device plans the shortest route for the obstacle to enter the visual range, preferably backward. Figure 8 As shown in the figure, the device continues to move along the edge of the obstacle, and the dark gray narrow band of the object to be detected represents the obstacle model or map position that has been detected and constructed. Fig. 9 As shown in , the device moves to the next unexplored obstacle (or the nearest location). Fig.10 As shown, the device rotates and plans the shortest route for the obstacle to enter the visual range, preferably backwards. Repeat the above operation until the entire obstacle is visible.

[0078] The equipment route diagram is as follows Fig.11 As shown, the arrows in the figure are the directions in which the device moves, and the device moves backward in routes 3, 6, and 9. This figure is only a schematic diagram of the walking route to ensure coverage when the device just passes by an obstacle. If the obstacle is large, the device may need to move forward and backward multiple times to build the entire obstacle model.

[0079] The route diagram of the device can also be Fig.12 As shown, the arrows in the figure are the directions in which the device moves. In routes 3, 6, and 9, the device moves backward. This figure is only a schematic diagram of the movement of the device to ensure coverage when it is facing an obstacle. If the obstacle is large, the device may need to move forward and backward multiple times to build the entire obstacle model.

[0080] In an optional embodiment, the device may first detect obstacles from a distance and then move along the edge.

[0081] B) When the device performs path planning without requiring coverage, such as Fig.13 As shown in , the device finds an obstacle and moves directly to the next observation point where it can observe the unknown area. Fig.14 As shown, the device rotates at the observation point to observe the obstacle. Repeat the above steps until the entire obstacle is detected.

[0082] The equipment route diagram is as follows Fig.15 As shown, the straight arrow in the figure is the direction in which the device moves, and the curved arrow in the figure represents the rotation of the device.

[0083] In an optional embodiment, the present implementation may further include the following steps:

[0084] In step S1601, the device walks a fixed distance along the edge of the obstacle.

[0085] In step S1602, the device rotates backward to make the obstacle enter the visible range of the device to construct obstacle information.

[0086] In step S1603, the device continues to walk a fixed distance along the edge of the obstacle until the obstacle information is completely constructed.

[0087] In an optional embodiment, the device may plan a route through an algorithm, which may include but is not limited to A*, D*, Dijkstra, artificial potential field, resource points, etc.

[0088] In an optional embodiment, the visual image detection range of the device may be 15cm-100cm, and may also be adjusted according to the visible range of different cameras. For example, the detection range may be 10cm-200cm, or 50cm-150cm, and so on.

[0089] In an optional embodiment, the device may detect visual images using, including but not limited to, binocular vision, or may use other depth cameras for detection.

[0090] In an optional embodiment, the device may also bypass obstacles and perform predetermined actions of the device while moving forward, and construct an obstacle model while detecting obstacles from different directions when performing predetermined actions of the device.

[0091] The main purpose of this application is to enable robots or equipment to obtain more comprehensive models / information of target objects or obstacles in real time, and to propose a path planning method for real-time planning and detection based on the situation of obstacles. It can also quickly build obstacle information as needed (such as building maps), or build obstacle information while ensuring robot coverage (such as when a cleaning robot is cleaning), so as to achieve better obstacle avoidance effects with high efficiency or high coverage.

[0092] The second embodiment of the present application relates to an intelligent path planning device, which includes: a memory for storing computer executable instructions; and a processor, coupled to the memory, for implementing the steps in the above-mentioned method implementations when executing computer executable instructions. Among them, the processor can be a central processing unit (Central Processing Unit, referred to as "CPU"), a graphic processing unit (Graphic Processing Unit, referred to as "GPU"), a digital signal processor (Digital Signal Processor, referred to as "DSP"), a microcontroller unit (Microcontroller Unit, referred to as "MCU"), a neural network processor (referred to as "NPU"), an application specific integrated circuit (Application Specific Integrated Circuit, referred to as "ASIC"), a field programmable gate array (Field Programmable Gate Array, referred to as "FPGA") or other programmable logic devices. The aforementioned memory can be a read-only memory (read-only memory, referred to as "ROM"), a random access memory (random access memory, referred to as "RAM"), a flash memory (Flash), a hard disk or a solid state drive, etc. The steps of the method disclosed in each embodiment of the present invention can be directly embodied as a hardware processor to be executed, or a combination of hardware and software modules in the processor to be executed.

[0093] The first implementation manner is a method implementation manner corresponding to the present implementation manner. The technical details in the first implementation manner can be applied to the present implementation manner, and the technical details in the present implementation manner can also be applied to the first implementation manner.

[0094] It should be noted that those skilled in the art should understand that the implementation functions of each module shown in the implementation of the above-mentioned intelligent path planning device can be understood with reference to the relevant description of the above-mentioned intelligent path planning method. The functions of each module shown in the implementation of the above-mentioned intelligent path planning device can be implemented by a program (executable instruction) running on a processor, or by a specific logic circuit. If the above-mentioned intelligent path planning method of the embodiment of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM, Read Only Memory), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0095] Accordingly, the embodiments of the present application also provide a computer-readable storage medium, in which computer executable instructions are stored, and when the computer executable instructions are executed by the processor, the various method embodiments of the present application are implemented. Computer-readable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be a computer-readable instruction, a data structure, a module of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include temporary computer-readable media (transitory media), such as modulated data signals and carriers.

[0096] It should be noted that, in the present application, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements defined by the sentence "include one" do not exclude the existence of other identical elements in the process, method, article or equipment including the elements. In the present application, if it is mentioned that a certain action is performed according to a certain element, it means at least the meaning of performing the action according to the element, which includes two situations: performing the action only according to the element and performing the action according to the element and other elements. Multiple, multiple, multiple, etc. expressions include 2, 2 times, 2 kinds and more than 2, more than 2 times, more than 2 kinds.

[0097] The serial numbers used in describing the steps of the method do not themselves constitute any limitation on the order of these steps. For example, the step with a larger serial number does not necessarily have to be executed after the step with a smaller serial number. The step with a larger serial number may be executed first and then the step with a smaller serial number. They may also be executed in parallel, as long as this execution order is reasonable for those skilled in the art. For another example, multiple steps with consecutive serial numbers (e.g., step 101, step 102, step 103, etc.) do not limit other steps that can be executed in between. For example, there may be other steps between step 101 and step 102.

[0098] This specification includes combinations of the various embodiments described herein. Individual references to embodiments (e.g., "one embodiment" or "some embodiments" or "preferred embodiments"); however, these embodiments are not mutually exclusive unless indicated as mutually exclusive or it is clear to a person skilled in the art that they are mutually exclusive. It should be noted that the word "or" is used in this specification in a non-exclusive sense unless the context clearly indicates or requires otherwise.

[0099] In addition, it should be understood that the above are only preferred embodiments of this specification and are not intended to limit the protection scope of this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of one or more embodiments of this specification should be included in the protection scope of one or more embodiments of this specification.

[0100] In some cases, the actions or steps described in the claims may be performed in a different order than in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. An intelligent path planning method, characterized in that: include: The device moves forward along a predetermined route, obtains visual images and / or point cloud data, and determines whether there are obstacles based on the visual images and / or point cloud data. If there are no obstacles, the device continues to move forward along the predetermined route; If there is an obstacle, construct obstacle information according to the image and / or point cloud data and determine whether the obstacle information is completely constructed. If it is completely constructed, the device continues to move along the predetermined route; If it is not fully constructed, the device moves to the obstacle edge point of the currently detected area and then retreats until the obstacle edge point appears in the visible area of ​​the device, and the device continues to construct the obstacle information until the obstacle information is fully constructed.

2. The intelligent path planning method according to claim 1, characterized in that: The device continues to construct the obstacle information until the obstacle information is completely constructed, further comprising: The device plans a route for continuing to move forward and exploring the obstacle; The device moves to the obstacle edge point of the currently detected area according to the planned route; The device rotates and plans the shortest route for the edge point of the obstacle to enter the visual range of the device; The device continues to move along the edge of the obstacle until the obstacle information is completely constructed.

3. The intelligent path planning method according to claim 1, characterized in that: include: The device avoids the obstacle and performs a predetermined action of the device during the forward movement; The device constructs the obstacle model while detecting the obstacle from different directions when executing the predetermined action of the device.

4. The intelligent path planning method according to claim 1, characterized in that: include: The device moves a fixed distance along the edge of the obstacle; The device rotates backward to make the obstacle enter the visual range of the device to construct the obstacle information; The device continues to walk a fixed distance along the edge of the obstacle until the obstacle information is completely constructed.

5. The intelligent path planning method according to claim 1, characterized in that: The point cloud data are points on the surface of the obstacle observed by the camera.

6. The intelligent path planning method according to claim 1, characterized in that: The visual image detection range of the device is 15cm-100cm.

7. The intelligent path planning method according to claim 1, characterized in that: The device detects the visual image through binocular vision.

8. The intelligent path planning method according to claim 1, characterized in that: The device detects the visual image through a depth camera.

9. An intelligent path planning device, characterized in that: include: A memory for storing computer executable instructions; as well as, A processor, coupled to the memory, configured to implement the steps of the method according to any one of claims 1 to 8 when executing the computer executable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps in the method according to any one of claims 1 to 8 are implemented.