Mechanical arm path planning method and system

Abstract

The method comprises the following steps: acquiring three-dimensional point cloud information corresponding to an object to be scanned, the three-dimensional point cloud information being obtained by performing profiling scanning on the object to be scanned by an image acquisition device according to a scanning mode; processing the three-dimensional point cloud information according to a preset distance to obtain robot arm running three-dimensional point cloud information, the preset distance being a safety distance between the robot arm and the object to be scanned; rasterizing the robot arm running three-dimensional point cloud information to obtain raster three-dimensional point cloud information; processing the raster three-dimensional point cloud information that meets a first preset condition according to an order of acquiring the three-dimensional point cloud information corresponding to the scanning mode to obtain a key point, the first preset condition being that a longitudinal coordinate of the raster three-dimensional point cloud information meets a preset threshold; and performing path planning according to the key point to obtain a robot arm running path.

Description

Technical Field

[0001] This disclosure relates to the fields of mechanical technology and artificial intelligence technology, and in particular to a method and system for path planning of a robotic arm. Background Technology

[0002] Work equipment (such as detection, welding, etc.) is installed at the end of the robotic arm. A vision system is used to create a 3D model of the surface of the scanned object. Based on the 3D model of the surface of the scanned object, a path planning algorithm is used to complete the 3D contour detection of the surface of the scanned object.

[0003] In robotic arms performing detection and welding operations, the operating path of the robotic arm is similar to the path of scanning an object to obtain a 3D point cloud. However, the existing method is to first scan the object and then plan the path of the robotic arm based on the scan information, which cannot meet the requirement that the robotic arm can complete detection and welding tasks during the scanning process. Summary of the Invention

[0004] In view of the above problems, this disclosure provides a method and system for robotic arm path planning.

[0005] According to the first aspect of this disclosure, a robotic arm path planning method is provided, comprising: acquiring three-dimensional point cloud information corresponding to an object to be scanned, wherein the three-dimensional point cloud information is obtained by a contour scanning of the object to be scanned using an image acquisition device in a scanning manner; processing the three-dimensional point cloud information according to a preset distance to obtain three-dimensional point cloud information of robotic arm operation, wherein the preset distance is a safe distance between the robotic arm and the object to be scanned; rasterizing the three-dimensional point cloud information of robotic arm operation to obtain rasterized three-dimensional point cloud information; processing the rasterized three-dimensional point cloud information that meets a first preset condition according to the order of acquiring the three-dimensional point cloud information corresponding to the scanning method to obtain key points, wherein the first preset condition is that the ordinate of the rasterized three-dimensional point cloud information meets a preset threshold; and performing path planning based on the key points to obtain a robotic arm operation path, wherein the robotic arm operation path is used for the robotic arm to detect or weld the object to be scanned.

[0006] According to embodiments of this disclosure, processing raster 3D point cloud information that meets a first preset condition to obtain key points, in accordance with the order in which the 3D point cloud information is acquired corresponding to the scanning method, includes: extracting key points to be processed from the raster 3D point cloud information according to the first preset condition; and sorting the key points to be processed in accordance with the order in which the 3D point cloud information is acquired corresponding to the scanning method to obtain key points.

[0007] According to embodiments of this disclosure, the key point shown includes at least one sub-key point. Path planning based on the key point yields a robotic arm running path comprising: using the (N-1)th sub-key point as the root node of a first tree; expanding the first tree based on its root node to obtain new nodes; using the Nth sub-key point as the root node of a second tree; expanding the second tree based on its root node to obtain new nodes; and determining a robotic arm running sub-path from the (N-1)th sub-key point to the Nth sub-key point when the new nodes of the first and second trees satisfy a second preset condition.

[0008] According to embodiments of this disclosure, the second preset condition includes a step size constraint and a collision-free path condition. When the new nodes of the first and second trees satisfy the second preset condition, determining the robotic arm's sub-path from the (N-1)th sub-keypoint to the Nth sub-keypoint includes: obtaining a first sub-path from the (N-1)th sub-keypoint to the Nth sub-keypoint when the new nodes of the first and second trees satisfy the step size constraint; obtaining a second sub-path when the first sub-path satisfies the collision-free path condition; interpolating the second sub-path to obtain a third sub-path from the (N-1)th sub-keypoint to the Nth sub-keypoint; and smoothing the third sub-path using a mean algorithm to obtain the robotic arm's sub-path from the (N-1)th sub-keypoint to the Nth sub-keypoint.

[0009] According to an embodiment of this disclosure, the scanning method is an arc scanning method. The arc scanning method scans the object to be scanned from a first preset direction, then continues to scan the object to be scanned from a second preset direction by a unit distance, and then continues to scan the object to be scanned from the first preset direction. The above steps are repeated until the object to be scanned is completely scanned.

[0010] According to embodiments of this disclosure, the bow-shaped scanning method includes the xyx method and the yxy method. In the xyx method, the first preset direction is the horizontal coordinate and the second preset direction is the vertical coordinate. In the yxy method, the first preset direction is the vertical coordinate and the second preset direction is the horizontal coordinate.

[0011] According to embodiments of this disclosure, before acquiring the 3D point cloud information corresponding to the object to be scanned, the method further includes: calibrating the position of the image acquisition device and the position of the robotic arm to obtain calibration information. Based on the calibration information, determining the spatial relative positional relationship between the image acquisition device and the robotic arm. Based on the spatial relative positional relationship, transforming the object coordinate system with the image acquisition device as the origin to obtain an object coordinate system with the robotic arm as the origin.

[0012] The second aspect of this disclosure provides a path planning system, comprising an image acquisition device, a robotic arm, and a processor. The image acquisition device performs contour scanning of the object to be scanned using a scanning method to obtain three-dimensional point cloud information. The processor, connected to the image acquisition device, receives the three-dimensional point cloud information sent by the image acquisition device and executes the aforementioned robotic arm path planning method to process the three-dimensional point cloud information to obtain the robotic arm's running path. The robotic arm, connected to the processor, receives the robotic arm's running path sent by the processor and performs detection or welding work according to the robotic arm's running path.

[0013] According to embodiments of this disclosure, the image acquisition device does not contact the robotic arm, and the height of the acquisition device is greater than the height of the robotic arm. Alternatively, the acquisition device is mounted at the end of the robotic arm.

[0014] According to embodiments of this disclosure, when an image acquisition device receives a scanning request sent by a processor, it performs contour scanning of the object to be scanned according to the scanning method in the scanning request.

[0015] A third aspect of this disclosure provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the methods described above.

[0016] A fourth aspect of this disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the methods described above.

[0017] The fifth aspect of this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0018] According to the robotic arm path planning method and system provided in this disclosure, the object to be scanned is scanned using a contour scan according to the scanning method specified in the scanning request, thereby obtaining three-dimensional point cloud information corresponding to the object. Then, based on the three-dimensional point cloud information and a preset distance, three-dimensional point cloud information of the robotic arm operation is obtained; this three-dimensional point cloud information is then rasterized to obtain rasterized three-dimensional point cloud information. Key points are extracted from the rasterized three-dimensional point cloud information according to the scanning method and a first preset condition; path planning is then performed based on these key points to obtain the robotic arm's operating path. This enables the robotic arm to complete tasks such as detection and welding during the scanning process, improving work efficiency. Attached Figure Description

[0019] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0020] Figure 1 This diagram schematically illustrates an application scenario of the robotic arm path planning method according to embodiments of the present disclosure.

[0021] Figure 2 A flowchart illustrating a robotic arm path planning method according to an embodiment of the present disclosure is shown schematically.

[0022] Figure 3 A schematic diagram illustrating the logic of a robotic arm path planning method according to an embodiment of the present disclosure is shown.

[0023] Figure 4 A schematic diagram illustrating the xyx mode according to an embodiment of the present disclosure is shown;

[0024] Figure 5 A schematic diagram of a yxy configuration according to an embodiment of the present disclosure is shown;

[0025] Figure 6 This schematically illustrates an image acquisition device according to an embodiment of the present disclosure that is not in contact with a robotic arm;

[0026] Figure 7 A schematic diagram illustrating a data acquisition device according to an embodiment of the present disclosure mounted on the end of a robotic arm; and

[0027] Figure 8 A block diagram schematically illustrates an electronic device suitable for implementing a robotic arm path planning method according to an embodiment of the present disclosure. Detailed Implementation

[0028] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0029] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0030] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0031] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0032] In the technical solution of this invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0033] In robotic arms performing detection and welding operations, the operating path of the robotic arm is similar to the path of scanning an object to obtain a 3D point cloud. However, the existing method is to first scan the object and then plan the path of the robotic arm based on the scan information, which cannot meet the requirement that the robotic arm can complete detection and welding tasks during the scanning process.

[0034] This disclosure provides a robotic arm path planning method. The method acquires 3D point cloud information corresponding to an object to be scanned. The 3D point cloud information is obtained by a contour scan of the object using an image acquisition device. The 3D point cloud information is processed according to a preset distance to obtain robotic arm operation 3D point cloud information. The preset distance is a safe distance between the robotic arm and the object to be scanned. The robotic arm operation 3D point cloud information is rasterized to obtain rasterized 3D point cloud information. Following the order of acquiring the 3D point cloud information corresponding to the scanning method, the rasterized 3D point cloud information that meets a first preset condition is processed to obtain key points. The first preset condition is that the ordinate of the rasterized 3D point cloud information meets a preset threshold. Path planning is performed based on the key points to obtain the robotic arm's operating path. The robotic arm's operating path is used for the robotic arm's detection or welding work on the object to be scanned.

[0035] Figure 1 The diagram illustrates an application scenario of the robotic arm path planning method according to an embodiment of the present disclosure.

[0036] like Figure 1As shown, application scenario 100 according to this embodiment may include a robotic arm 101, an image acquisition device 102, a third terminal device 103, a network 104, and a processor 105. The network 104 serves as a medium for providing a communication link between the robotic arm 101, the image acquisition device 102, the third terminal device 103, and the processor 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0037] Users can interact with processor 105 via network 104 using at least one of robotic arm 101, image acquisition device 102, and third terminal device 103 to receive or send messages, etc. Various communication client applications can be installed on robotic arm 101, image acquisition device 102, and third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0038] The robotic arm 101, image acquisition device 102, and third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0039] The processor 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the robotic arm 101, image acquisition device 102, and third terminal device 103 (for example only). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0040] It should be noted that the robotic arm path planning method provided in this embodiment can generally be executed by the processor 105. Correspondingly, the robotic arm path planning method apparatus provided in this embodiment can generally be housed within the processor 105. The robotic arm path planning method provided in this embodiment can also be executed by a server or server cluster that is different from the processor 105 and capable of communicating with the robotic arm 101, the image acquisition device 102, the third terminal device 103, and / or the processor 105. Correspondingly, the robotic arm path planning method apparatus provided in this embodiment can also be housed in a server or server cluster that is different from the processor 105 and capable of communicating with the robotic arm 101, the image acquisition device 102, the third terminal device 103, and / or the processor 105.

[0041] It should be understood that Figure 1The number of robotic arms, image acquisition devices, terminal equipment, networks, and servers shown is merely illustrative. Depending on the implementation requirements, any number of terminal equipment, networks, and servers can be included.

[0042] Figure 2 A flowchart illustrating a robotic arm path planning method according to an embodiment of the present disclosure is shown schematically.

[0043] like Figure 2 As shown, the robotic arm path planning method of this embodiment includes operations S210 to S250.

[0044] Operation S210 acquires the 3D point cloud information corresponding to the object to be scanned.

[0045] When operating S220, the three-dimensional point cloud information is processed according to the preset distance to obtain the three-dimensional point cloud information of the robotic arm operation.

[0046] By operating S230, the three-dimensional point cloud information of the robotic arm is rasterized to obtain raster three-dimensional point cloud information.

[0047] In operation S240, following the order of acquiring 3D point cloud information corresponding to the scanning method, the raster 3D point cloud information that meets the first preset condition is processed to obtain key points.

[0048] When operating the S250, path planning is performed based on key points to obtain the robotic arm's running path.

[0049] According to embodiments of this disclosure, the three-dimensional point cloud information is obtained by a scanning device performing contour scanning on the object to be scanned. For example, the three-dimensional point cloud information can be obtained by performing contour scanning on the object to be scanned in a scanning manner to obtain the three-dimensional point cloud information corresponding to the scanned portion of the object to be scanned. Alternatively, the three-dimensional point cloud information can be obtained by performing contour scanning on the object to be scanned in a scanning manner to obtain the three-dimensional point cloud information of the entire object to be scanned.

[0050] According to embodiments of this disclosure, the preset distance is a safe distance between the robotic arm and the object to be scanned.

[0051] According to embodiments of this disclosure, the first preset condition is that the ordinate of the raster 3D point cloud information meets a preset threshold. For example, the preset threshold may be the raster 3D point cloud information with the largest ordinate within a single raster.

[0052] According to embodiments of this disclosure, the robotic arm's operating path is used for the robotic arm to detect or weld objects to be scanned.

[0053] According to embodiments of this disclosure, the acquired 3D point cloud information can be partial or complete 3D point cloud information of the object to be scanned. Based on the safe distance between the robotic arm and the object to be scanned, the 3D point cloud information is processed to obtain 3D point cloud information of the robotic arm's operation in the area that the robotic arm can safely probe or weld. Rasterizing the 3D point cloud information of the robotic arm's operation allows for the division of the 3D point cloud information into regions, facilitating the selection of rasterized 3D point cloud information that meets the first preset condition.

[0054] According to embodiments of this disclosure, the acquired 3D point cloud information can be partial 3D point cloud information of the object to be scanned. Following the order in which the 3D point cloud information is acquired corresponding to the scanning method, the grid 3D point cloud information that meets the first preset condition is processed to obtain key points. Using MoveIt, path planning can be performed based on the key points to obtain the robotic arm's operating path, enabling the robotic arm to complete tasks such as detection and welding during the scanning process.

[0055] According to the robotic arm path planning method and system provided in this disclosure, the object to be scanned is scanned using a contour scan according to the scanning method specified in the scanning request, thereby obtaining three-dimensional point cloud information corresponding to the object. Then, based on the three-dimensional point cloud information and a preset distance, three-dimensional point cloud information of the robotic arm operation is obtained; this three-dimensional point cloud information is then rasterized to obtain rasterized three-dimensional point cloud information. Key points are extracted from the rasterized three-dimensional point cloud information according to the scanning method and a first preset condition; path planning is then performed based on these key points to obtain the robotic arm's operating path. This enables the robotic arm to complete tasks such as detection and welding during the scanning process, improving work efficiency.

[0056] According to embodiments of this disclosure, processing raster 3D point cloud information that meets a first preset condition to obtain key points, in accordance with the order in which the 3D point cloud information is acquired corresponding to the scanning method, includes: extracting key points to be processed from the raster 3D point cloud information according to the first preset condition; and sorting the key points to be processed in accordance with the order in which the 3D point cloud information is acquired corresponding to the scanning method to obtain key points.

[0057] According to embodiments of this disclosure, by sorting the key points to be processed according to the order in which the 3D point cloud information is acquired, corresponding to the scanning method, the key points can be obtained. This allows for simultaneous scanning operations, and path planning based on the 3D point cloud information corresponding to the scanned portion of the object to be scanned, resulting in a robotic arm running path. The robotic arm then performs detection or welding work on the scanned portion of the object according to the robotic arm running path. Similarly, path planning can be performed after scanning the entire object to be scanned.

[0058] Figure 3 The illustration shows a logical diagram of a robotic arm path planning method according to an embodiment of the present disclosure.

[0059] like Figure 3 As shown, the robotic arm path planning method of this embodiment includes operations S310 to S380.

[0060] In operation S310, the (N-1)th sub-key point is taken as the root node of the first tree. Based on the root node of the first tree, the first tree is searched and expanded to obtain new nodes of the first tree.

[0061] In operation S320, the Nth sub-key point is taken as the root node of the second tree. Based on the root node of the second tree, the second tree is searched and expanded to obtain new nodes of the second tree.

[0062] In operation S330, check whether the new nodes of the first tree and the second tree meet the step size constraint. If yes, execute operation S340; otherwise, execute operation S310.

[0063] In operation S340, the first sub-path from the (N-1)th sub-keypoint to the Nth sub-keypoint is obtained.

[0064] In operation S350, check if the first sub-path meets the collision-free path condition. If yes, execute operation S360; otherwise, execute operation S310.

[0065] By operating S360, the second sub-path is obtained.

[0066] In operation S370, interpolation is performed on the second sub-path to obtain the third sub-path from the (N-1)th sub-keypoint to the Nth sub-keypoint.

[0067] During operation of S380, the third sub-path is smoothed using the mean algorithm to obtain the robotic arm running sub-path from the (N-1)th sub-key point to the Nth sub-key point.

[0068] According to embodiments of this disclosure, the key point shown includes at least one sub-key point.

[0069] According to embodiments of this disclosure, the second preset condition includes a step size limit condition and a collision-free path condition.

[0070] According to embodiments of this disclosure, based on the root node of the first tree, a search expansion of the first tree can be performed to obtain a new node of the first tree. The new node of the first tree can be randomly selected. It can also be randomly selected. It is determined whether the new node of the first tree and the new node of the second tree satisfy a second preset condition. If not, a further search expansion can be performed on the first and second trees to randomly obtain new nodes of both trees.

[0071] According to an embodiment of this disclosure, the scanning method is an arc scanning method. The arc scanning method scans the object to be scanned from a first preset direction, then continues to scan the object to be scanned from a second preset direction by a unit distance, and then continues to scan the object to be scanned from the first preset direction. The above steps are repeated until the object to be scanned is completely scanned.

[0072] According to embodiments of this disclosure, the bow-shaped scanning method includes the xyx method and the yxy method. In the xyx method, the first preset direction is the horizontal coordinate and the second preset direction is the vertical coordinate. In the yxy method, the first preset direction is the vertical coordinate and the second preset direction is the horizontal coordinate.

[0073] Figure 4 A schematic diagram of the xyx mode according to an embodiment of the present disclosure is shown.

[0074] Figure 5 A schematic diagram of the yxy mode according to an embodiment of the present disclosure is shown.

[0075] The image acquisition device provides the three-dimensional information of the surface of the object to be scanned, and combined with visible light information, extracts the three-dimensional point cloud data of the surface of the object. For example... Figures 4 to 5 As shown, contour scanning of the object to be scanned is performed using an xyx or yxy method, which can smooth the scanning path. Three-dimensional point cloud information corresponding to the object to be scanned is acquired. Based on a preset distance, the three-dimensional point cloud information is processed to obtain the three-dimensional point cloud information of the robotic arm operation. This robotic arm operation three-dimensional point cloud information is rasterized to obtain raster three-dimensional point cloud information. The raster three-dimensional point cloud information in the scanning path that meets the first preset condition is identified as several key points. MoveIt can be used for path planning between these key points to ensure smooth robotic arm movement during scanning and / or welding operations.

[0076] According to embodiments of this disclosure, before acquiring the 3D point cloud information corresponding to the object to be scanned, the method further includes: calibrating the position of the image acquisition device and the position of the robotic arm to obtain calibration information. Based on the calibration information, determining the spatial relative positional relationship between the image acquisition device and the robotic arm. Based on the spatial relative positional relationship, transforming the object coordinate system with the image acquisition device as the origin to obtain an object coordinate system with the robotic arm as the origin.

[0077] First, the positions of the camera and the robotic arm need to be calibrated to determine their spatial relative positional relationship, providing a coordinate transformation method for subsequent hand-eye coordination.

[0078] According to embodiments of this disclosure, the three-dimensional point cloud information of the robotic arm operation can be rasterized by first dividing the three-dimensional point cloud information of the robotic arm operation into several equally divided regions according to the x and y directions.

[0079] According to embodiments of this disclosure, the raster 3D point cloud information that meets the first preset condition is processed in the order of acquiring 3D point cloud information corresponding to the scanning method to obtain key points. The first preset condition may be the highest point in each region. The 3D position coordinates (x, y, z) of the highest point are obtained, and these coordinate points are concatenated in the manner of xyx or yxy to form key points p1_all[p1, p2, p3… pn].

[0080] The third sub-path is smoothed using a mean-square algorithm to prevent the robotic arm from moving too smoothly. The coordinates of the robotic arm's sub-path are p2_all[p1,p2,p3…pn].

[0081] Using MoveIt or the robotic arm's built-in control software, move along the coordinates p2_all of the robotic arm's running sub-path to complete the entire contour scanning process and detection work.

[0082] According to the embodiments of this disclosure, it is possible to realize surface contour scanning of objects based on robotic arms and vision systems. It can complete adaptive scanning adaptation of the robotic arm end effector according to different surface conditions of the objects, without the need for secondary calibration and teaching. It has a high degree of automation and meets the needs of flexible detection and processing.

[0083] According to embodiments of this disclosure, it can be used for fixed-point sampling schemes on object surfaces, performing fixed-point reconnaissance and sampling within a fixed distance above the object surface.

[0084] According to embodiments of this disclosure, a path planning system includes an image acquisition device, a robotic arm, and a processor. The image acquisition device performs contour scanning of the object to be scanned using a scanning method to obtain three-dimensional point cloud information. The processor, connected to the image acquisition device, receives the three-dimensional point cloud information sent by the image acquisition device and executes the aforementioned robotic arm path planning method to process the three-dimensional point cloud information to obtain the robotic arm's running path. The robotic arm, connected to the processor, receives the robotic arm's running path sent by the processor and performs detection or welding work according to the robotic arm's running path.

[0085] According to embodiments of this disclosure, the image acquisition device does not contact the robotic arm, and the height of the acquisition device is greater than the height of the robotic arm. Alternatively, the acquisition device is mounted at the end of the robotic arm.

[0086] Figure 6 The diagram illustrates an image acquisition device according to an embodiment of the present disclosure that is not in contact with a robotic arm.

[0087] like Figure 6As shown, an outside-eye approach is adopted, where the image acquisition device can be a camera (including but not limited to 3D cameras, 2D cameras, 3D LiDAR, etc.), placed outside the robotic arm. When the scanning task begins, the camera triggers the scanning of the object surface to acquire 3D information of the object surface. Based on the set distance, a shortest path planning algorithm is proposed, and the robotic arm and equipment are controlled to scan the entire object surface, completing tasks such as detection and welding, thus realizing flexible measurement and manufacturing.

[0088] Figure 7 A schematic diagram illustrating a data acquisition device installed at the end of a robotic arm according to an embodiment of the present disclosure is shown.

[0089] like Figure 7 As shown, an eye-in-the-in approach is adopted, where the image acquisition device can be a camera (including but not limited to 3D cameras, 2D cameras, 3D LiDAR, etc.), placed at the end of the robotic arm. When the scanning task begins, the end of the robotic arm is first controlled to complete the surface scanning of the object to be scanned, acquiring 3D information of the object's surface. Based on the set distance, an optimal path planning algorithm is proposed, and the robotic arm and equipment are controlled to scan the entire surface of the object, completing tasks such as detection and welding.

[0090] According to embodiments of this disclosure, when an image acquisition device receives a scanning request sent by a processor, it performs contour scanning of the object to be scanned according to the scanning method in the scanning request.

[0091] Figure 8 A block diagram schematically illustrates an electronic device suitable for implementing a robotic arm path planning method according to an embodiment of the present disclosure.

[0092] like Figure 8 As shown, an electronic device 800 according to an embodiment of this disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage portion 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this disclosure.

[0093] RAM 803 stores various programs and data required for the operation of electronic device 800. Processor 801, ROM 802, and RAM 803 are interconnected via bus 804. Processor 801 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 802 and / or RAM 803. It should be noted that the programs may also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0094] According to embodiments of this disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to a bus 804. The electronic device 800 may also include one or more of the following components connected to the input / output (I / O) interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 810 as needed so that computer programs read from it can be installed into the storage section 808 as needed.

[0095] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0096] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 802 and / or RAM 803 and / or one or more memories other than ROM 802 and RAM 803 described above.

[0097] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this disclosure.

[0098] When the computer program is executed by the processor 801, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0099] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 809, and / or installed from a removable medium 811. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0100] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by processor 801, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0101] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0102] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0103] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0104] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A path planning method for a robotic arm, comprising: Acquire three-dimensional point cloud information corresponding to the object to be scanned. The three-dimensional point cloud information is obtained by a contour scanning of the object to be scanned using an image acquisition device in a scanning manner. The three-dimensional point cloud information is processed according to a preset distance to obtain the three-dimensional point cloud information of the robotic arm operation. The preset distance is the safe distance between the robotic arm and the object to be scanned. The three-dimensional point cloud information of the robotic arm operation is rasterized to obtain rasterized three-dimensional point cloud information; According to the order of acquiring the three-dimensional point cloud information corresponding to the scanning method, the grid three-dimensional point cloud information that meets the first preset condition is processed to obtain key points. The first preset condition is that the vertical coordinate of the grid three-dimensional point cloud information meets a preset threshold. Based on the key points, path planning is performed to obtain the robotic arm's running path, which is used by the robotic arm to detect or weld the object to be scanned.

2. The method according to claim 1, wherein, The step of processing the raster 3D point cloud information that meets the first preset condition according to the order of acquiring the 3D point cloud information corresponding to the scanning method to obtain key points includes: Based on the first preset condition, extract the key points to be processed from the grid 3D point cloud information; The key points to be processed are sorted according to the order in which the three-dimensional point cloud information is acquired, corresponding to the scanning method, to obtain the key points.

3. The method according to claim 1, wherein the key point includes at least one sub-key point, and the step of path planning based on the key point to obtain the robotic arm running path includes: The (N-1)th sub-key point is taken as the root node of the first tree; Based on the root node of the first tree, the first tree is searched and expanded to obtain new nodes of the first tree; The Nth sub-key point is taken as the root node of the second tree; Based on the root node of the second tree, the second tree is expanded by search to obtain new nodes of the second tree; If the new nodes of the first tree and the new nodes of the second tree meet the second preset condition, determine the robotic arm running sub-path from the (N-1)th sub-key point to the Nth sub-key point.

4. The method according to claim 3, wherein the second preset condition includes a step size limit condition and a collision-free path condition, and determining the robotic arm running sub-path from the (N-1)th sub-key point to the Nth sub-key point when the new nodes of the first tree and the second tree satisfy the second preset condition includes: If the new nodes of the first tree and the new nodes of the second tree satisfy the step size constraint, the first sub-path from the (N-1)th sub-key point to the Nth sub-key point is obtained; If the first sub-path satisfies the collision-free path condition, a second sub-path is obtained; Interpolation is performed on the second sub-path to obtain the third sub-path from the (N-1)th sub-key point to the Nth sub-key point; The third sub-path is smoothed using a mean algorithm to obtain the robotic arm running sub-path from the (N-1)th sub-key point to the Nth sub-key point.

5. The method according to claim 1, wherein, The scanning method is an arc scanning method, which involves scanning the object to be scanned from a first preset direction, then scanning the object to be scanned a unit distance in a second preset direction, and then continuing to scan the object to be scanned from the first preset direction, and repeating the scanning until the object to be scanned is completely scanned.

6. The method according to claim 5, wherein, The bow-shaped scanning method includes the xyx method and the yxy method. In the xyx method, the first preset direction is the horizontal coordinate and the second preset direction is the vertical coordinate. In the yxy method, the first preset direction is the vertical coordinate and the second preset direction is the horizontal coordinate.

7. The method according to claim 1, further comprising, before acquiring the three-dimensional point cloud information corresponding to the object to be scanned: The positions of the image acquisition device and the robotic arm are calibrated to obtain calibration information; Based on the calibration information, the spatial relative positional relationship between the image acquisition device and the robotic arm is determined; Based on the spatial relative position relationship, the object coordinate system with the image acquisition device as the origin is transformed to obtain the object coordinate system with the robotic arm as the origin.

8. A path planning system, the system comprising an image acquisition device, a robotic arm, and a processor; An image acquisition device is used to perform contour scanning of the object to be scanned according to a scanning method to obtain three-dimensional point cloud information; A processor, connected to the image acquisition device, is used to receive the three-dimensional point cloud information sent by the image acquisition device and to process the three-dimensional point cloud information by the method of any one of claims 1 to 7 to obtain the robotic arm running path; A robotic arm, connected to the processor, is used to receive the robotic arm's running path sent by the processor and to perform detection or welding work according to the robotic arm's running path.

9. The path planning system according to claim 8, wherein, The image acquisition device does not contact the robotic arm, and the height of the acquisition device is greater than the height of the robotic arm; or the acquisition device is installed at the end of the robotic arm.

10. The path planning system according to claim 8, wherein, Upon receiving a scanning request from the processor, the image acquisition device performs contour scanning on the object to be scanned according to the scanning method specified in the scanning request.

Images