Self-adaptive part operation method and system

Abstract

The invention discloses a self-adaptive part operation method and system, and relates to the technical field of intelligent manufacturing equipment, and the method comprises the steps: carrying out the part target detection of transportation image data through an improved sparse convolutional network, and obtaining the part recognition information corresponding to a current transportation part; sLAM positioning is carried out according to the 3D point cloud environment data, and robot position information is obtained; determining a target transportation track corresponding to the current transportation part demand based on the acquired transportation obstacle information, the weight information of the current transportation part, the part identification information and the robot position information; joint pose reasoning is carried out according to the weight information of the currently transported part and the part identification information, and mechanical arm control information is obtained; and based on the target transportation track and the mechanical arm control information, self-adaptive operation is conducted on the current transportation part. Through precise part positioning detection, the part transfer process is optimized by combining automatic conveying of the robot and automatic carrying and operation of the mechanical arm, and the part operation efficiency and precision are improved.

Description

Technical Field

[0001] This application relates to the technical field of intelligent manufacturing equipment, and particularly relates to an adaptive part operation method and system. Background Art

[0002] During the production process, the accuracy and timeliness of the supply of materials such as parts are crucial for ensuring the smoothness and efficiency of the entire production process. However, during the production process, due to various reasons, such as equipment aging, process limitations, and human errors and delays, in the existing production operation process, the quality and production efficiency of parts often have deficiencies.

[0003] Therefore, how to solve the problem of inaccurate and untimely supply in the existing part operation process and improve the part operation efficiency and accuracy has become an urgent problem to be solved. Summary of the Invention

[0004] The main purpose of this application is to provide an adaptive part operation method and system, aiming to solve the technical problem of inaccurate and untimely supply in the existing part operation process and improve the part operation efficiency and accuracy.

[0005] To achieve the above object, this application proposes an adaptive part operation method, which includes:

[0006] Performing part target detection on the transportation image data through an improved sparse convolutional network to obtain part recognition information corresponding to the currently transported part;

[0007] Performing SLAM positioning based on the 3D point cloud environment data to obtain the robot position information;

[0008] Determining a target transportation trajectory corresponding to the current transportation part requirement based on the collected transportation obstacle information, the weight information of the current transportation part, the part recognition information, and the robot position information;

[0009] Performing joint pose reasoning based on the weight information of the current transportation part and the part recognition information to obtain the manipulator control information; the manipulator control information includes the manipulator grasping pose and the manipulator movement trajectory;

[0010] Performing adaptive operation on the current transportation part based on the target transportation trajectory and the manipulator control information.

[0011] In one embodiment, the improved sparse convolutional network includes: an improved sparse feature extraction module, an anchor box calibration module, and a classification module; the improved sparse convolutional network is a DetNet network integrating the Inception structure;

[0012] The step of performing part target detection on the transportation image data through the improved sparse convolution network to obtain the part recognition information corresponding to the current transported part includes:

[0013] Performing feature analysis on the transportation image data through the improved sparse feature extraction module to obtain initial part features;

[0014] Performing target detection on the initial part features according to the anchor box calibration module to obtain target part anchor boxes;

[0015] Performing sample classification on the target part anchor boxes through the classification module to obtain part recognition information.

[0016] In one embodiment, the step of determining the target transportation trajectory corresponding to the current transportation part requirement based on the collected transportation obstacle information, the weight information of the current transported part, the part recognition information, and the robot position information includes:

[0017] Performing rapid-exploration random tree operation based on the robot position information, the part recognition information, and the weight information of the current transported part to obtain an initial transportation trajectory;

[0018] Optimizing the initial transportation trajectory according to the collected transportation obstacle information to obtain the target transportation trajectory corresponding to the current transportation part requirement.

[0019] In one embodiment, the step of optimizing the initial transportation trajectory according to the collected transportation obstacle information to obtain the target transportation trajectory corresponding to the current transportation part requirement includes:

[0020] Determining the target transportation part in the current transported part based on the current transportation part requirement and the weight information of the current transported part;

[0021] Performing random point sampling optimization on the initial transportation trajectory to obtain an optimized transportation trajectory;

[0022] Performing greedy path iteration on the optimized transportation trajectory based on the collected transportation obstacle information to obtain the target transportation trajectory.

[0023] In one embodiment, the step of performing joint pose inference based on the weight information of the current transported part and the part recognition information to obtain the manipulator control information includes:

[0024] Performing parallel pose detection according to the weight information of the current transported part and the part recognition information to obtain the target grasping pose;

[0025] Performing joint inference based on the target grasping pose, the weight information of the current transported part, and the part recognition information to obtain the manipulator control information.

[0026] In one embodiment, the step of performing parallel pose detection based on the weight information of the current transported part and the part identification information to obtain the target grasping pose includes:

[0027] Performing preset partition pose detection based on the part identification information to obtain an initial grasping pose;

[0028] Performing multi-scale fusion feature analysis based on the weight information of the current transported part to obtain pose credibility;

[0029] Determining the target grasping pose based on the initial grasping pose and the pose credibility.

[0030] In one embodiment, the step of adaptively operating on the current transported part based on the target transportation trajectory and the robotic arm control information includes:

[0031] Determining the target grasping position based on the target transportation trajectory, the movement speed of the current transported part, and the movement speed of the robotic arm;

[0032] Optimizing the movement trajectory of the robotic arm according to the target grasping position to obtain an optimized movement trajectory;

[0033] Controlling the robot to transport the current transported part to the target operation position according to the target transportation trajectory;

[0034] When the current transported part reaches the target operation position, controlling the robotic arm to transport the current transported part to the target operation position for adaptive operation according to the robotic arm grasping pose and the optimized movement trajectory.

[0035] In addition, to achieve the above object, the present application also proposes an adaptive part operation system, which includes:

[0036] A part detection module, configured to perform part target detection on transportation image data through an improved sparse convolutional network to obtain part identification information corresponding to the current transported part;

[0037] A robot positioning module, configured to perform SLAM positioning according to 3D point cloud environment data to obtain robot position information;

[0038] A trajectory planning module, configured to determine a target transportation trajectory corresponding to the current transported part demand based on the collected transportation obstacle information, the weight information of the current transported part, the part identification information, and the robot position information;

[0039] The robotic arm analysis module is used to perform joint pose inference based on the weight information of the currently transported part and the part identification information to obtain robotic arm control information; the robotic arm control information includes the robotic arm grasping pose and the robotic arm movement trajectory;

[0040] The operation control module is used to perform adaptive operation on the currently transported part based on the target transportation trajectory and the robotic arm control information.

[0041] In addition, to achieve the above object, the present application also provides an adaptive part operation device, which includes: a memory, a processor, and an adaptive part operation program stored on the memory and executable on the processor. The adaptive part operation program is configured to implement the steps of the adaptive part operation method as described above.

[0042] In addition, to achieve the above object, the present application also provides a storage medium, which is a computer-readable storage medium. A program for implementing the adaptive part operation method is stored on the computer-readable storage medium. The program for implementing the adaptive part operation method is executed by a processor to implement the steps of the adaptive part operation method as described above.

[0043] The present application provides an adaptive part operation method and system. The method includes: performing part target detection on transportation image data through an improved sparse convolutional network to obtain part identification information corresponding to the currently transported part; performing SLAM positioning based on 3D point cloud environment data to obtain robot position information; determining a target transportation trajectory corresponding to the current transportation part requirement based on the collected transportation obstacle information, the weight information of the currently transported part, the part identification information, and the robot position information; performing joint pose inference based on the weight information of the currently transported part and the part identification information to obtain robotic arm control information; the robotic arm control information includes the robotic arm grasping pose and the robotic arm movement trajectory; performing adaptive operation on the currently transported part based on the target transportation trajectory and the robotic arm control information.

[0044] Therefore, the present application accurately locates and detects the target part that meets the requirements of the current transported part for the transported part by introducing an improved sparse convolutional network, and combines robot SLAM positioning and robotic arm automatic control to perform automatic handling and operation of the part, effectively optimizing the storage and transportation process of the part, reducing unnecessary transfer and waiting time, improving the timeliness of part transportation, and improving the production accuracy and efficiency of the part. Therefore, the present application can not only greatly improve the operation efficiency and accuracy of the part, avoid human errors and delays, but also formulate an automatic part operation plan that meets the actual production requirements, effectively reducing the part operation cost. Description of the Drawings

[0045] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0046] To more clearly illustrate the technical solutions in the embodiments of the present application or in the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0047] Figure 1 It is a first process schematic diagram of the first embodiment of the adaptive part operation method of the present application;

[0048] Figure 2 It is a second process schematic diagram of the first embodiment of the adaptive part operation method of the present application;

[0049] Figure 3 It is a schematic structural diagram of the improved sparse feature extraction module of the first embodiment of the adaptive part operation method of the present application;

[0050] Figure 4 It is a schematic diagram of the part adaptive operation process of the first embodiment of the adaptive part operation method of the present application;

[0051] Figure 5 It is a process schematic diagram of the second embodiment of the adaptive part operation method of the present application;

[0052] Figure 6 It is a schematic diagram of the initial transportation trajectory of the second embodiment of the adaptive part operation method of the present application;

[0053] Figure 7 It is a schematic diagram of the first optimization process of the initial transportation trajectory of the second embodiment of the adaptive part operation method of the present application;

[0054] Figure 8 It is a schematic diagram of the second optimization process of the initial transportation trajectory of the second embodiment of the adaptive part operation method of the present application;

[0055] Figure 9 It is a schematic structural diagram of the module of the adaptive part operation system in the embodiment of the present application;

[0056] Figure 10 It is a schematic diagram of the device structure of the hardware operating environment involved in the adaptive part operation method in the embodiment of the present application.

[0057] The realization of the purpose, functional features, and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. Detailed implementation manners

[0058] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not used to limit the present application.

[0059] To better understand the technical solutions of the present application, the following will be described in detail in conjunction with the accompanying drawings of the specification and specific implementation manners.

[0060] The main solution of the present application is as follows: By improving the sparse convolutional network, perform part target detection on the transportation image data to obtain part recognition information corresponding to the current transported part; perform SLAM positioning based on the 3D point cloud environmental data to obtain the robot position information; determine the target transportation trajectory corresponding to the current transportation part requirement based on the collected transportation obstacle information, the weight information of the current transported part, the part recognition information, and the robot position information; perform joint pose inference based on the weight information and part recognition information of the current transported part to obtain the manipulator control information; the manipulator control information includes the manipulator grasping pose and the manipulator movement trajectory; perform adaptive operation on the current transported part based on the target transportation trajectory and the manipulator control information.

[0061] At present, it is urgent to solve the problem of inaccurate and untimely supply during the part production operation process to ensure the timeliness of the part production operation.

[0062] Therefore, the present application effectively optimizes the storage and transportation processes of parts, reduces unnecessary transfer and waiting times, improves the timeliness of part transportation, and improves the production accuracy and efficiency of parts by introducing an improved sparse convolutional network to accurately locate and detect target parts corresponding to the current transportation part requirements, and combining robot SLAM positioning and manipulator automation control for automatic handling and operation of parts. Therefore, the present application can not only greatly improve the operation efficiency and accuracy of parts, avoid human errors and delays, but also formulate an automatic part operation plan that meets the actual production requirements, effectively reducing the part operation cost.

[0063] It should be noted that the execution subject of this embodiment can be an adaptive part operation system, or a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an adaptive part operation device capable of implementing the above functions, etc. This embodiment does not make specific limitations in this regard. The following takes an adaptive part operation device (abbreviated as device) as the execution subject as an example to describe this embodiment and the following embodiments.

[0064] Based on this, the embodiments of the present application provide an adaptive part operation method, referring to Figure 1 , Figure 1 which is the first process schematic diagram of the first embodiment of the adaptive part operation method of the present application.

[0065] In this embodiment, the adaptive part operation method includes steps S10 to S50:

[0066] Step S10, perform part target detection on the transportation image data through an improved sparse convolutional network to obtain part recognition information corresponding to the currently transported part;

[0067] It is easy to understand that the above transportation image data is an image collected by a monocular camera during part transportation in the part operation process. To ensure the transportation timeliness in the subsequent part operation process, in this embodiment, a high-precision visual detection and sensor data fusion model, that is, the above improved sparse convolutional network, can be used to achieve accurate recognition and detection of parts of various sizes. Therefore, the above part recognition information may include the category and size information of the part for subsequent accurate transportation.

[0068] In a feasible implementation manner, the improved sparse convolutional network includes: an improved sparse feature extraction module, an anchor box calibration module, and a classification module; the improved sparse convolutional network is a DetNet network integrating the Inception structure; refer to Figure 2 , Figure 2 As shown in the second process schematic diagram of the first embodiment of the adaptive part operation method of this application, in this embodiment, step S10 may include steps A1 to A3:

[0069] Step A1, perform feature analysis on the transportation image data through the improved sparse feature extraction module to obtain initial part features;

[0070] Step A2, perform target detection on the initial part features according to the anchor box calibration module to obtain target part anchor boxes;

[0071] Step A3, perform sample classification on the target part anchor boxes through the classification module to obtain part recognition information.

[0072] It is easy to understand that in this embodiment, the above improved sparse convolutional network may be an improved DetNet network integrating the Inception structure. Therefore, the improved sparse feature extraction module in the improved sparse convolutional network may be a DetNet backbone network integrating the Inception structure.

[0073] It should be understood that in this embodiment, an Inception structure is added to the basic feature extraction network of the DETNet model to increase the width of the network and the adaptability of the network to image scales. Specifically, compared with the existing method of improving the feature extraction accuracy by increasing the number of network layers, in this embodiment, by introducing the inception structure, multiple convolutional or pooling operations are assembled together into a network module, and the entire network structure is assembled with the module as a unit. Therefore, in this embodiment, the introduced Inception structure generates dense data on the basis of a sparse network structure, which can not only improve the performance of the neural network but also ensure the use efficiency of computing resources.

[0074] For the sake of easy understanding, with reference to Figure 3 as an example, this embodiment will be illustrated. Figure 3 FIG. is a schematic diagram of the improved sparse feature extraction module structure of the first embodiment of the adaptive part operation method of the present application. As Figure 3 shown, the improved sparse feature extraction module proposed in this embodiment can be cleverly incorporated with two Inception layer structures. These two Inception layers are respectively composed of conv#1×1, conv#3×3, conv#5×5, and pool#3×3.

[0075] At the same time, compared with the original DetNet backbone network structure, in this embodiment, three 3×3 convolutional layers in the 8th, 12th, and 16th layers can also be removed to reduce unnecessary computational burdens. At the same time, in the network architecture after removing these convolutional layers, in this embodiment, Inception structures are cleverly embedded in the 13th and 15th layers respectively, thus forming a more efficient and powerful network algorithm architecture.

[0076] It should be noted that although the Inception layer is added in this embodiment, the increased number of parameters is far less than the number of parameters reduced by removing three convolutional layers. Therefore, in this embodiment, by improving the sparse convolution module, while significantly increasing the network depth, the number of parameters and the amount of calculation are also greatly reduced, achieving a more efficient object detection performance.

[0077] In addition, after performing feature analysis on the transportation image data through the improved sparse feature extraction module to obtain the initial part features, in this embodiment, the extracted feature map can be used as the input of the anchor box calibration module, that is, the RPN network, and a sliding window operation is performed on the feature map using a fixed-size sliding window, and multiple anchor boxes of different sizes and ratios are generated with each position as the center point, that is, the above-mentioned target part anchor boxes.

[0078] Finally, to further improve the detection accuracy, in this embodiment, the classification module can be a dual-branch structure. Among them, the first branch can use Softmax classification to classify positive and negative samples for the anchor boxes, and determine each anchor box as a positive classification that may contain the target or a negative classification that does not contain the target;

[0079] The second branch can be used for bounding box regression. This regression operation can calculate the offset between the predicted bounding box and the true target bounding box to obtain a more accurate candidate box position. Specifically, this branch can determine the position of each anchor box in the original image through a scale mapping function (im_info), and further determine whether it exceeds the boundary of the original image. When the anchor box seriously exceeds the boundary, these anchor boxes basically do not contain target information, are invalid and need to be eliminated. Therefore, the second branch can use bounding box regression to correct the eliminated anchor boxes and obtain the final proposed boxes.

[0080] Therefore, this embodiment can use a dual-branch to divide multi-scale anchor boxes to obtain a more accurate candidate box position.

[0081] Step S20: Perform SLAM positioning based on the 3D point cloud environment data to obtain the robot position information;

[0082] It should be understood that in order to optimize the part transportation process, this embodiment can use the 3D point cloud environment data corresponding to the collected part transportation environment for SLAM (Simultaneous Localization and Mapping) positioning, that is, perform robot simultaneous localization and mapping positioning. While determining the real-time position and attitude of the robot, that is, the above-mentioned robot position information, the machine can move autonomously in an unknown environment and construct a map of the part transportation environment in real time.

[0083] Step S30: Determine the target transportation trajectory corresponding to the current transported part demand based on the collected transportation obstacle information, the weight information of the current transported part, the part identification information, and the robot position information;

[0084] It should be noted that in order to solve the problems of inaccurate and untimely material supply and ensure the timeliness of material transportation in the production process, this embodiment can also pre-establish an efficient material management system. By real-time monitoring the inventory level and predicting the current material processing demand, the corresponding parts that need to be automated are determined, so as to ensure the timely replenishment of materials through timely part operations, so as to avoid production stagnation caused by material shortage or part quality problems and ensure the stability and reliability of the production process.

[0085] Therefore, the above current transportation part requirements can be the part operation requirements determined according to the material management system. Correspondingly, in this embodiment, the target parts that need to be automated in the current transportation parts can be determined according to the current transportation part requirements, and the target parts can be transported to the corresponding positions for adaptive processing, welding and other operations.

[0086] It is easy to understand that the weight information of the above current transportation parts can be obtained through the strain gauge sensor weighing module, and the transportation obstacle information can be obtained through the ultrasonic sensor. The characterized obstacles refer to the obstacles such as equipment, desks and chairs in the factory environment during the transportation process of the robot. In this embodiment, the part types corresponding to the current transportation parts can be determined based on the weight information of the current transportation parts and the accurate part identification information, so as to select the target parts that meet the current transportation part requirements, and based on the target operation positions corresponding to the target parts, the collected transportation obstacle information and the robot position information, an efficient transportation trajectory planning can be carried out to obtain the target transportation trajectory corresponding to the current transportation part requirements.

[0087] Step S40, perform joint pose inference according to the weight information of the current transportation part and the part identification information to obtain the manipulator control information; the manipulator control information includes the manipulator grasping pose and the manipulator movement trajectory;

[0088] Step S50, perform adaptive operations on the current transportation part based on the target transportation trajectory and the manipulator control information.

[0089] It is easy to understand that in this embodiment, during the corresponding operations of parts with different sizes, different weights and different transportation distances, the manipulator can adopt different grasping poses and movement trajectories for accurate grasping and delivery of parts. Therefore, in this embodiment, joint pose inference of the manipulator can be performed based on the part weight information and the part identification information to obtain the manipulator movement trajectory that can achieve accurate grasping and efficient delivery of parts.

[0090] In a feasible implementation manner, in this embodiment, step S50 may include steps B1 to B4:

[0091] Step B1, determine the target grasping position based on the target transportation trajectory, the movement speed of the current transportation part and the movement speed of the manipulator;

[0092] Step B2, optimize the manipulator movement trajectory according to the target grasping position to obtain the optimized movement trajectory;

[0093] Step B3, control the robot to transport the current transportation part to the target operation position according to the target transportation trajectory;

[0094] Step B4, when the current transported part reaches the target operation position, control the robotic arm to transport the current transported part to the target operation position according to the robotic arm grasping pose and the optimized motion trajectory for adaptive operation.

[0095] It should be understood that, in order to further improve the operation efficiency, after initially determining the target transportation trajectory for transporting the target part from the placement location to the robotic arm, this embodiment can further determine the meeting position of the robot and the robotic arm where the part transportation time is the shortest when the robot and the robotic arm move simultaneously according to the target transportation trajectory, the movement speed of the current transported part, and the movement speed of the robotic arm, that is, the above-mentioned target grasping position. Therefore, this embodiment can optimize the movement trajectory of the robotic arm through the target grasping position, so that when controlling the robot to transport the current transported part to the target operation position according to the target transportation trajectory corresponding to each target part, the robotic arm directly performs precise grasping and precise delivery of each target part at the target operation position according to the target grasping pose and the optimized motion trajectory, so as to further reduce the part transportation time and improve the part operation efficiency.

[0096] Therefore, as Figure 4 shown, Figure 4 is a schematic diagram of the part adaptive operation process of the first embodiment of the adaptive part operation method of this application. As Figure 4 shown, in this embodiment, the part information and the robotic arm state can be transmitted to the adaptive welding system. The entire system takes the computing board and two controllers as the core to control the data flow, so as to complete the cooperation between the part adaptive transportation and the adaptive welding system. The design of the two controllers ensures the parallel progress of transportation and grasping and guarantees the timely transmission of information between modules. This embodiment can use the target detection module, SLAM mapping, and the sensor group to jointly experiment on the precise recognition of parts, not only effectively recognize the type, size, shape, and weight of the parts, but also accurately recognize the position of the parts and the state of the parts in the overall space environment, providing real-time information sharing for the transportation robot and the robotic arm, so as to achieve adaptive precise part recognition. That is, this embodiment can accurately find the target parts through multiple part judgment schemes such as part target detection, SLAM three-dimensional mapping, and weight testing. At the same time, based on the 5G technology information real-time sharing platform, the real-time data and control information of each device can be shared in real time, improving the decision-making efficiency and realizing the complete integration of the adaptive operation and the adaptive transportation system.

[0097] In addition, in Figure 4There are also two information feedback lines. That is, in this embodiment, through the feedback from output to input, specifically the accuracy of part recognition is feedback. Through real-time feedback, the parameters of the part recognition model are adjusted to form self-learning, continuously optimize the model, and improve the model accuracy. Specifically, in the target detection module, the feedback information forms a function: the loss function. This function adjusts the model parameters in real time by calculating the error between the true value and the predicted value. Second, during SLAM mapping, loop detection is continuously performed through feedback to improve the mapping accuracy. The self-learning of the system is realized, enabling the system to resist the influence of cumulative error on the system accuracy during actual operation and ensuring the effective and accurate transmission of information between modules.

[0098] In this embodiment, by introducing high-precision image analysis parts, precise positioning and detection of target parts corresponding to the current transportation part requirements are carried out, and based on the automatic control of the robotic arm, the handling and assembly of parts are carried out to effectively optimize the storage and transportation processes of parts, reduce unnecessary transfer and waiting times, improve the timeliness of material transportation, and improve the production accuracy and efficiency of parts. Therefore, this embodiment can not only greatly improve the operation efficiency and accuracy of parts, but also reduce human errors and delays, formulate an automatic operation plan for parts that meets the actual production requirements, and effectively reduce the operation cost of parts.

[0099] This embodiment provides an adaptive part operation method, which includes: performing feature analysis on the transportation image data through an improved sparse feature extraction module to obtain initial part features; performing object detection on the initial part features according to the anchor box calibration module to obtain target part anchor boxes; performing sample classification on the target part anchor boxes through a classification module to obtain part recognition information. Performing SLAM positioning according to the 3D point cloud environmental data to obtain the robot position information; determining the target transportation trajectory corresponding to the current transportation part requirement based on the collected transportation obstacle information, the weight information of the current transportation part, the part recognition information, and the robot position information; performing joint pose inference according to the weight information and part recognition information of the current transportation part to obtain the robotic arm control information; the robotic arm control information includes the robotic arm grasping pose and the robotic arm movement trajectory; determining the target grasping position based on the target transportation trajectory, the movement speed of the current transportation part, and the movement speed of the robotic arm; optimizing the robotic arm movement trajectory according to the target grasping position to obtain an optimized movement trajectory; controlling the robot to transport the current transportation part to the target operation position according to the target transportation trajectory; when the current transportation part reaches the target operation position, controlling the robotic arm to transport the current transportation part to the target operation position for adaptive operation according to the robotic arm grasping pose and the optimized movement trajectory. This embodiment effectively optimizes the part storage and transportation processes by introducing high-precision image analysis parts for accurate positioning and detection of target parts corresponding to the current transportation part requirements, and performing part handling and assembly based on robotic arm automation control, reducing unnecessary transfer and waiting times, improving the timeliness of material transportation, and enhancing the production accuracy and efficiency of parts. Therefore, this embodiment can not only greatly improve the operation efficiency and accuracy of parts, but also reduce human errors and delays, formulate an automated part operation plan that meets the actual production requirements, and effectively reduce the part operation cost.

[0100] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar content as that in the above-mentioned first embodiment can be referred to the above introduction and will not be repeated hereinafter.

[0101] On the basis of the first embodiment, in this embodiment, referring to Figure 5 , Figure 5 is a schematic flowchart of the second embodiment of the adaptive part operation method of the present application. Step S30 includes steps S31 to S32:

[0102] Step S31, performing rapid-exploration random tree operation based on the robot position information, the part recognition information, and the weight information of the current transportation part to obtain an initial transportation trajectory;

[0103] Step S32, optimizing the initial transportation trajectory according to the collected transportation obstacle information to obtain the target transportation trajectory corresponding to the current transportation part requirement.

[0104] It is easy to understand that in this embodiment, first, the target transport part corresponding to the current transport part demand in the current transport parts can be determined according to the current transport part demand, part identification information, and the weight information of the current transport part, and then the initial transport trajectory can be determined in combination with the position of the operation robotic arm corresponding to the target transport part.

[0105] For ease of understanding, refer to Figure 6 An example is given to illustrate the process of obtaining the initial transport trajectory according to the rapidly-exploring random tree operation in this embodiment. Figure 6 FIG. is a schematic diagram of the initial transport trajectory of the second embodiment of the adaptive part operation method of the present application. As Figure 6 shown, in this embodiment, starting from the starting point Xs corresponding to the robot position information, random exploration can be performed in space through the rapidly-exploring random tree algorithm (RRT).

[0106] First, a random point Xrand can be generated. Taking this random point Xrand as the target, all nodes on the growing tree are traversed, the distance from each node to Xrand is calculated, and the node with the smallest distance is selected as the nearest point Xnearest. Taking the connection line between the nearest point and the random point as the growth direction, a new node Xnew is generated at a certain step size, and a new expanded tree structure is obtained by connecting Xnearest and Xnew. When the new node Xnew collides with an obstacle, this expansion of the tree structure is stopped.

[0107] Then, the above process is repeated for the next round of sampling and expansion. Finally, when the new node Xnew enters the set range of the target point Xt corresponding to the position of the operation robotic arm, the sampling is stopped, and then the parent nodes of each node are traced to obtain a collision-free path from the starting point Xs to the ending point Xt, thereby obtaining the above-mentioned initial transport trajectory.

[0108] In a feasible implementation manner, in this embodiment, step S32 includes steps C1 to C2:

[0109] Step C1, randomly sampling and optimizing the initial transport trajectory to obtain an optimized transport trajectory;

[0110] Step C2, performing greedy path iteration on the optimized transport trajectory based on the collected transport obstacle information to obtain a target transport trajectory.

[0111] It is easy to understand that the transmission time of obtaining the initial transport trajectory based on the original rapidly-exploring random number algorithm (RRT) may be relatively long and still needs to be optimized. Therefore, to improve the trajectory planning efficiency, this embodiment can improve the initial transport trajectory. Specifically, this embodiment can adopt random sampling within the target range and the idea of extreme greed to reduce the time and nodes of the sampling path and increase the success rate of the sampling path.

[0112] It should be understood that, first, in this embodiment, the target transport part in the current transport parts can be determined according to the current transport part requirements and the weight information of the current transport parts, and then the initial transport trajectory can be optimized according to the position of the robotic arm corresponding to the target transport part, that is, the above position transport information.

[0113] Specifically, with reference to Figure 7 and Figure 8 an example is given to illustrate the initial transport trajectory optimization process of this embodiment. Figure 7 FIG. is a schematic diagram of the first optimization process of the initial transport trajectory of the second embodiment of the adaptive part operation method of the present application. Figure 8 FIG. is a schematic diagram of the second optimization process of the initial transport trajectory of the second embodiment of the adaptive part operation method of the present application. As Figure 7 shown, in this embodiment, first, when setting sampling points, a threshold is artificially set, which is randomly and uniformly taken between (0, 1). When the random value is greater than the set threshold, a point is randomly selected in the free space; when the random value is not greater than the set threshold, a random value is taken again between (0, 1). When the value is less than the threshold, the target point is directly used as the sampling point, otherwise a point is randomly selected within the range with the target point as the center and the distance from the nearest node in the random tree of RRT to the target point as the radius. In this way, the position information of the target point can be utilized to assist in path planning. The role of the random point selection method within the target range is to enable the random tree to grow towards the target point with a certain probability. At the same time, since the sampling points are selected within a range of the target position, obstacles can be avoided to a certain extent.

[0114] In this embodiment, adding random point selection within the target range during the process of exploring nodes can make full use of the known information of the target point position (the end point of part transportation) to give a certain guidance to the growth direction of the nodes. However, although this can accelerate the generation of the path, when the exploration step size is fixed, even if there are no obstacles between some nodes on the tree and the target point, iterative sampling still needs to be performed to explore the path, which is extremely time-consuming, especially in some scenarios with a relatively simple environment.

[0115] To avoid this situation and smooth the path further, further, with reference to Figure 8 , this embodiment can introduce an extremely greedy idea for path planning. Specifically, after sampling a new node in this embodiment, first, it is judged whether it can directly reach the target position according to the collected transport obstacle information, that is, there are no obstacles between the new node and the target position. If it can reach, the iteration is terminated and the path is traced back; if it cannot directly reach, then the subsequent path iteration steps are continued, thereby further reducing the path planning time.

[0116] In a feasible implementation manner, in this embodiment, step S40 includes steps S41 to S42:

[0117] Step S41: Perform parallel pose detection based on the weight information of the currently transported part and the part identification information to obtain the target grasping pose.

[0118] It should be noted that, in order to improve the part grasping accuracy of the robotic arm, in this embodiment, parallel pose detection can be performed based on the weight information of the currently transported part and the part identification information to obtain a high-precision target grasping pose. In a feasible implementation manner, in this embodiment, step S41 includes steps D1 to D3:

[0119] Step D1: Perform preset partition pose detection based on the part identification information to obtain the initial grasping pose.

[0120] Step D2: Perform multi-scale fusion feature analysis based on the weight information of the currently transported part to obtain the pose confidence.

[0121] Step D3: Determine the target grasping pose based on the initial grasping pose and the pose confidence.

[0122] It should be noted that in this embodiment, the above-mentioned part identification information may include multi-scale data features at different levels. At this time, in this embodiment, the image features corresponding to the part identification information can be equally divided into several (such as 57×57) candidate regions, and then the parameters of the best grasping rectangular frame (such as the center point coordinates, the width of the mechanical finger, the opening length, and the direction) that may exist in each candidate region can be calculated in combination with the preset grasping pose measurement algorithm, that is, the above-mentioned initial grasping pose. For the weight information of the currently transported part, in this embodiment, it is necessary to first perform multi-scale feature analysis on it, and then use the preset grasping pose measurement algorithm to calculate the pose confidence that may exist in each candidate region for the multi-scale feature data after its analysis, and then determine the target grasping pose based on the initial grasping pose and the pose confidence.

[0123] Step S42: Perform joint reasoning based on the target grasping pose, the weight information of the currently transported part, and the part identification information to obtain the robotic arm control information.

[0124] It is easy to understand that in this embodiment, the category of the target part in the complex scene and the position information of the rectangular frame where its outer contour is located in the image can be determined based on the weight information of the currently transported part and the part identification information. Combining the possible target grasping pose and its grasping confidence information of the object in each determined grasping candidate region, in this embodiment, joint pose calculation can be performed to obtain an accurate robotic arm grasping pose. The corresponding formula is as follows:

[0125]

[0126] Among them, N represents the number of detected target components, which comes from the final result output by the object detection network, that is, the number of object rectangular boxes after being processed by the non-maximum suppression method (NMS); M represents the number of candidate grasping regions in the image, and its value is 57×57, which is the number of grid cells into which the network divides the input image. Each cell corresponds to a possible grasping position, and this is completed inside the grasping pose measurement module, aiming to cover all potential grasping points within the entire image range; Sgj represents the credibility of the j-th grasping rectangular box in the image, which is a part of the output of the grasping pose measurement module and reflects the confidence level of the model for a specific grasping pose; Aj represents the area of the j-th component grasping rectangular box; Ai represents the area of the i-th component detection rectangular box. These area information directly comes from the rectangular box size parameters output by their respective modules. Aj comes from the grasping pose measurement module, based on the predicted grasping rectangular box (including center coordinates, width, height, etc.); Ai comes from the object detection module, based on the detected target object bounding box (which also contains position and size information).

[0127] It is easy to understand that after determining the grasping pose of the robotic arm, this embodiment can quickly determine the corresponding movement trajectory of the robotic arm. Therefore, this embodiment can perform parallel pose detection based on the weight information of the currently transported parts and the part recognition information to obtain a high-precision target grasping pose and improve the part grasping accuracy of the robotic arm.

[0128] This embodiment discloses performing rapid exploration random tree operation based on the robot position information, part recognition information, and the weight information of the currently transported parts to obtain an initial transportation trajectory; determining the target transported part in the currently transported parts based on the current transported part demand and the weight information of the currently transported parts; performing random sampling point optimization on the initial transportation trajectory to obtain an optimized transportation trajectory; and performing greedy path iteration on the optimized transportation trajectory based on the collected transportation obstacle information to obtain a target transportation trajectory. This embodiment can, on the basis of the original rapid exploration random number algorithm, reduce the time and nodes of the sampling path through random sampling points within the target range and the extremely greedy idea, increase the success rate of the sampling path, and further reduce the time consumed for part transportation.

[0129] This embodiment also discloses performing preset partition pose detection based on the part recognition information to obtain an initial grasping pose; performing multi-scale fusion feature analysis based on the weight information of the currently transported parts to obtain pose credibility; and determining the target grasping pose based on the initial grasping pose and the pose credibility. Based on the target grasping pose, the weight information of the currently transported parts, and the part recognition information, joint reasoning is performed to obtain robotic arm control information. This embodiment can perform parallel pose detection based on the weight information of the currently transported parts and the part recognition information to obtain a high-precision target grasping pose and improve the part grasping accuracy of the robotic arm.

[0130] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the adaptive part operation method of this application. Based on this technical concept, more forms of simple transformations are within the protection scope of this application.

[0131] This application also provides an adaptive part operation. Please refer to Figure 9 , Figure 9 which is a schematic diagram of the module structure of the adaptive part operation system in an embodiment of this application. In this embodiment, the adaptive part operation system includes:

[0132] A part detection module T1, which is used to perform part target detection on the transportation image data through an improved sparse convolutional network to obtain part recognition information corresponding to the current transported part;

[0133] A robot positioning module T2, which is used to perform SLAM positioning based on the 3D point cloud environment data to obtain robot position information;

[0134] A trajectory planning module T3, which is used to determine the target transportation trajectory corresponding to the current transported part demand based on the collected transportation obstacle information, the weight information of the current transported part, the part recognition information, and the robot position information;

[0135] A robotic arm analysis module T4, which is used to perform joint pose inference based on the weight information of the current transported part and the part recognition information to obtain robotic arm control information; the robotic arm control information includes the robotic arm grasping pose and the robotic arm movement trajectory;

[0136] An operation control module T5, which is used to perform adaptive operation on the current transported part based on the target transportation trajectory and the robotic arm control information.

[0137] As an implementable manner, in this embodiment, the improved sparse convolutional network includes: an improved sparse feature extraction module, an anchor box calibration module, and a classification module; the improved sparse convolutional network is a DetNet network integrating the Inception structure;

[0138] The part detection module T1 is also used to perform feature analysis on the transportation image data through the improved sparse feature extraction module to obtain initial part features;

[0139] The part detection module T1 is also used to perform target detection on the initial part features according to the anchor box calibration module to obtain target part anchor boxes;

[0140] The part detection module T1 is also used to perform sample classification on the target part anchor boxes through the classification module to obtain part recognition information.

[0141] As an implementable manner, in this embodiment, the trajectory planning module T3 is further configured to perform a Rapidly-Exploring Random Tree (RRT) operation based on the robot position information, the part identification information, and the weight information of the currently transported part, so as to obtain an initial transportation trajectory.

[0142] The trajectory planning module T3 is further configured to optimize the initial transportation trajectory according to the collected transportation obstacle information, so as to obtain a target transportation trajectory corresponding to the current transported part requirement.

[0143] As an implementable manner, in this embodiment, the trajectory planning module T3 is further configured to determine a target transported part in the current transported part based on the current transported part requirement and the weight information of the current transported part.

[0144] The trajectory planning module T3 is further configured to perform random sampling point optimization on the initial transportation trajectory to obtain an optimized transportation trajectory.

[0145] The trajectory planning module T3 is further configured to perform greedy path iteration on the optimized transportation trajectory based on the collected transportation obstacle information to obtain a target transportation trajectory.

[0146] As an implementable manner, in this embodiment, the robotic arm analysis module T4 is further configured to perform parallel pose detection according to the weight information of the current transported part and the part identification information to obtain a target grasping pose.

[0147] The robotic arm analysis module T4 is further configured to perform joint reasoning based on the target grasping pose, the weight information of the current transported part, and the part identification information to obtain robotic arm control information.

[0148] As an implementable manner, in this embodiment, the robotic arm analysis module T4 is further configured to perform preset partition pose detection based on the part identification information to obtain an initial grasping pose.

[0149] The robotic arm analysis module T4 is further configured to perform multi-scale fusion feature analysis according to the weight information of the current transported part to obtain pose credibility.

[0150] The robotic arm analysis module T4 is further configured to determine a target grasping pose based on the initial grasping pose and the pose credibility.

[0151] As an implementable manner, in this embodiment, the operation control module T5 is further configured to determine a target grasping position based on the target transportation trajectory, the movement speed of the current transported part, and the movement speed of the robotic arm.

[0152] The operation control module T5 is further configured to optimize the movement trajectory of the robotic arm according to the target grasping position to obtain an optimized movement trajectory.

[0153] The operation control module T5 is further configured to control the robot to transport the current transported part to the target operation position according to the target transport trajectory;

[0154] The operation control module T5 is further configured to, when the current transported part reaches the target operation position, control the robotic arm to transport the current transported part to the target operation position for adaptive operation according to the robotic arm grasping pose and the optimized motion trajectory.

[0155] The adaptive part operation system provided by the present application adopts the adaptive part operation method in the above embodiment, and can solve the technical problem of improving the part operation efficiency and accuracy. Compared with the prior art, the beneficial effects of the adaptive part operation system provided by the present application are the same as those of the adaptive part operation method provided by the above embodiment, and other technical features in the adaptive part operation system are the same as the features disclosed in the method of the above embodiment, and will not be elaborated herein.

[0156] The present application provides an adaptive part operation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the adaptive part operation method in the first embodiment above.

[0157] Next, refer to Figure 10 , which shows a schematic structural diagram of an adaptive part operation device suitable for implementing the embodiment of the present application. As Figure 10As shown, the adaptive part operation device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 into the random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the adaptive part operation device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the adaptive part operation device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows an adaptive part operation device having various systems, it should be understood that it is not required to implement or have all the shown systems. More or fewer systems may be implemented or had alternatively.

[0158] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include an adaptive part operation program product, which includes an adaptive part operation program carried on a computer-readable medium, and the adaptive part operation program contains program codes for performing the methods shown in the flowcharts. In such an embodiment, the adaptive part operation program can be downloaded and installed from the network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the adaptive part operation program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments disclosed in the present application are executed.

[0159] The adaptive part operation device provided by the present application adopts the adaptive part operation method in the above embodiments and can solve the technical problem of improving the efficiency and accuracy of part operation. Compared with the prior art, the beneficial effects of the adaptive part operation device provided by the present application are the same as those of the adaptive part operation method provided by the above embodiments, and the other technical features in the adaptive part operation device are the same as those disclosed in the method of the previous embodiment, and will not be elaborated here.

[0160] It should be understood that each part disclosed in this application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0161] The above are only the specific embodiments of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application, and all should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0162] This application provides a storage medium having computer-readable program instructions (i.e., the adaptive part operation program) stored thereon, and the computer-readable program instructions are used to execute the adaptive part operation method in the above embodiments.

[0163] The storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of the storage medium can include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the storage medium can be any tangible medium that contains or stores a program, and this program can be used by or combined with an instruction execution system, device, or device. The program code contained on the storage medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0164] The above storage medium can be included in the adaptive part operation device; or it can exist separately without being assembled into the adaptive part operation device.

[0165] The above storage medium carries one or more programs, and when the one or more programs are executed by the adaptive part operation device, the adaptive part operation device is enabled to: perform adaptive part operation.

[0166] The adaptive part operation program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above-mentioned programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0167] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and adaptive part operation program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and this module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the accompanying drawings. For example, two consecutively represented blocks can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0168] The modules involved in the embodiments of this application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation on the unit itself in some cases.

[0169] The readable storage medium provided by this application is a storage medium that stores computer-readable program instructions (i.e., the adaptive part operation program) for performing the above-mentioned adaptive part operation method, and can solve the technical problem of improving the efficiency and accuracy of part operations. Compared with the prior art, the beneficial effects of the storage medium provided by this application are the same as those of the adaptive part operation method provided by the above embodiments, and will not be elaborated here.

[0170] The above are only some embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structural transformation made under the technical concept of the present application by using the content of the specification and drawings of the present application, or any direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. An adaptive part operation method, characterized in that, The adaptive part operation method includes: Performing part target detection on the transportation image data through an improved sparse convolutional network to obtain part recognition information corresponding to the current transported part; Performing SLAM positioning based on the 3D point cloud environment data to obtain the robot position information; Determining a target transportation trajectory corresponding to the current transportation part requirement based on the collected transportation obstacle information, the weight information of the current transported part, the part recognition information, and the robot position information; Performing joint pose inference based on the weight information of the current transported part and the part recognition information to obtain manipulator control information; the manipulator control information includes the manipulator grasping pose and the manipulator movement trajectory; Performing adaptive operation on the current transported part based on the target transportation trajectory and the manipulator control information.

2. The adaptive part operation method according to claim 1, characterized in that The improved sparse convolutional network includes: an improved sparse feature extraction module, an anchor box calibration module, and a classification module; the improved sparse convolutional network is a DetNet network integrating the Inception structure; The step of performing part target detection on the transportation image data through the improved sparse convolutional network to obtain part recognition information corresponding to the current transported part includes: Performing feature analysis on the transportation image data through the improved sparse feature extraction module to obtain initial part features; Performing target detection on the initial part features according to the anchor box calibration module to obtain target part anchor boxes; Performing sample classification on the target part anchor boxes through the classification module to obtain part recognition information.

3. The adaptive part operation method according to claim 1, characterized in that The step of determining a target transportation trajectory corresponding to the current transportation part requirement based on the collected transportation obstacle information, the weight information of the current transported part, the part recognition information, and the robot position information includes: Performing rapid-exploration random tree operation based on the robot position information, the part recognition information, and the weight information of the current transported part to obtain an initial transportation trajectory; Optimizing the initial transportation trajectory according to the collected transportation obstacle information to obtain a target transportation trajectory corresponding to the current transportation part requirement.

4. The adaptive part operation method according to claim 3, wherein The step of optimizing the initial transportation trajectory according to the collected transportation obstacle information to obtain a target transportation trajectory corresponding to the current transportation part requirement includes: Performing random point sampling optimization on the initial transportation trajectory to obtain an optimized transportation trajectory; Performing greedy path iteration on the optimized transportation trajectory based on the collected transportation obstacle information to obtain a target transportation trajectory.

5. The adaptive part operation method according to claim 1, characterized in that The step of performing joint pose inference based on the weight information of the current transported part and the part recognition information to obtain manipulator control information includes: Performing parallel pose detection according to the weight information of the current transported part and the part recognition information to obtain a target grasping pose; Performing joint inference based on the target grasping pose, the weight information of the current transported part, and the part recognition information to obtain manipulator control information.

6. The adaptive part operation method according to claim 5, wherein, The step of performing parallel pose detection according to the weight information of the current transported part and the part recognition information to obtain a target grasping pose includes: Performing preset partition pose detection based on the part recognition information to obtain an initial grasping pose; Perform multi-scale fusion feature analysis based on the weight information of the current transported part to obtain the pose credibility; Determine the target grasping pose based on the initial grasping pose and the pose credibility.

7. The adaptive part operation method according to claim 1, characterized in that, The step of adaptively operating on the current transported part based on the target transportation trajectory and the robotic arm control information includes: Determine the target grasping position based on the target transportation trajectory, the movement speed of the current transported part, and the movement speed of the robotic arm; Optimize the movement trajectory of the robotic arm according to the target grasping position to obtain an optimized movement trajectory; Control the robot to transport the current transported part to the target operation position according to the target transportation trajectory; When the current transported part reaches the target operation position, control the robotic arm to transport the current transported part to the target operation position for adaptive operation according to the robotic arm grasping pose and the optimized movement trajectory.

8. An adaptive part operation system, characterized in that, The system includes: A part detection module for performing part target detection on the transportation image data through an improved sparse convolutional network to obtain part identification information corresponding to the current transported part; A robot positioning module for performing SLAM positioning based on the 3D point cloud environment data to obtain the robot position information; A trajectory planning module for determining the target transportation trajectory corresponding to the current transported part requirement based on the collected transportation obstacle information, the weight information of the current transported part, the part identification information, and the robot position information; A robotic arm analysis module for performing joint pose inference according to the weight information of the current transported part and the part identification information to obtain the robotic arm control information; the robotic arm control information includes the robotic arm grasping pose and the robotic arm movement trajectory; An operation control module for adaptively operating on the current transported part based on the target transportation trajectory and the robotic arm control information.

9. An adaptive part working device, characterized in that, The device includes: a memory, a processor, and an adaptive part operation program stored on the memory and executable on the processor, and the adaptive part operation program is configured to implement the steps of the adaptive part operation method according to any one of claims 1 to 7.

10. A storage medium, characterized in that, An adaptive part operation program is stored on the storage medium, and when the adaptive part operation program is executed by the processor, the steps of the adaptive part operation method according to any one of claims 1 to 7 are implemented.

Images