A method and system for predicting and grasping the pose of disordered workpieces based on artificial intelligence

Abstract

This invention discloses an artificial intelligence-based method and system for predicting and grasping the pose of disordered workpieces, relating to the field of robotics. The method includes: acquiring a global scene image containing multiple disordered workpieces using a visual sensor; identifying the initial pose information of the workpiece group using a preset pose prediction model, outputting a globally optimal grasping sequence and an initial motion path for grasping the first workpiece; controlling a four-bar linkage robot to grasp the workpiece; acquiring the global scene image again, and updating the workpiece group pose information generated in the previous grasp based on the global scene image; using the current position of the four-bar linkage robot's grasping end as the new path starting point, and dynamically adjusting the globally optimal grasping sequence for the remaining disordered workpieces based on the updated workpiece group pose information. This invention combines global sequence optimization with closed-loop dynamic replanning, achieving a synergistic improvement in overall grasping efficiency and single-pass success rate in disordered and dynamic scenes.

Description

Technical Field

[0001] This invention relates to the field of robotics, specifically to an artificial intelligence-based method and system for predicting and grasping the pose of disordered workpieces. Background Technology

[0002] With the rapid advancement of intelligent manufacturing and industrial automation, production lines are placing higher demands on the flexibility and intelligent operation capabilities of industrial robots. Especially in scenarios such as logistics sorting and parts loading / unloading, achieving precise and efficient grasping of randomly piled and differently shaped "out-of-order workpieces" has become a key technological bottleneck in the industry. Traditional teach-programming or fixed-path grasping methods are no longer suitable for the flexible production needs of small-batch, multi-batch operations. Developing intelligent grasping technology capable of autonomous perception, decision-making, and execution is of great significance for improving production efficiency and the level of intelligence.

[0003] Currently, some technical solutions have attempted to address the problem of out-of-order grasping by incorporating visual perception and path planning. A common approach is to use a vision system to identify the poses of multiple workpieces in a scene at once, and then generate an initial grasping sequence based on some rule (such as from near to far) or a simple optimization algorithm. During execution, these solutions typically follow a predetermined sequence for grasping, or only allow simple skipping when a single workpiece cannot be grasped due to obvious obstacles. When adjusting the grasping order, existing methods often focus on replanning based on the current static "snapshot," failing to fully consider the dynamic changes in the environment caused by the previous grasping action, such as the displacement of remaining workpieces, the exposure of occluded workpieces, etc., and also failing to use the effectiveness of historical grasping decisions as a basis for new decisions.

[0004] The existing practices mentioned above do not adequately consider the dynamic nature of the environment, limiting their application in real-world, continuously changing, and disordered scenarios. On one hand, uncorrected pose and cost models lead to replanning based on outdated or erroneous scenario information, potentially resulting in inefficient or infeasible paths, making it difficult to guarantee capture efficiency and accuracy. On the other hand, since each replanning is treated as an independent event, failing to draw on the experience of successful historical decisions, the system's decision-making may lack continuity and stability, easily generating unreasonable or jittery action sequences, affecting operational smoothness and overall efficiency.

[0005] Therefore, how to enable robots to have continuous adaptive intelligent replanning capabilities in dynamic disordered environments remains a technical problem that urgently needs to be solved. Summary of the Invention

[0006] In view of the shortcomings of the prior art, the purpose of this invention is to provide an artificial intelligence-based method and system for predicting and grasping the pose of disordered workpieces, aiming to solve the above-mentioned problems described in the prior art.

[0007] The first aspect of this invention is to provide an artificial intelligence-based method for predicting and grasping the pose of disordered workpieces, applied to a four-bar linkage robot, the method comprising:

[0008] A global scene image containing multiple disordered workpieces is acquired by visual sensors deployed at the grasping site;

[0009] Based on the global scene image, the initial pose information of the workpiece group is identified by a preset pose prediction model, and the globally optimal grasping sequence that takes into account both the probability of successful grasping and the robot's motion efficiency, as well as the initial motion path for grasping the first workpiece, are output.

[0010] The four-bar linkage robot is controlled to grasp the workpiece according to the initial motion path and transfer the workpiece to a preset position;

[0011] After the workpiece is grasped and transferred, the global scene image is acquired again, and the workpiece group pose information generated in the previous grasp is updated according to the global scene image.

[0012] Using the current position of the four-bar linkage robot's grasping end as the new path starting point, the global optimal grasping sequence of the remaining disordered workpieces is dynamically adjusted based on the updated workpiece group pose information.

[0013] According to one aspect of the above technical solution, based on the global scene image, the steps of identifying the initial pose information of the workpiece group through a preset pose prediction model, outputting a globally optimal grasping sequence that balances the probability of successful grasping and the robot's motion efficiency, and the initial motion path for grasping the first workpiece, include:

[0014] Based on the initial pose information of the workpiece group, the grasping benefit value and the motion cost of the robot moving from the current path starting point to the next workpiece are calculated for each workpiece; wherein, the grasping benefit value is determined based on the grasping success probability output by the pose prediction model;

[0015] Based on the grabbing revenue value and the movement cost, multiple candidate global grabbing sequences are generated, and the comprehensive utility value of each global grabbing sequence is calculated.

[0016] From all candidate global crawling sequences, the global crawling sequence with the highest comprehensive utility value is selected as the global optimal crawling sequence for output.

[0017] According to one aspect of the above technical solution, in the step of calculating the grasping benefit value of each workpiece and the movement cost of the robot moving from the current path starting point to the next workpiece based on the initial pose information of the workpiece group:

[0018] The expression for calculating the capture yield is:

[0019] ;

[0020] In the formula, G i To capture the profit value, P i The pose prediction model outputs the probability of successful grasping of the workpiece. V i Priority weighting coefficients are pre-defined based on workpiece type;

[0021] The expression for calculating the cost of the motion is:

[0022] ;

[0023] In the formula, C i For the cost of exercise, The duration of exercise, of which D i This is the Euclidean distance from the current path start point to the next workpiece grabbing point. vavg The average speed of the robot's end effector. k×θ sum The cost of joint movement, among which θ sum This is the sum of the absolute values ​​of the required rotation angles of each joint of the robot. k This is the time conversion factor for joint rotation.

[0024] According to one aspect of the above technical solution, the step of generating multiple candidate global crawling sequences based on the crawling revenue value and the motion cost, and calculating the comprehensive utility value of each global crawling sequence, includes:

[0025] Based on the grasping benefit value and the movement cost, multiple candidate global grasping sequences are generated using an optimization algorithm; wherein each global grasping sequence contains a unique grasping order arrangement for all workpieces in the workpiece group;

[0026] According to a preset order, the comprehensive utility value of each global crawling sequence in the multiple candidate global crawling sequences is calculated sequentially;

[0027] The calculation expression for the comprehensive utility value is as follows:

[0028] ;

[0029] In the formula, U The overall utility value, n This represents the total number of workpieces in the global capture sequence. π ( i ) represents the first digit in the global crawl sequence. iThe number of the captured workpiece. Gπ ( i ) is the first i The revenue value of grabbing each workpiece. Cπ ( i ) is from the first i The gripping point of one workpiece moves to the first... i The motion cost of the gripping point of each workpiece. α These are weighting coefficients used to balance revenue and cost;

[0030] when i When =1, Cπ (1) is the motion cost from the robot's current path starting point to the first workpiece.

[0031] According to one aspect of the above technical solution, the step of dynamically adjusting the globally optimal grasping sequence of the remaining disordered workpieces based on the updated workpiece group pose information, using the current position of the four-bar linkage robot's grasping end as the new path starting point, includes:

[0032] Based on each remaining workpiece in the updated workpiece group pose information, the grasping benefit value and motion cost are recalculated according to its state at the new path starting point.

[0033] Based on the recalculated grasping benefit value and motion cost, multiple new candidate global grasping sequences corresponding to the remaining workpiece group are generated, and the comprehensive utility value of each global grasping sequence is calculated.

[0034] From all new candidate global crawling sequences, select the global crawling sequence with the highest comprehensive utility value as the adjusted global optimal crawling sequence to guide subsequent crawling operations.

[0035] According to one aspect of the above technical solution, when recalculating the grasping benefit value and motion cost based on the updated workpiece group pose information, adaptive correction is also performed through environmental change perception, specifically including:

[0036] The workpiece whose position has changed due to the previous gripping action is identified, and its motion cost is weighted and corrected based on the displacement change. The correction formula is as follows:

[0037] ;

[0038] In the formula, Ci For the initial motion cost, Δ d This represents the change in displacement of the workpiece. γ Displacement influence factor;

[0039] And for identifying newly exposed workpieces due to the previous gripping action, the gripping benefit value is boosted with confidence based on the proportion of its visible surface area, and the correction formula is as follows:

[0040] ;

[0041] In the formula, δ The confidence coefficient is determined based on the proportion of the visible surface area of ​​the workpiece.

[0042] According to one aspect of the above technical solution, when generating multiple new candidate global fetch sequences corresponding to the remaining workpiece group and calculating the utility, an incremental optimization strategy based on historical utility is adopted, specifically including:

[0043] Calculate the structural similarity between the new global fetch sequence and the unexecuted remaining segments of the previous round's global optimal fetch sequence;

[0044] The similarity and the historical utility value of the previous round of globally optimal crawling sequence are used together as positive feedback and incorporated into the comprehensive utility calculation of the new sequence;

[0045] A convergence threshold is set. If the difference between the highest comprehensive utility value of the new sequence and the historical utility value of the previous round of global optimal crawling sequence is less than the convergence threshold, then the new sequence with the highest similarity to the remaining segment of the previous round of global optimal crawling sequence is selected.

[0046] The expression for calculating the comprehensive utility of the new sequence, which incorporates the similarity and the historical utility value of the previous sequence as positive feedback, is as follows:

[0047] ;

[0048] in, S For structural similarity, U previous This is the historical utility value of the previous round's globally optimal fetching sequence. β This is a weighting factor that is dynamically set based on the degree of change in the scenario.

[0049] A second aspect of the present invention is to provide an artificial intelligence-based system for predicting and grasping the pose of disordered workpieces, applied to a four-bar linkage robot, wherein the method includes:

[0050] The image acquisition module is used to acquire a global scene image containing multiple disordered workpieces through a vision sensor deployed at the grasping site;

[0051] The path generation module is used to identify the initial pose information of the workpiece group based on the global scene image through a preset pose prediction model, and output the globally optimal grasping sequence that takes into account both the probability of successful grasping and the robot's motion efficiency, as well as the initial motion path for grasping the first workpiece.

[0052] The grasping execution module is used to control the four-bar robot to grasp the workpiece according to the initial motion path and transfer the workpiece to a preset position;

[0053] The pose update module is used to acquire the global scene image again after the workpiece is grasped and transferred, and update the pose information of the workpiece group generated in the previous grasping according to the global scene image.

[0054] The dynamic sequencing module is used to dynamically adjust the global optimal grasping sequence of the remaining disordered workpieces based on the updated workpiece group pose information, taking the current position of the grasping end of the four-bar robot as the new path starting point.

[0055] A third aspect of the present invention provides a readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the method described in the above-described technical solutions.

[0056] A fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described in the above technical solutions.

[0057] Compared with existing technologies, the advantages of using the AI-based method and system for predicting and grasping disordered workpiece poses as shown in this invention are as follows:

[0058] The method presented in this invention combines global sequence optimization with closed-loop dynamic replanning to achieve a synergistic improvement in overall grasping efficiency and single-shot success rate in disordered and dynamic scenarios. Specifically, the method not only plans a globally near-optimal grasping sequence based on initial perception to minimize the total movement path, but also dynamically adjusts the subsequent sequence after each grasping action through re-perception and rolling optimization starting from the robot's real-time position. This effectively addresses scene changes and uncertainties caused by grasping actions, forming a continuously adaptive intelligent closed loop of perception-planning-execution-optimization, significantly enhancing robustness and operational efficiency in real industrial environments. Attached Figure Description

[0059] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0060] Figure 1 This is a flowchart illustrating the artificial intelligence-based method for predicting and capturing the pose of disordered workpieces, as provided in an embodiment of the present invention.

[0061] Figure 2 This is a structural block diagram of an artificial intelligence-based disordered workpiece pose prediction and grasping system provided in an embodiment of the present invention. Detailed Implementation

[0062] To make the objectives, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Several embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of the present invention will be more thorough and complete.

[0063] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0065] Example 1

[0066] Please see Figure 1 The first embodiment of the present invention provides an artificial intelligence-based method for predicting and grasping the pose of disordered workpieces, applied to a four-bar linkage robot. The method includes steps S10-S50:

[0067] Step S10: Acquire a global scene image containing multiple disordered workpieces using a vision sensor deployed at the grasping site.

[0068] First, it should be noted that the four-bar linkage robot shown in this embodiment includes four links connected end to end, which are connected to the output end of a drive mechanism. The drive mechanism is, for example, a drive motor. The connection between the four-bar linkage robot and the output end is a fixed connection. For example, the links of the four-bar linkage robot are directly fixed to the output end. Or, in some other feasible embodiments, the four-bar linkage robot and the output end of the drive mechanism can also be connected in a liftable manner through a transfer device.

[0069] In this embodiment, the method shown is an artificial intelligence-based method for predicting and grasping the pose of disordered workpieces. The disordered workpieces refer to multiple workpieces that are stacked, occluded, or otherwise separated in the grasping area and lack obvious patterns. Therefore, before grasping the disordered workpieces, it is necessary to first predict their poses and then perform a series of analyses to grasp them. The prediction of the poses of the disordered workpieces includes acquiring a global scene image containing multiple disordered workpieces through a visual sensor deployed at the grasping site.

[0070] Among them, the vision sensor, such as an industrial camera, can be fixed by a specific bracket, thereby allowing the acquisition of global scene images of multiple disordered workpieces at any time. It can also be set on a four-bar robot, preferably at the link joint, so that the four-bar robot can quickly acquire global scene images of multiple disordered workpieces during its movement.

[0071] Step S20: Based on the global scene image, identify the initial pose information of the workpiece group through a preset pose prediction model, output the globally optimal grasping sequence that takes into account both the probability of successful grasping and the robot's motion efficiency, as well as the initial motion path for grasping the first workpiece.

[0072] In this embodiment, based on the global scene image, the initial pose information of the workpiece group is identified through a preset pose prediction model, and a globally optimal grasping sequence that balances the probability of successful grasping and the robot's motion efficiency is output, along with the initial motion path for grasping the first workpiece. This includes:

[0073] Based on the initial pose information of the workpiece group, the grasping benefit value and the motion cost of the robot moving from the current path starting point to the next workpiece are calculated for each workpiece; wherein, the grasping benefit value is determined based on the grasping success probability output by the pose prediction model;

[0074] Based on the grabbing revenue value and the movement cost, multiple candidate global grabbing sequences are generated, and the comprehensive utility value of each global grabbing sequence is calculated.

[0075] From all candidate global crawling sequences, the global crawling sequence with the highest comprehensive utility value is selected as the global optimal crawling sequence for output.

[0076] Among them, the steps of calculating the grasping benefit value of each workpiece and the motion cost of the robot moving from the current path starting point to the next workpiece based on the initial pose information of the workpiece group are as follows:

[0077] The expression for calculating the capture yield is:

[0078] ;

[0079] In the formula,G i To capture the profit value, P i The pose prediction model outputs the probability of successful grasping of the workpiece. V i Priority weighting coefficients are pre-defined based on workpiece type;

[0080] The expression for calculating the cost of the motion is:

[0081] ;

[0082] In the formula, C i For the cost of exercise, The duration of exercise, of which D i This is the Euclidean distance from the current path start point to the next workpiece grabbing point. vavg The average speed of the robot's end effector. k×θ sum The cost of joint movement, among which θ sum This is the sum of the absolute values ​​of the required rotation angles of each joint of the robot. k This is the time conversion factor for joint rotation.

[0083] The step of generating multiple candidate global crawling sequences based on the crawling revenue value and the motion cost, and calculating the comprehensive utility value of each global crawling sequence, includes:

[0084] Based on the grasping benefit value and the movement cost, multiple candidate global grasping sequences are generated using an optimization algorithm; wherein each global grasping sequence contains a unique grasping order arrangement for all workpieces in the workpiece group;

[0085] According to a preset order, the comprehensive utility value of each global crawling sequence in the multiple candidate global crawling sequences is calculated sequentially.

[0086] The calculation expression for the comprehensive utility value is as follows:

[0087] ;

[0088] In the formula, U The overall utility value, n This represents the total number of workpieces in the global capture sequence. π ( i ) represents the first digit in the global crawl sequence. i The number of the captured workpiece. Gπ ( i ) is the first i The revenue value of grabbing each workpiece.Cπ ( i ) is from the first i The gripping point of one workpiece moves to the first... i The motion cost of the gripping point of each workpiece. α These are weighting coefficients used to balance revenue and cost;

[0089] when i When =1, Cπ (1) is the motion cost from the robot's current path starting point to the first workpiece.

[0090] Specifically, in this embodiment, a quantitative, multi-objective optimization decision model is constructed to replace the traditional method of generating grasping sequences that relies on human experience or simple rules. This model unifies the two optimization objectives of grasping success probability and movement efficiency into a computable mathematical framework through two core quantitative indicators: grasping benefit value and movement cost. It also evaluates any possible grasping order using a single scalar of comprehensive utility value, thereby automatically and efficiently finding the globally optimal or near-optimal grasping sequence.

[0091] First, the calculation of the capture revenue value integrates two layers of intelligence: perception and business. Among them, the probability of successful capture... P i Derived from an AI-based pose prediction model, it predicts the probability of successful grasping under current conditions based on visual information. Priority weight coefficients. V i These are pre-defined priority weights that represent the business logic in the production process, such as prioritizing the acquisition of precision workpieces, high-value workpieces, or critical workpieces that act as bottlenecks in downstream processes. By multiplying these two factors, the probability of success is combined with the value of acquisition, enabling decisions to consider not only whether acquisition is possible but also whether acquisition should be prioritized, thus achieving a shift from low-level perception to high-level task planning.

[0092] Secondly, the calculation of motion cost accurately models the actual motion loss of the four-bar linkage robot, rather than simply using spatial distance. D i Instead, it is converted into the expected motion time ( This is directly related to the core indicator of production efficiency, namely cycle time. At the same time, it innovatively introduces the cost of joint movement (…). k×θ sum Even if the end-effectors move similar distances, different combinations of joint rotations can lead to different energy consumption, wear and tear, and time consumption.

[0093] Finally, the overall utility value is calculated by generating multiple candidate sequences through enumeration or heuristic algorithms, simulating the execution for each sequence, and summing their total benefit and total cost. This model maximizes the overall utility value.U It seeks the sequence with the largest difference between total revenue and total cost, that is, to maximize the overall value under the constraints of limited resources (time, amount of exercise), thus achieving global sequence optimization.

[0094] Step S30: Control the four-bar linkage robot to grasp the workpiece according to the initial motion path and transfer the workpiece to the preset position.

[0095] In this embodiment, after controlling the four-bar linkage robot to move along the path to the precise grasping posture, the system will drive the end effector, such as a pneumatic gripper or an electric suction cup, to perform the grasping action. After successful grasping, the robot will continue to move the workpiece to the preset placement position along the planned path or another transfer path, such as transferring the workpiece to a conveyor belt.

[0096] Step S40: After the workpiece is grasped and transferred, the global scene image is acquired again, and the workpiece group pose information generated in the previous grasp is updated according to the global scene image.

[0097] It should be noted that after step S30 is executed, the physical state of the work scene has changed fundamentally. One of the workpieces has been removed, and the remaining workpieces may be displaced, have changed posture, or even roll over due to slight collisions or vibrations during the grasping process. The workpieces that were previously blocked may also be partially or completely exposed.

[0098] Therefore, in this embodiment, a global scene image will be acquired again using a visual sensor. The purpose is to re-perceive the latest state and obtain an accurate and up-to-date understanding of the true situation of the remaining workpiece group. Subsequently, based on this new scene image, the pose prediction model is invoked to re-identify and estimate the pose of the remaining workpieces that may have changed, thereby updating the pre-generated workpiece group pose information in real time.

[0099] Step S50: Using the current position of the four-bar robot's grasping end as the new path starting point, dynamically adjust the global optimal grasping sequence of the remaining disordered workpieces based on the updated workpiece group pose information.

[0100] In this embodiment, step S50 starts from the robot's latest position and re-executes global sequence planning based on the updated scene information, forming a closed loop of execution-perception-replanning. This step, through rolling temporal optimization, responds in real time to changes in the environment and robot state, dynamically adjusting the subsequent grasping order to ensure that each step of the robot's movement maintains local optimality based on the current latest state. This allows for the continuous maximization of overall operational efficiency while dealing with the uncertainties of disordered scenarios.

[0101] Compared with existing technologies, the artificial intelligence-based method for predicting and grasping disordered workpiece poses, as shown in this embodiment, has the following advantages:

[0102] The method described in this embodiment combines global sequence optimization with closed-loop dynamic replanning to achieve a synergistic improvement in overall grasping efficiency and single-shot success rate in disordered and dynamic scenarios. Specifically, the method not only plans a globally near-optimal grasping sequence based on initial perception to minimize the total movement path, but also dynamically adjusts the subsequent sequence after each grasping action through re-perception and rolling optimization starting from the robot's real-time position. This effectively addresses scene changes and uncertainties caused by grasping actions, forming a continuously adaptive intelligent closed loop of perception-planning-execution-optimization, significantly enhancing robustness and operational efficiency in real industrial environments.

[0103] Example 2

[0104] The second embodiment of the present invention also provides a method for predicting and grasping the pose of disordered workpieces based on artificial intelligence. The method shown in this embodiment is basically similar to the method shown in the first embodiment, except that:

[0105] In this embodiment, the step of dynamically adjusting the globally optimal grasping sequence of the remaining disordered workpieces based on the updated workpiece group pose information, using the current position of the four-bar linkage robot's grasping end as the new path starting point, includes:

[0106] Based on each remaining workpiece in the updated workpiece group pose information, the grasping benefit value and motion cost are recalculated according to its state at the new path starting point.

[0107] Based on the recalculated grasping benefit value and motion cost, multiple new candidate global grasping sequences corresponding to the remaining workpiece group are generated, and the comprehensive utility value of each global grasping sequence is calculated.

[0108] From all new candidate global crawling sequences, select the global crawling sequence with the highest comprehensive utility value as the adjusted global optimal crawling sequence to guide subsequent crawling operations.

[0109] Furthermore, when recalculating the grasping benefit value and motion cost based on the updated workpiece group pose information, adaptive correction is also performed through environmental change perception, specifically including:

[0110] The workpiece whose position has changed due to the previous gripping action is identified, and its motion cost is weighted and corrected based on the displacement change. The correction formula is as follows:

[0111] ;

[0112] In the formula, Ci For the initial motion cost, Δ d This represents the change in displacement of the workpiece. γ Displacement influence factor;

[0113] And for identifying newly exposed workpieces due to the previous gripping action, the gripping benefit value is boosted with confidence based on the proportion of its visible surface area, and the correction formula is as follows:

[0114] ;

[0115] In the formula, δ The confidence coefficient is determined based on the proportion of the visible surface area of ​​the workpiece.

[0116] Furthermore, when generating multiple new candidate global fetch sequences corresponding to the remaining workpiece group and calculating the utility, an incremental optimization strategy based on historical utility is adopted, specifically including:

[0117] Calculate the structural similarity between the new global fetch sequence and the unexecuted remaining segments of the previous round's global optimal fetch sequence;

[0118] The similarity and the historical utility value of the previous round of globally optimal crawling sequence are used together as positive feedback and incorporated into the comprehensive utility calculation of the new sequence;

[0119] A convergence threshold is set. If the difference between the highest comprehensive utility value of the new sequence and the historical utility value of the previous round of global optimal crawling sequence is less than the convergence threshold, then the new sequence with the highest similarity to the remaining segment of the previous round of global optimal crawling sequence is selected.

[0120] The expression for calculating the comprehensive utility of the new sequence, which incorporates the similarity and the historical utility value of the previous sequence as positive feedback, is as follows:

[0121] ;

[0122] in, S For structural similarity, U previous This is the historical utility value of the previous round's globally optimal fetching sequence. β This is a weighting factor that is dynamically set based on the degree of change in the scenario.

[0123] Specifically, in this embodiment, based on the intelligent closed loop of perception-planning-execution-replanning constructed in the first embodiment, dynamic replanning is further deepened and refined, focusing on the specific implementation of step S50, and endowed with a higher level of environmental understanding and historical learning ability, thereby further improving the quality and efficiency of decision-making in extremely chaotic and dynamic scenarios.

[0124] First, this embodiment does not simply fine-tune the original sequence, but takes the robot's new end position as the absolute starting point, treats the remaining workpiece as a completely new subtask, and re-executes the complete optimization process, which includes benefit / cost calculation, candidate sequence generation, and utility evaluation. This ensures that every decision is based on fully updated state information, which is a manifestation of the rigorous rolling time-domain optimization concept.

[0125] Secondly, this embodiment also introduces an adaptive cost-benefit correction mechanism based on environmental change perception, which can intelligently identify two types of workpieces whose state has changed drastically due to the last grasp: one is the workpiece that has been displaced, whose motion cost will be determined by the formula... The increased penalty, along with the introduction of the coefficient γ and displacement Δd, allows cost calculations to quantify the uncertainty of the workpiece position, guiding the robot to prioritize grasping workpieces with more stable positions and reducing the risk of failure. Secondly, the grasping benefit for newly exposed workpieces is calculated using the formula... Increased confidence is achieved. The coefficient δ (typically ≥1) is linked to its visible surface area, meaning the system assigns a higher grasping confidence value to workpieces that are more fully visible, resulting in more accurate benefit assessment.

[0126] Finally, this embodiment proposes an incremental optimization strategy based on historical utility, which calculates the structural similarity between the new candidate sequence and the unexecuted portion of the previous round's optimal sequence. S Formulating historical success experiences Quantitatively incorporate into current decision-making. Dynamic trade-off coefficients. β This allows the system to achieve an intelligent balance between relying on current new information and drawing on good historical experience. A further convergence threshold determination mechanism, when performance improvements are not significant, tends to select the sequence most similar to historical plans. This greatly enhances the continuity and stability of decision-making, avoids unnecessary policy fluctuations, and makes the robot's actions smoother and more reliable.

[0127] In summary, the method shown in this embodiment upgrades dynamic replanning from basic recalculation to an intelligent adaptive process that senses environmental changes and integrates historical experience through environmental adaptive correction and historical incremental optimization. This not only solves the uncertainty problem caused by the dynamic nature of the scene, but also improves the efficiency and quality of planning by utilizing effective information from past decisions. As a result, the entire grasping system exhibits stronger robustness, higher success rate, and better overall operational efficiency when facing complex out-of-order workpieces.

[0128] Example 3

[0129] Please see Figure 2 The third embodiment of the present invention provides an artificial intelligence-based system for predicting and grasping the pose of disordered workpieces, applied to a four-bar linkage robot. The method includes:

[0130] The image acquisition module 10 is used to acquire a global scene image containing multiple disordered workpieces through a vision sensor deployed at the grasping site;

[0131] The path generation module 20 is used to identify the initial pose information of the workpiece group based on the global scene image through a preset pose prediction model, and output the globally optimal grasping sequence that takes into account both the probability of successful grasping and the robot's motion efficiency, as well as the initial motion path for grasping the first workpiece.

[0132] The grasping execution module 30 is used to control the four-bar linkage robot to grasp the workpiece according to the initial motion path and transfer the workpiece to a preset position;

[0133] The pose update module 40 is used to acquire the global scene image again after the workpiece is grasped and transferred, and update the pose information of the workpiece group generated in the previous grasping according to the global scene image.

[0134] The dynamic sequencing module 50 is used to dynamically adjust the global optimal grasping sequence of the remaining disordered workpieces based on the updated workpiece group pose information, taking the current position of the grasping end of the four-bar robot as the new path starting point.

[0135] Compared with existing technologies, the artificial intelligence-based disordered workpiece pose prediction and grasping system shown in this embodiment has the following advantages:

[0136] The system shown in this embodiment combines global sequence optimization with closed-loop dynamic replanning to achieve a synergistic improvement in overall grasping efficiency and single-shot success rate in disordered and dynamic scenarios. Specifically, the system not only plans a globally near-optimal grasping sequence based on initial perception to minimize the total movement path, but also dynamically adjusts the subsequent sequence after each grasping action through re-perception and rolling optimization starting from the robot's real-time position. This effectively addresses scene changes and uncertainties caused by grasping actions, forming a continuously adaptive intelligent closed loop of perception-planning-execution-optimization, significantly enhancing robustness and operational efficiency in real industrial environments.

[0137] Example 4

[0138] A fourth embodiment of the present invention provides a readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the methods described in the above embodiments.

[0139] Example 5

[0140] A fifth embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the methods described in the above embodiments.

[0141] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0142] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

Claims

1. A method for predicting and grasping the pose of disordered workpieces based on artificial intelligence, characterized in that, Applied to a four-bar linkage robot, the method includes: A global scene image containing multiple disordered workpieces is acquired by visual sensors deployed at the grasping site; Based on the global scene image, the initial pose information of the workpiece group is identified by a preset pose prediction model, and the globally optimal grasping sequence that takes into account both the probability of successful grasping and the robot's motion efficiency, as well as the initial motion path for grasping the first workpiece, is output. The four-bar linkage robot is controlled to grasp the workpiece according to the initial motion path and transfer the workpiece to a preset position; After the workpiece is grasped and transferred, the global scene image is acquired again, and the workpiece group pose information generated in the previous grasp is updated according to the global scene image. Using the current position of the four-bar robot's grasping end as the new path starting point, and based on the updated workpiece group pose information, dynamically adjust the global optimal grasping sequence of the remaining disordered workpieces. The steps include: identifying the initial pose information of the workpiece group based on the global scene image using a preset pose prediction model; outputting a globally optimal grasping sequence that balances the probability of successful grasping with the robot's motion efficiency; and the initial motion path for grasping the first workpiece. Based on the initial pose information of the workpiece group, the grasping benefit value and the motion cost of the robot moving from the current path starting point to the next workpiece are calculated for each workpiece; wherein, the grasping benefit value is determined based on the grasping success probability output by the pose prediction model; Based on the grabbing revenue value and the movement cost, multiple candidate global grabbing sequences are generated, and the comprehensive utility value of each global grabbing sequence is calculated. From all candidate global crawling sequences, the global crawling sequence with the highest comprehensive utility value is selected as the global optimal crawling sequence for output.

2. The method for predicting and grasping the pose of disordered workpieces based on artificial intelligence according to claim 1, characterized in that, Based on the initial pose information of the workpiece group, the steps of calculating the grasping benefit value of each workpiece and the motion cost of the robot moving from the current path starting point to the next workpiece are as follows: The expression for calculating the crawling revenue value is: ； In the formula, G i To capture the profit value, P i The pose prediction model outputs the probability of successful grasping of the workpiece. V i Priority weighting coefficients are pre-defined based on workpiece type; The expression for calculating the cost of the motion is: ； In the formula, C i For the cost of exercise, The duration of exercise, of which D i This is the Euclidean distance from the current path start point to the next workpiece grabbing point. vavg The average speed of the robot's end effector. k×θ sum The cost of joint movement, among which θ sum This is the sum of the absolute values ​​of the required rotation angles of each joint of the robot. k This is the time conversion factor for joint rotation.

3. The method for predicting and grasping the pose of disordered workpieces based on artificial intelligence according to claim 2, characterized in that, The steps of generating multiple candidate global crawling sequences based on the crawling revenue value and the motion cost, and calculating the comprehensive utility value of each global crawling sequence, include: Based on the grasping benefit value and the movement cost, multiple candidate global grasping sequences are generated using an optimization algorithm; wherein each global grasping sequence contains a unique grasping order arrangement for all workpieces in the workpiece group; According to a preset order, the comprehensive utility value of each global crawling sequence in the multiple candidate global crawling sequences is calculated sequentially; The calculation expression for the comprehensive utility value is as follows: ； In the formula, U The overall utility value, n This represents the total number of workpieces in the global capture sequence. π ( i ) represents the first digit in the global crawl sequence. i The number of the captured workpiece. Gπ ( i ) is the first i The revenue value of grabbing each workpiece. Cπ ( i ) is from the first i The gripping point of one workpiece moves to the first... i The motion cost of the gripping point of each workpiece. α These are weighting coefficients used to balance revenue and cost; when i When =1, Cπ (1) is the motion cost from the robot's current path starting point to the first workpiece.

4. The method for predicting and grasping the pose of disordered workpieces based on artificial intelligence according to any one of claims 1-3, characterized in that, The steps of dynamically adjusting the globally optimal grasping sequence of the remaining disordered workpieces based on the updated workpiece group pose information, using the current position of the grasping end of the four-bar robot as the new path starting point, include: Based on each remaining workpiece in the updated workpiece group pose information, the grasping benefit value and motion cost are recalculated according to its state at the new path starting point. Based on the recalculated grasping benefit value and motion cost, multiple new candidate global grasping sequences corresponding to the remaining workpiece group are generated, and the comprehensive utility value of each global grasping sequence is calculated. From all new candidate global crawling sequences, select the global crawling sequence with the highest comprehensive utility value as the adjusted global optimal crawling sequence to guide subsequent crawling operations.

5. The method for predicting and grasping the pose of disordered workpieces based on artificial intelligence according to claim 4, characterized in that, When recalculating the grasping benefit value and motion cost based on the updated workpiece group pose information, adaptive correction is also performed through environmental change perception, specifically including: The workpiece whose position has changed due to the previous gripping action is identified, and its motion cost is weighted and corrected based on the displacement change. The correction formula is as follows: ； In the formula, Ci For the initial motion cost, Δ d This represents the change in displacement of the workpiece. γ Displacement influence factor; And for identifying newly exposed workpieces due to the previous gripping action, the gripping benefit value is boosted with confidence based on the proportion of its visible surface area, and the correction formula is as follows: ； In the formula, δ The confidence coefficient is determined based on the proportion of the visible surface area of ​​the workpiece.

6. The method for predicting and grasping the pose of disordered workpieces based on artificial intelligence according to claim 4, characterized in that, When generating multiple new candidate global fetch sequences corresponding to the remaining workpiece group and calculating their utility, an incremental optimization strategy based on historical utility is adopted, specifically including: Calculate the structural similarity between the new global fetch sequence and the unexecuted remaining segments of the previous round's global optimal fetch sequence; The similarity and the historical utility value of the previous round of globally optimal crawling sequence are used together as positive feedback and incorporated into the comprehensive utility calculation of the new sequence; A convergence threshold is set. If the difference between the highest comprehensive utility value of the new sequence and the historical utility value of the previous round of global optimal crawling sequence is less than the convergence threshold, then the new sequence with the highest similarity to the remaining segment of the previous round of global optimal crawling sequence is selected. The expression for calculating the comprehensive utility of the new sequence, which incorporates the similarity and the historical utility value of the previous sequence as positive feedback, is as follows: ； in, S For structural similarity, U previous This is the historical utility value of the previous round's globally optimal fetching sequence. β This is a weighting factor that is dynamically set based on the degree of change in the scenario.

7. A system for predicting and grasping the pose of disordered workpieces based on artificial intelligence, characterized in that, Applied to a four-bar linkage robot, implementing the method according to any one of claims 1-6, the system comprises: The image acquisition module is used to acquire a global scene image containing multiple disordered workpieces through a vision sensor deployed at the grasping site; The path generation module is used to identify the initial pose information of the workpiece group based on the global scene image through a preset pose prediction model, and output the globally optimal grasping sequence that takes into account both the probability of successful grasping and the robot's motion efficiency, as well as the initial motion path for grasping the first workpiece. The grasping execution module is used to control the four-bar robot to grasp the workpiece according to the initial motion path and transfer the workpiece to a preset position; The pose update module is used to acquire the global scene image again after the workpiece is grasped and transferred, and update the pose information of the workpiece group generated in the previous grasping according to the global scene image. The dynamic sequencing module is used to dynamically adjust the global optimal grasping sequence of the remaining disordered workpieces based on the updated workpiece group pose information, taking the current position of the grasping end of the four-bar robot as the new path starting point.

8. A readable storage medium having computer instructions stored thereon, characterized in that, When executed by the processor, this instruction implements the steps of the method as described in any one of claims 1-6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1-6.

Images