Industrial robot trajectory control method based on visual image

Abstract

The invention relates to the field of trajectory control, in particular to an industrial robot trajectory control method based on a visual image, and the method comprises the steps: collecting execution data of a mechanical assembly executing a task and image data corresponding to the execution task in real time; generating a plurality of movable tracks for the mechanical assembly to execute the task, and determining a track complexity characterization value according to the number of corresponding obstacles in each movable track and the number of times of rotation required by the mechanical assembly, so as to sort each movable track and determine an initial moving track; acquiring distribution characteristics of the target object according to the image data, and calculating execution constraint characterization parameters of the mechanical component so as to mark the mechanical component; responding to a marking result, and analyzing an execution process corresponding to the mechanical component; the moving track is adjusted according to the rising duration of the joint current and the number of jammed joints, the dynamic environment is effectively responded, intelligence and adaptation of the moving track are achieved, and meanwhile the stability and safety of task execution are guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of trajectory control, in particular to a trajectory control method for industrial robot based on visual image. BACKGROUND

[0002] In the scene of high-mixed logistics sorting and dense assembly, robots are essential automation equipment. When selecting target objects, the moving trajectory of the robot needs to accurately avoid obstacles, and the trajectory control needs to meet the "target tracking" and "obstacle avoidance constraint" at the same time. Meanwhile, there are real-time changes in the environment (such as new obstacles). With the continuous development of machine vision technology and image processing algorithms, the visual system can more accurately perceive and recognize the environment and targets, providing high-precision target positioning and attitude information. It can detect and identify target objects of different shapes and sizes, providing a better perception means for industrial robot trajectory control, improving the adaptability and operation flexibility of industrial robots in complex environments, and making the trajectory control method based on visual image have the basis for practical application.

[0003] Chinese patent application publication No. CN120080325A discloses an industrial robot manipulator operation trajectory planning control system, relating to the field of industrial vision technology. The system includes four modules: an image acquisition and preprocessing module that acquires images in real time and creates a three-dimensional model; an obstacle identification module that matches the model to find obstacles; a collision prediction module that provides early warning and predicts the time; and a trajectory planning module that plans the trajectory after comparing the time and returns to the preprocessing module. The invention defines the early warning range through three-dimensional modeling, calculates the obstacle movement parameters, uses cosine similarity to screen key positions and quantifies risks to predict the time, identifies possible collisions in advance, reduces risks, adjusts the pace based on the predicted time, and ensures efficient and stable operation.

[0004] However, the prior art still has the following problems, Traditional trajectory control methods mainly rely on pre-programmed paths and fixed sensor feedback, which cannot adapt to scenes with high-mixed objects and dynamic changes in the environment, reducing the flexibility of the moving trajectory. SUMMARY

[0005] Therefore, the present application provides a trajectory control method for industrial robot based on visual image to overcome the problem that the prior art cannot adapt to scenes with high-mixed objects and dynamic changes in the environment, reducing the flexibility of the moving trajectory.

[0006] To achieve the above purpose, the present application provides a trajectory control method for industrial robot based on visual image, which includes: real-time acquisition of execution data of mechanical components performing tasks and image data corresponding to the execution tasks; Several movable trajectories are generated for the mechanical component to perform the task. The complexity characterization value of the trajectory is determined based on the number of obstacles in each movable trajectory and the number of rotations required by the mechanical component. The movable trajectories are then sorted to determine the initial movement trajectory. Based on the image data, the distribution characteristics of the target object are obtained, and the execution constraint characterization parameters of the mechanical component are calculated to mark the mechanical component. The distribution characteristics include the occluded area of ​​the target object and the shortest distance between the target object and other objects. In response to the marking results, the execution process corresponding to the mechanical component is analyzed, including, The second movable trajectory of the mechanical component is changed, and the grasping state characteristics after the robotic arm grasps the target object and the duration of the linear movement of the robotic arm are combined to evaluate the connection stability characterization coefficient of the mechanical component in order to determine whether the execution process has entered the execution disorder stage. Based on the duration of the increase in joint current and the number of jammed joints, the movement trajectory is adjusted, including switching to a safe return trajectory or immediately stopping the current movement trajectory. The grasping state characteristics include the contraction range and the number of times the robotic arm opens and closes.

[0007] Furthermore, the process of determining the trajectory complexity characterization value includes: The ratio of the number of obstacles in the movable trajectory to the obstacle number threshold is used as the first trajectory complexity feature; The ratio of the number of rotations required for the mechanical component to the threshold number of rotations required is used as the second trajectory complexity feature; The sum of the first trajectory complexity features and the second trajectory complexity features is used as the trajectory complexity characterization value.

[0008] Furthermore, the process of sorting the movable trajectories to determine the initial movement trajectory includes: Obtain the trajectory complexity representation value corresponding to each movable trajectory; The movable trajectories are sorted in descending order based on their complexity values. Based on the descending sorting result, a sequence of movable trajectories is obtained; The movable trajectory corresponding to the first position of the movable trajectory sequence is determined as the initial movable trajectory.

[0009] Furthermore, the process of calculating the execution constraint characterization parameters of the mechanical component includes: The ratio of the occluded area of ​​the target object to the occluded area threshold is used as the first execution constraint feature. The ratio of the shortest distance threshold to the shortest distance between the target object and other objects is used as the second execution constraint feature; The sum of the first execution constraint feature and the second execution constraint feature is used as the execution constraint representation parameter.

[0010] Furthermore, the mechanical components are marked, including: If the execution constraint characterization parameter of a mechanical component is greater than or equal to the execution constraint characterization parameter threshold, then the mechanical component is marked as an execution-disrupted component.

[0011] Furthermore, in response to the marking results, the execution process corresponding to the mechanical component is analyzed, including: If any mechanical component is marked as an execution-disrupted component, then the execution process corresponding to that mechanical component is analyzed.

[0012] Furthermore, the process of changing the movable trajectory of the mechanical component includes: Call the movable trajectory sequence; Based on the sequence of movable trajectories, the corresponding second movable trajectories are determined sequentially.

[0013] Furthermore, the process of evaluating the connection stability characterization coefficient of the mechanical components includes: The sum of the ratio of the contraction amplitude of the robotic arm to the contraction amplitude threshold and the ratio of the number of opening and closing operations to the number of opening and closing operations threshold is used as the first connection stability feature. The ratio of the duration of linear movement of the robotic arm to the threshold duration of linear movement is used as the second connection stability feature. The first connection stability feature and the second connection stability feature are weighted and summed to determine the connection stability characterization coefficient.

[0014] Furthermore, determining whether the execution process has entered the stage of execution disorder includes: If the stability coefficient of the connection is greater than or equal to the threshold of the stability coefficient of the connection, the execution process is determined to have entered the execution disorder stage.

[0015] Furthermore, the process of adjusting the movement trajectory includes: If the duration of the increase in joint current exceeds the threshold for duration of increase or / and the number of jammed joints in the mechanical components exceeds the threshold for the number of jammed joints, then the current movement trajectory shall be stopped immediately. Otherwise, switch to a safe rollback path.

[0016] Compared with existing technologies, this invention collects execution data and corresponding image data of mechanical components performing tasks in real time; generates several movable trajectories for the mechanical components to perform tasks; determines the trajectory complexity characterization value based on the number of obstacles in each movable trajectory and the number of rotations required by the mechanical component, and sorts the movable trajectories to determine the initial movement trajectory; obtains the distribution characteristics of the target object based on the image data, calculates the execution constraint characterization parameters of the mechanical component, and marks the mechanical component; analyzes the execution process corresponding to the mechanical component in response to the marking results; and adjusts the movement trajectory based on the duration of the joint current increase and the number of jammed joints. This invention effectively responds to the dynamic environment, realizes the intelligence and adaptability of the movement trajectory, and at the same time ensures the stability and safety of task execution.

[0017] In particular, this invention determines the priority of movable trajectories based on environmental factors of several movable trajectories. Given multiple selectable trajectories, and avoiding the rigidity of a single predetermined trajectory, the number of obstacles in a movable trajectory directly reflects the trajectory risk during initial selection. Prioritizing trajectories with fewer obstacles reduces the probability of collisions between mechanical components and obstacles. Simultaneously, prioritizing trajectories requiring fewer rotational operations from mechanical components reduces frequent starts and stops of joints and attitude adjustments, thereby reducing mechanical wear and energy consumption. This invention effectively responds to dynamic environments and achieves intelligent and adaptive movable trajectories by dynamically generating and intelligently sorting the trajectories for mechanical components performing tasks, while ensuring the stability and safety of task execution.

[0018] In particular, this invention considers the complexity of the environment in which the target object is located. By determining the occluded area of ​​the target object, it identifies the visibility risk of the robotic arm's grasping point. Simultaneously, during the grasping process, it is crucial to ensure that the robotic arm maintains a safe distance from surrounding objects as it approaches the target object to avoid collisions due to confined space. Therefore, this invention uses visual data-driven quantitative analysis of the execution constraints of the mechanical components to calculate the execution constraint representation parameters of the mechanical components. This characterizes the degree of interference from environmental factors surrounding the target object when the mechanical components grasp it, thereby quantifying the limitations on grasping the target object and providing data support for subsequent labeling of the mechanical components. This invention effectively responds to dynamic environments, achieving intelligent and adaptive movement trajectories while ensuring the stability and safety of task execution.

[0019] In particular, this invention analyzes mechanical components marked as experiencing execution interference, prioritizing dynamic trajectory switching. While reducing the degree of obstruction to subsequent task execution, it considers the state characteristics exhibited by the mechanical components during execution. Specifically, the contraction amplitude of the robotic arm as it grasps the target object reflects the adaptability of the robotic arm's gripping force; furthermore, it identifies over-adjustment behavior by monitoring the number of opening and closing cycles of the robotic arm; and the robotic arm experiences a certain degree of positioning drift due to motion inertia, with excessive positioning drift affecting the grasping quality. Based on the combination of these characteristics, it determines whether the mechanical components are in an abnormal friction state. Therefore, this invention quantifies the stability of the corresponding robotic arm and robotic arm's coordinated operation by evaluating the connection stability characterization coefficient of the mechanical components, providing data support for subsequent determination of whether the execution process has entered a stage of execution disorder. This invention effectively responds to dynamic environments, achieving intelligent and adaptive movement trajectories while ensuring the stability and safety of task execution.

[0020] In particular, after entering the execution disorder stage, this invention further analyzes the abnormal state of the entire mechanical system and takes corresponding measures. It monitors the duration of joint current elevation, accurately distinguishes between instantaneous load fluctuations and continuous abnormalities, and reflects the severity of continuous abnormalities. At the same time, it counts the number of jammed joints of the mechanical components, quantifies the fault risk, and then determines whether to urgently stop the movement of the mechanical components through dual criteria to avoid accidental emergency stops. In addition, combined with flexible adjustment measures, it switches to a safe return trajectory only when a single condition is met, which can preserve the task state of the mechanical components and quickly resume task execution after the fault is eliminated. Based on this, this invention achieves intelligent and adaptive movement trajectory through real-time monitoring and dynamic response of joint current and jamming state, while ensuring the stability and safety of task execution. Attached Figure Description

[0021] Figure 1 This is a schematic diagram illustrating the steps of a vision-based industrial robot trajectory control method according to an embodiment of the invention. Figure 2 A logic diagram for marking mechanical components in an embodiment of the invention; Figure 3 A logic diagram for determining whether the execution process has entered the execution disorder stage in an embodiment of the invention; Figure 4 This is a logic diagram for adjusting the movement trajectory according to an embodiment of the invention. Detailed Implementation

[0022] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0023] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0024] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0025] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0026] Please see Figure 1 The diagram illustrates the steps of a vision-based industrial robot trajectory control method according to an embodiment of the present invention. The vision-based industrial robot trajectory control method according to an embodiment of the present invention includes: Step S1: Real-time acquisition of execution data of mechanical components performing tasks and image data corresponding to the performed tasks; Step S2: Generate several movable trajectories for the mechanical component to perform the task, and determine the trajectory complexity characterization value based on the number of obstacles in each movable trajectory and the number of rotations required by the mechanical component, so as to sort the movable trajectories and determine the initial movement trajectory. Step S3: Obtain the distribution characteristics of the target object based on the image data, calculate the execution constraint characterization parameters of the mechanical component, and mark the mechanical component. The distribution characteristics include the occluded area of ​​the target object and the shortest distance between the target object and other objects. Step S4: In response to the marking result, analyze the execution process corresponding to the mechanical component, including, The second movable trajectory of the mechanical component is changed, and the grasping state characteristics after the robotic arm grasps the target object and the duration of the linear movement of the robotic arm are combined to evaluate the connection stability characterization coefficient of the mechanical component in order to determine whether the execution process has entered the execution disorder stage. Step S5: Adjust the movement trajectory based on the duration of the increase in joint current and the number of jammed joints, including switching to a safe return trajectory or immediately stopping the current movement trajectory. The grasping state characteristics include the contraction range and the number of times the robotic arm opens and closes.

[0027] Specifically, the execution data includes the number of rotations required for the mechanical components in the movable trajectory, the grasping state characteristics, the duration of the increase in joint current, and the number of jammed joints, etc.; the image data includes the number of obstacles in the movable trajectory, the distribution characteristics of the target object, and the duration of the linear movement of the robotic arm, etc., wherein the mechanical components are a robotic hand and a robotic arm.

[0028] Specifically, firstly, several movable trajectories are generated according to the path planning algorithm. Each movable trajectory can enable the mechanical component to move from its current position to the task execution position. Then, the number of rotations required for the mechanical component to perform the task according to the movable trajectory is determined. The path planning algorithm generates a safe backtracking trajectory for the movable trajectory at the same time as generating the movable trajectory, which will not be elaborated further.

[0029] Specifically, there are no specific limitations on the acquisition method of grasping state characteristics. The sensor can be fixed on the movement trajectory of the telescopic arm or gripper of the robot (such as the slide rail for opening and closing the gripper). When the robot moves, it drives the sensor rod / slider to move synchronously to determine the contraction range of the robot. The number of times the robot arm opens and closes is determined by trigger counting based on the position signal. Limit switches (such as proximity switches or photoelectric switches) can be installed at the extreme positions of the robot arm's "fully open" and "fully closed" states. When the gripper reaches the position, the switch outputs a signal (such as a high level). When a complete opening and closing sequence of "open signal - close signal - open signal" is detected, the system automatically increments the count, which will not be elaborated further.

[0030] Specifically, there is no specific limitation on the method for acquiring the duration of the rise in joint current. The current sensor can be connected in series in the power supply circuit of the joint drive motor to directly acquire the motor's operating current. The time interval from when the joint current exceeds the current threshold until it returns to normal or stops is taken as the duration of the rise, and the joint current exceeding the current threshold is taken as a current abnormality. For setting the current threshold, those skilled in the art can record the normal current range of each joint through no-load / light-load testing, and determine the maximum current value in the normal current range as the current threshold, which will not be elaborated further.

[0031] Specifically, there is no specific limitation on the method of collecting the number of jammed joints. Motion sensors can be placed at the end of the joint rotation axis to directly measure the actual rotation angle of the joint (e.g., using a miniature potentiometer to mechanically connect to the joint axis). Then, joints that meet the jamming conditions are identified as jammed joints, and the number of jammed joints is counted. The jamming conditions include that the current in three consecutive samplings (within 30ms) is greater than the jamming current threshold, and that the deviation angle between the actual position of the joint and the predetermined position is greater than the predetermined deviation angle during the corresponding sampling period. In practice, the blocking current threshold is set to 1.5 times the current threshold, the predetermined deviation angle corresponding to the joint of the robotic arm is set to 5°, and the predetermined deviation angle corresponding to the joint of the robotic arm is set to 1°.

[0032] Specifically, there are no specific limitations on the method of acquiring image data. Images can be acquired through cameras in the workshop or other work environment where the industrial robot is performing its work, and the image data can be determined by combining the images with image analysis algorithms. This is existing technology and will not be elaborated further.

[0033] Specifically, the process of determining the trajectory complexity representation value includes: The ratio of the number of obstacles in the movable trajectory to the obstacle number threshold is used as the first trajectory complexity feature; The ratio of the number of rotations required for the mechanical component to the threshold number of rotations required is used as the second trajectory complexity feature; The sum of the first trajectory complexity features and the second trajectory complexity features is used as the trajectory complexity characterization value.

[0034] In this embodiment, the purpose of setting the obstacle quantity threshold and the required number of rotations threshold is to characterize situations where the environment of the movable trajectory significantly interferes with the task execution. Since the environmental factors corresponding to the mechanical component are dynamically changing when performing a task, meaning that the movable trajectory may not be the same for several executions of the same task, and the environmental factors (the number of obstacles and the number of rotations required by the mechanical component due to environmental distribution) will also change, historical execution data and historical image data from several executions of the same task are obtained, and the corresponding obstacle quantity in the movable trajectory is retrieved. Historical data and historical data on the number of rotations required for mechanical components are used to calculate the average number of obstacles and the average number of rotations required, which are then used as the baseline values ​​under normal conditions. Based on the purpose of setting the above two thresholds, the obstacle number threshold is determined as the product of the average number of obstacles and the number deviation coefficient, and the required number of rotations threshold is determined as the product of the average number of rotations required and the rotation deviation coefficient. The number deviation coefficient is selected within the interval [1.2, 1.3], preferably 1.2 in practice, and the rotation deviation coefficient is selected within the interval [1.15, 1.2], preferably 1.15 in practice.

[0035] Specifically, the process of sorting the movable trajectories and determining the initial movement trajectory includes: Obtain the trajectory complexity representation value corresponding to each movable trajectory; The movable trajectories are sorted in descending order based on their complexity values. Based on the descending sorting result, a sequence of movable trajectories is obtained; The movable trajectory corresponding to the first position of the movable trajectory sequence is determined as the initial movable trajectory.

[0036] Specifically, this invention determines the priority of movable trajectories based on environmental factors of several movable trajectories. Given multiple selectable trajectories, and avoiding the rigidity of a single predetermined trajectory, the number of obstacles in a movable trajectory directly reflects the trajectory risk during initial selection. Prioritizing trajectories with fewer obstacles reduces the probability of collisions between mechanical components and obstacles. Simultaneously, prioritizing trajectories requiring fewer rotational operations from mechanical components reduces frequent starts and stops of joints and attitude adjustments, thereby reducing mechanical wear and energy consumption. This invention effectively responds to dynamic environments by dynamically generating and intelligently sorting trajectories for tasks performed by mechanical components, achieving intelligent and adaptive movable trajectories while ensuring the stability and safety of task execution.

[0037] Specifically, the process of calculating the execution constraint characterization parameters of the mechanical component includes: The ratio of the occluded area of ​​the target object to the occluded area threshold is used as the first execution constraint feature. The ratio of the shortest distance threshold to the shortest distance between the target object and other objects is used as the second execution constraint feature; The sum of the first execution constraint feature and the second execution constraint feature is used as the execution constraint representation parameter.

[0038] In this embodiment, the purpose of setting the occluded area threshold and the shortest distance threshold is to characterize the severe limitation of the mechanical component in grasping the target object. By acquiring historical image data of several times performing the same task, calling the historical data of the shortest distance between the target object and other objects, the mean of the shortest distance is calculated and used as the benchmark value under normal circumstances. Based on the purpose of setting the shortest distance threshold, the shortest distance threshold is determined to be the product of the mean of the shortest distance and the distance deviation coefficient. The distance deviation coefficient is selected in the interval [0.85, 0.9], and is preferably 0.85 in practice. It is understandable that occlusion of the target object by other objects or environmental factors will affect the successful grasping of the target object by the robotic arm. Therefore, in this implementation, 30% of the target object area is determined as the occluded area threshold, which will not be elaborated further.

[0039] Specifically, this invention considers the complexity of the environment in which the target object is located. By determining the occluded area of ​​the target object, it identifies the visibility risk of the robotic arm's grasping point. Simultaneously, during the grasping process, it is crucial to ensure that the robotic arm maintains a safe distance from surrounding objects as it approaches the target object to avoid collisions due to confined space. Therefore, this invention uses visual data-driven quantitative analysis of the execution constraints of the mechanical components to calculate the execution constraint representation parameters of the mechanical components. This characterizes the degree of interference from surrounding environmental factors when the mechanical components grasp the target object, thereby quantifying the limitations on grasping the target object and providing data support for subsequent labeling of the mechanical components. This invention effectively responds to dynamic environments, achieving intelligent and adaptive movement trajectories while ensuring the stability and safety of task execution.

[0040] Specifically, please refer to Figure 2 As shown, this is a logic decision diagram for marking mechanical components according to an embodiment of the present invention. Marking the mechanical components includes: If the execution constraint characterization parameter of a mechanical component is greater than or equal to the execution constraint characterization parameter threshold, then the mechanical component is marked as an execution-disrupted component. If the execution constraint characterization parameter of a mechanical component is less than the execution constraint characterization parameter threshold, then there is no need to mark the mechanical component.

[0041] The execution constraint characterization parameter threshold is predetermined. The execution constraint characterization parameter threshold is determined by calculating the target object's occluded area as equal to the occluded area threshold and the shortest distance threshold as equal to the shortest distance between the target object and other objects.

[0042] Specifically, in response to the marking result, the execution process corresponding to the mechanical component is analyzed, including: If any mechanical component is marked as an execution-disrupted component, then the execution process corresponding to that mechanical component is analyzed.

[0043] Specifically, the process of changing the movable trajectory of the mechanical component includes: Call the movable trajectory sequence; Based on the sequence of movable trajectories, the corresponding second movable trajectories are determined sequentially.

[0044] Specifically, the process of evaluating the connection stability characterization coefficient of the mechanical components includes: The sum of the ratio of the contraction amplitude of the robotic arm to the contraction amplitude threshold and the ratio of the number of opening and closing operations to the number of opening and closing operations threshold is used as the first connection stability feature. The ratio of the duration of linear movement of the robotic arm to the threshold duration of linear movement is used as the second connection stability feature. The first connection stability feature and the second connection stability feature are weighted and summed to determine the connection stability characterization coefficient.

[0045] Specifically, in practice, the contraction range and opening / closing frequency of the robotic arm can directly reflect the coupling effect of the bearing force and the deformation of the object. As for the duration of the linear movement of the robotic arm, a certain response delay is allowed. Therefore, in implementation, the grasping state characteristics are given priority. Thus, the first connection stability characteristic calculated based on the grasping state characteristics is given a slightly higher weight. Therefore, when performing weighted summation, the weight of the first connection stability characteristic is set to 0.6, and the weight of the second connection stability characteristic is set to 0.4. In this embodiment, the purpose of setting the contraction amplitude threshold, opening and closing number threshold, and linear movement duration threshold is to characterize the poor stability of the connection and cooperation between mechanical components. By acquiring historical execution data and historical image data of several times performing the same task, and calling historical data of the robot arm's contraction amplitude, opening and closing number, and linear movement duration, the average contraction amplitude, average opening and closing number, and average linear movement duration are calculated and used as the baseline values ​​under normal conditions. Based on the purpose of setting the above three thresholds, the... The contraction amplitude threshold is determined as the product of the average contraction amplitude and the contraction deviation coefficient. The opening and closing number threshold is determined as the product of the average opening and closing number and the opening and closing deviation coefficient. The linear movement duration threshold is determined as the product of the average linear movement duration and the movement deviation coefficient. The contraction deviation coefficient is selected within the interval [1.15, 1.2], preferably 1.15 in practice. The opening and closing deviation coefficient is selected within the interval [1.2, 1.25], preferably 1.2 in practice. The movement deviation coefficient is selected within the interval [1.2, 1.3], preferably 1.2 in practice.

[0046] Specifically, this invention analyzes mechanical components marked as experiencing execution interference, prioritizing dynamic trajectory switching. While reducing the degree of obstruction to subsequent task execution, it considers the state characteristics exhibited by the mechanical components during execution. Specifically, the contraction amplitude of the robotic arm as it grasps the target object reflects the adaptability of its gripping force; for example, excessive gripping leads to vibration. Furthermore, monitoring the number of opening and closing cycles of the robotic arm identifies over-adjustment behavior, such as repeated clamping. The robotic arm experiences a certain degree of positioning drift due to motion inertia, and excessive positioning drift affects the grasping quality. Based on the combination of these characteristics, it determines whether the mechanical components are in an abnormal friction state. Therefore, this invention quantifies the stability of the corresponding robotic arm and robotic arm's coordinated operation by evaluating the connection stability characterization coefficient of the mechanical components, providing data support for subsequent determination of whether the execution process has entered a disordered stage. This invention effectively responds to dynamic environments, achieving intelligent and adaptive movement trajectories while ensuring the stability and safety of task execution.

[0047] Specifically, please refer to Figure 3 As shown, this is a logic diagram for determining whether the execution process has entered the execution disorder stage according to an embodiment of the present invention. Determining whether the execution process has entered the execution disorder stage includes: If the stability coefficient of the connection is greater than or equal to the threshold of the stability coefficient of the connection, the execution process is determined to have entered the execution disorder stage. If the stability coefficient of the connection is less than the threshold of the stability coefficient of the connection, it is determined that the execution process has not entered the stage of execution disorder.

[0048] The threshold value of the connection stability characterization coefficient is predetermined. The connection stability characterization coefficient calculated is determined when the contraction amplitude of the robotic arm is equal to the contraction amplitude threshold value, the number of opening and closing operations is equal to the number of opening and closing operations threshold value, and the duration of linear movement of the robotic arm is equal to the duration of linear movement threshold value.

[0049] Specifically, please refer to Figure 4 As shown, this is a logic decision diagram for adjusting the movement trajectory according to an embodiment of the present invention. The process of adjusting the movement trajectory includes: If the duration of the increase in joint current exceeds the threshold for duration of increase or / and the number of jammed joints in the mechanical components exceeds the threshold for the number of jammed joints, then the current movement trajectory shall be stopped immediately. Otherwise, switch to a safe rollback path.

[0050] In this embodiment, the purpose of setting the threshold for the duration of the rise and the threshold for the number of jammed joints is to characterize the situation where the abnormal risk of the entire mechanical system is relatively serious during the task execution. By obtaining historical execution data of several times completing the same task, calling historical data of the duration of the rise of joint current and historical data of the number of jammed joints of mechanical components, the average duration of the rise and the average number of jammed joints are calculated and used as the benchmark values ​​under normal conditions. Based on the purpose of setting the above two thresholds, the duration of the rise is determined as the product of the average duration of the rise and the duration deviation coefficient, and the threshold for the number of jammed joints is determined as the product of the average number of jammed joints and the jamming deviation coefficient. The duration deviation coefficient is selected in the interval [1.15, 1.2], preferably 1.15 in the implementation, and the jamming deviation coefficient is selected in the interval [1.1, 1.2], preferably 1.1 in the implementation.

[0051] Specifically, joint current refers to the current consumed by the motor that drives the joints of a robotic arm or hand during operation. It can directly reflect the load status and health of the motor. In the practical application of industrial robots, monitoring joint current is a key means of diagnosing mechanical jamming or electrical faults, which will not be elaborated here.

[0052] Specifically, after entering the execution disorder stage, this invention further analyzes the abnormal state of the entire mechanical system and takes corresponding measures. It monitors the duration of joint current elevation, accurately distinguishes between instantaneous load fluctuations (normal gripping resistance) and continuous abnormalities (mechanical jamming), and reflects the severity of continuous abnormalities. At the same time, it counts the number of jammed joints of the mechanical components, quantifies the fault risk, and then determines whether to urgently stop the movement of the mechanical components through dual criteria to avoid accidental emergency stop (brief current fluctuation). In addition, combined with flexible adjustment measures, it switches to a safe return trajectory only when a single condition is met, which can preserve the task state of the mechanical components and quickly resume task execution after the fault is eliminated. Based on this, this invention achieves intelligent and adaptive movement trajectory through real-time monitoring and dynamic response of joint current and jamming status, while ensuring the stability and safety of task execution.

[0053] If the vision-based industrial robot trajectory control method of the present invention is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0054] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A trajectory control method for industrial robots based on visual images, characterized in that, include: Real-time acquisition of execution data of mechanical components performing tasks and image data corresponding to the performed tasks; Several movable trajectories are generated for the mechanical component to perform the task. The complexity characterization value of the trajectory is determined based on the number of obstacles in each movable trajectory and the number of rotations required by the mechanical component. The movable trajectories are then sorted to determine the initial movement trajectory. Based on the image data, the distribution characteristics of the target object are obtained, and the execution constraint characterization parameters of the mechanical component are calculated to mark the mechanical component. The distribution characteristics include the occluded area of ​​the target object and the shortest distance between the target object and other objects. In response to the marking results, the execution process corresponding to the mechanical component is analyzed. include, The second movable trajectory of the mechanical component is changed, and the grasping state characteristics after the robotic arm grasps the target object and the duration of the linear movement of the robotic arm are combined to evaluate the connection stability characterization coefficient of the mechanical component in order to determine whether the execution process has entered the execution disorder stage. Based on the duration of the increase in joint current and the number of jammed joints, the movement trajectory is adjusted, including switching to a safe return trajectory or immediately stopping the current movement trajectory. The grasping state characteristics include the contraction range and the number of times the robotic arm opens and closes.

2. The industrial robot trajectory control method based on vision images according to claim 1, characterized in that, The process of determining the trajectory complexity characterization value includes: The ratio of the number of obstacles in the movable trajectory to the obstacle number threshold is used as the first trajectory complexity feature; The ratio of the number of rotations required for the mechanical component to the threshold number of rotations required is used as the second trajectory complexity feature; The sum of the first trajectory complexity features and the second trajectory complexity features is used as the trajectory complexity characterization value.

3. The industrial robot trajectory control method based on vision images according to claim 2, characterized in that, The process of sorting the movable trajectories and determining the initial movement trajectory includes: Obtain the trajectory complexity representation value corresponding to each movable trajectory; The movable trajectories are sorted in descending order based on their complexity values. Based on the descending sorting result, a sequence of movable trajectories is obtained; The movable trajectory corresponding to the first position of the movable trajectory sequence is determined as the initial movable trajectory.

4. The industrial robot trajectory control method based on vision images according to claim 1, characterized in that, The process of calculating the execution constraint characterization parameters of the mechanical component includes: The ratio of the occluded area of ​​the target object to the occluded area threshold is used as the first execution constraint feature. The ratio of the shortest distance threshold to the shortest distance between the target object and other objects is used as the second execution constraint feature; The sum of the first execution constraint feature and the second execution constraint feature is used as the execution constraint representation parameter.

5. The industrial robot trajectory control method based on vision images according to claim 4, characterized in that, Marking the mechanical components includes: If the execution constraint characterization parameter of a mechanical component is greater than or equal to the execution constraint characterization parameter threshold, then the mechanical component is marked as an execution-disrupted component.

6. The industrial robot trajectory control method based on vision images according to claim 5, characterized in that, In response to the marking results, the execution process corresponding to the mechanical component is analyzed, including: If any mechanical component is marked as an execution-disrupted component, then the execution process corresponding to that mechanical component is analyzed.

7. The industrial robot trajectory control method based on vision images according to claim 3, characterized in that, The process of changing the movable trajectory of the mechanical component includes: Call the movable trajectory sequence; Based on the sequence of movable trajectories, the corresponding second movable trajectories are determined sequentially.

8. The industrial robot trajectory control method based on vision images according to claim 1, characterized in that, The process of evaluating the connection stability characterization coefficient of the mechanical components includes: The sum of the ratio of the contraction amplitude of the robotic arm to the contraction amplitude threshold and the ratio of the number of opening and closing operations to the number of opening and closing operations threshold is used as the first connection stability feature. The ratio of the duration of linear movement of the robotic arm to the threshold duration of linear movement is used as the second connection stability feature. The first connection stability feature and the second connection stability feature are weighted and summed to determine the connection stability characterization coefficient.

9. The industrial robot trajectory control method based on vision images according to claim 8, characterized in that, Determining whether the execution process has entered a stage of execution disorder includes: If the stability coefficient of the connection is greater than or equal to the threshold of the stability coefficient of the connection, the execution process is determined to have entered the execution disorder stage.

10. The industrial robot trajectory control method based on vision images according to claim 1, characterized in that, The process of adjusting the movement trajectory includes: If the duration of the increase in joint current exceeds the threshold for duration of increase or / and the number of jammed joints in the mechanical components exceeds the threshold for the number of jammed joints, then the current movement trajectory shall be stopped immediately. Otherwise, switch to a safe rollback path.

Images