Robotic arm control method and system based on visual intelligent positioning

Abstract

The present application discloses a method and system for controlling a robotic arm based on visual intelligent positioning, which relates to the field of robotic arm control technology. The method for controlling a robotic arm based on visual intelligent positioning includes: obtaining a first path based on a first visual sensor according to the position of the robotic arm and a preset target; obtaining a second path based on a second visual sensor according to the position of the robotic arm and a preset target; comparing the first path with the second path, and if the deviation between the first path and the second path exceeds the preset deviation, adjusting the first path based on the second path to obtain a third path, or adjusting the second path based on the first path to obtain a fourth path; and controlling the robotic arm based on the third path or the fourth path. This technical solution generates two independent paths in the robotic arm motion path planning stage through the collaborative work of visual sensors, and compares the path deviations in real time, thereby improving the positioning robustness in complex environments.

Description

Technical Field

[0001] The present application relates to the field of robotic arm control technology, and in particular to a robotic arm control method and system based on visual intelligent positioning. Background Art

[0002] With the rapid development of industrial automation and intelligent manufacturing, robotic arms, with their high precision and repeatability, are widely used in assembly, welding, and handling scenarios. However, in complex and dynamic environments, the positioning accuracy of robotic arms is easily affected by factors such as ambient lighting changes, object occlusion, and sensor noise. This leads to the lack of stability in traditional single-vision positioning methods.

[0003] In existing technologies, visual positioning of robotic arms mainly relies on a single sensor (such as a monocular camera, lidar, or structured light camera) to collect environmental information and plan the motion path through feature matching or deep learning algorithms. However, such methods are significantly limited in the following scenarios:

[0004] Dynamic lighting interference: When the ambient lighting changes suddenly (for example, when doors and windows in a workshop open and close, causing changes in natural light intensity), the color feature extraction stability of the RGB camera decreases, and the path planning error increases.

[0005] Mirror reflection interference: For highly reflective metal workpieces (such as automotive parts), the monocular vision system is prone to generating false feature points due to mirror reflection, resulting in incorrect target object pose calculation;

[0006] Partial occlusion problem: In densely stacked material grasping scenarios, when the target object is partially obscured, the single-view sensor cannot reconstruct complete three-dimensional information, and the path planning failure rate increases.

[0007] In addition, some existing technologies propose to use binocular vision systems to calculate the depth information of the target object, but the dual camera data is only used for initial positioning. When a sensor fails due to environmental interference, the system's fault tolerance is significantly reduced. Summary of the Invention

[0008] The technical solution of this application primarily provides a robotic arm control method and system based on visual intelligent positioning. Through the collaborative work of a first visual sensor and a second visual sensor, two independent paths are generated during the robotic arm's motion path planning phase, and path deviations are compared in real time. When a single sensor experiences data anomalies due to environmental interference, the system triggers a dynamic adjustment mechanism based on a path deviation threshold, effectively avoiding positioning errors caused by localized failures of traditional single sensors and improving positioning robustness in complex environments.

[0009] To achieve the above objectives, this application provides the following technical solutions:

[0010] One aspect of the present application provides a method for controlling a robotic arm based on visual intelligent positioning, comprising: obtaining a first path based on a first visual sensor according to a position between the robotic arm and a preset target;

[0011] According to the position of the robot arm and the preset target, a second path is obtained based on the second visual sensor;

[0012] comparing the first path and the second path; if a deviation between the first path and the second path exceeds a preset deviation, adjusting the first path based on the second path to obtain a third path, or adjusting the second path based on the first path to obtain a fourth path; and controlling the robot arm based on the third path or the fourth path;

[0013] If the deviation between the first path and the second path does not exceed a preset deviation, the robotic arm is controlled based on the first path or the second path.

[0014] Preferably, comparing the first path with the second path includes:

[0015] discretizing the first path and the second path into a time series or a spatial coordinate series;

[0016] A minimum alignment distance between the discretized first path and the second path is calculated, and if the minimum alignment distance exceeds a deviation threshold, it is determined that the deviation between the first path and the second path exceeds a preset deviation.

[0017] Preferably, the preset deviation includes:

[0018] The first deviation threshold δ1 is set based on the repeatability of the robotic arm;

[0019] The second deviation threshold δ2 is a threshold adjusted based on the target movement speed or the density of environmental obstacles, where δ1 and δ2 satisfy:

[0020] δ2=δ1*(1+k*v);

[0021] Among them, v is the target movement speed, k is the speed influence coefficient;

[0022] When the deviation between the first path and the second path exceeds a larger value of a first deviation threshold and a second deviation threshold, it is determined that the deviation between the first path and the second path exceeds a preset deviation.

[0023] Preferably, comparing the first path and the second path includes:

[0024] Dividing the first path into a plurality of first sub-paths based on the time series;

[0025] Dividing the second path into a plurality of second sub-paths based on the time series, wherein the number of the second sub-paths is equal to the number of the first sub-paths;

[0026] Comparing the first subpath and the second subpath within the corresponding time zone, adjusting the subpath segments that exceed the preset deviation to obtain a third subpath;

[0027] splicing the third sub-path with the unadjusted portion of the first path to obtain a third path; or

[0028] The third sub-path and the unadjusted portion of the second path are spliced together to obtain a fourth path.

[0029] Preferably, the first visual sensor is a global fixed camera, which is used to obtain global three-dimensional information of the robot arm workspace;

[0030] The second camera is a camera mounted on the end of the robotic arm and is used to capture the local posture of the target in real time.

[0031] The method according to claim 4, characterized in that the first path weight α(d) and the second path weight 1-α(d) are assigned based on the real-time distance d between the end of the robotic arm and the preset target;

[0032] At the i-th moment in the time region where the first subpath and the second subpath exceed the preset deviation, the third subpath can be expressed as:

[0033]

[0034] Preferably, the weight distribution function α(d) is expressed as:

[0035] α(d)=α1+(α2-α1)·e -λd ;

[0036] Among them, α1∈[0.2, 0.4] is the lower limit of the weight of the first visual sensor;

[0037] α2∈[0.6, 0.8], is the upper limit of the weight of the second visual sensor;

[0038] λ∈[0.3, 0.7], which is used to control the rate at which the weight decays with distance.

[0039] Preferably, the weight distribution function α(d) is expressed as:

[0040]

[0041] Among them, α1∈[0.2, 0.4] is the lower limit of the weight of the first visual sensor;

[0042] α2∈[0.6, 0.8], is the upper limit of the weight of the second visual sensor;

[0043] λ∈[0.3, 0.7], used to control the rate at which the weight decays with distance;

[0044] d0 is the preset distance threshold.

[0045] The second aspect of the present application provides a control system for a robotic arm based on visual intelligent positioning, including:

[0046] a first visual sensor configured to obtain a first path based on the robotic arm and a preset target position;

[0047] a second visual sensor configured to acquire a second path based on the robotic arm and a preset target position;

[0048] a storage module storing a preset deviation between the first path and the second path;

[0049] a determination module configured to determine whether a deviation between the first path and the second path exceeds a preset deviation;

[0050] an adjustment module, configured to adjust the first path based on the second path to obtain a third path, or to adjust the second path based on the first path to obtain a fourth path, when a deviation between the first path and the second path exceeds a preset deviation;

[0051] The path execution module is configured to control the robotic arm to execute the first path or the second path when the deviation between the first path and the second path does not exceed the preset deviation; or to control the robotic arm to execute the third path or the fourth path when the deviation between the first path and the second path exceeds the preset deviation.

[0052] A third aspect of the present application provides an electronic device, comprising:

[0053] at least one processor; and

[0054] a memory communicatively coupled to the at least one processor, wherein:

[0055] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

[0056] Compared to known public technologies, the technical solution provided by this application offers the following advantages: Through the collaborative work of the first and second visual sensors, two independent paths are generated during the robot arm's motion path planning phase, and path deviations are compared in real time. When a single sensor experiences data anomalies due to environmental interference, the system triggers a dynamic adjustment mechanism based on the path deviation threshold, effectively avoiding positioning errors caused by localized failures of traditional single sensors and improving positioning robustness in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] To more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be derived from these drawings without inventive effort.

[0058] Figure 1 This is a workflow diagram of the robotic arm control method based on visual intelligent positioning disclosed in an embodiment of the present application.

[0059] Figure 2 This is a schematic diagram of the structural principle of the robotic arm control system based on visual intelligent positioning disclosed in an embodiment of the present application.

[0060] Figure 3 This is a schematic diagram of the structure of an electronic device for executing a robotic arm control method based on visual intelligent positioning disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0061] To make the purpose, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0062] One embodiment of the present application provides a robotic arm control method based on visual intelligent positioning. In a specific example, the hardware configuration is as follows:

[0063] Robotic arm: A six-degree-of-freedom collaborative robot (such as the UR5e) with a gripper and force sensor at the end;

[0064] First visual sensor: a global fixed RGB-D camera (such as Azure Kinect DK) (global camera), installed on the top of the work area, covering a range of 3m × 3m, used to obtain a global 3D point cloud;

[0065] Second visual sensor: an end-embedded binocular camera (such as ZED 2i) (end camera), with a resolution of 3840×1080 and a frame rate of 60fps, capturing the local posture of the target in real time;

[0066] Control unit: Embedded industrial computer (NVIDIA Jetson AGX Orin), running the path planning algorithm and judgment module, adjustment module, and path execution module.

[0067] Path generation and adjustment are performed in the following steps:

[0068] S100: Obtaining a first path based on a first visual sensor according to the position of the robotic arm and a preset target;

[0069] S200: Obtaining a second path based on a second visual sensor according to the position of the robot arm and a preset target;

[0070] S300: Compare the first path and the second path:

[0071] If the deviation between the first path and the second path exceeds a preset deviation, adjusting the first path based on the second path to obtain a third path, or adjusting the second path based on the first path to obtain a fourth path; and controlling the robot arm based on the third path or the fourth path;

[0072] If the deviation between the first path and the second path does not exceed a preset deviation, the robotic arm is controlled based on the first path or the second path.

[0073] In the above technical solution, the first path is generated by calculating the angle sequence of each joint of the manipulator by inverse kinematics based on the three-dimensional coordinates of the target object obtained by the global camera, and generating a time-space path by discretizing the time series;

[0074] The second path is generated by collecting local images of the target object in real time based on the end camera, identifying key feature points using the YOLO target detection model, converting them to the base coordinate system through hand-eye calibration, and generating a path represented by a sequence of spatial coordinates.

[0075] In addition, when the first path and the second path are compared in step 300 and the deviation between the first path and the second path exceeds the preset deviation, the following steps are mainly included:

[0076] Step 310: Divide the first path into a plurality of first sub-paths based on the time sequence;

[0077] Step 320: Divide the second path into a plurality of second sub-paths based on the time sequence, wherein the number of the second sub-paths is equal to the number of the first sub-paths;

[0078] Step S330: Compare the first subpath and the second subpath in the corresponding time zone, and adjust the subpath segments that exceed the preset deviation to obtain a third subpath;

[0079] Step S340: splicing the third sub-path with the unadjusted portion of the first path to obtain a third path; or splicing the third sub-path with the unadjusted portion of the second path to obtain a fourth path.

[0080] In the above technical solution, when the first path and the second path are compared in step S300 and the deviation between the first path and the second path does not exceed the preset deviation, the robotic arm is controlled based on the first path or the second path; in the actual working process, the robotic arm can first be controlled to quickly position the first path obtained by the global camera. As the robotic arm continues to approach the preset target, the second path obtained by the end camera can be used as the main method to control the robotic arm for precise positioning.

[0081] In the above technical solution, in step S310 and step S320, the first path / second path is divided into multiple sub-paths and the sub-path segments exceeding the preset deviation are adjusted to obtain the third sub-path mainly through the following steps:

[0082] 1) Discretize the first path and the second path into a spatial coordinate sequence corresponding to the timestamp at fixed time intervals (e.g., every 0.1 seconds);

[0083] 2) Align the time series of the two paths and calculate the minimum alignment distance D min If D min If the deviation exceeds the preset deviation D0 (e.g. 0.6 mm), the sub-section is marked as an area to be adjusted;

[0084] 3) In the area to be adjusted, using the current robot arm position as the starting point and the target position as the focus, a real-time obstacle map is constructed in combination with the end camera data to generate a collision-free optimized subpath;

[0085] 4) The optimized subpath is directly connected to the unadjusted portion of the original first or second path according to the timestamp. During the splicing process, several points are taken before and after the splicing point to generate a smooth transition curve to ensure acceleration continuity. The splicing point is used as the control point to generate the transition path segment to avoid sudden stops or jitters of the robot arm.

[0086] In the above 4), if the first visual sensor detects a change in the environmental structure (such as displacement of a support), the optimized sub-paths are spliced based on the first path to obtain a third path; if the second visual sensor detects a dynamic obstacle or a change in the target posture within the sub-path segment, the optimized sub-paths are spliced based on the second path to obtain a fourth path; if the above conditions do not exist, the weights of the first path and the second path are allocated based on the real-time distance d between the end of the robotic arm and the preset target, and the first path and the second path are spliced based on the weights of the first path and the second path to obtain a third path or a fourth path.

[0087] In a specific example, when the real-time distance between the end of the robotic arm and the preset target is d, at the time i when the first subpath and the second subpath exceed the preset deviation, the third subpath can be expressed as:

[0088]

[0089] Among them, α(d) is the first path weight, 1-α(d) is the second path weight;

[0090] is the first subpath at the i-th moment; is the second subpath at the i-th moment; is the third subpath at the i-th moment.

[0091] In the above example, α(d) can be a function that decreases as d decreases, indicating that as the distance between the end of the robot arm and the preset target decreases, the weight of the first path gradually decreases, while the weight of the second path gradually increases; that is, as the distance between the end of the robot arm and the preset target gets closer, the local image of the preset target obtained by the second visual sensor is more dominant, which can effectively improve the accuracy of the robot arm positioning based on the preset target.

[0092] After obtaining the third sub-path through the above method, the third sub-path is then spliced with the unadjusted portion of the first path / second path to form a third path / fourth path, and the robotic arm is subsequently controlled based on the third path / fourth path.

[0093] In a specific example, the weight distribution function α(d) is expressed as:

[0094] α(d)=α1+(α2-α1)·e -λd ;

[0095] Among them, α1∈[0.2, 0.4] is the lower limit of the weight of the first visual sensor;

[0096] α2∈[0.6, 0.8], is the upper limit of the weight of the second visual sensor;

[0097] λ∈[0.3, 0.7], which is used to control the rate at which the weight decays with distance.

[0098] In the above example, as the distance d between the end of the robotic arm and the preset target continues to decrease, the weight distribution function α(d) continues to decrease, which means that the proportion of global three-dimensional point cloud data captured by the first visual sensor continues to decrease, while the proportion of the target's local posture captured in real time by the second visual sensor that continues to approach the preset target continues to increase. This is conducive to continuously adjusting the proportion of the first path and the second path in the fusion path according to the real-time movement of the robotic arm, thereby improving the accuracy of the robotic arm's positioning.

[0099] In another example of the present application, the weight distribution function α(d) is expressed as:

[0100]

[0101] Among them, α1∈[0.2, 0.4] is the lower limit of the weight of the first visual sensor;

[0102] α2∈[0.6, 0.8], is the upper limit of the weight of the second visual sensor;

[0103] λ∈[0.3, 0.7], used to control the rate at which the weight decays with distance;

[0104] d0 is the preset distance threshold.

[0105] In the above example, the positioning of the robot arm is divided into two stages, the first stage is the coarse positioning stage, and the second stage is the fine positioning stage.

[0106] Among them, the stage when the distance d between the end of the robot arm and the preset target is greater than the preset distance threshold d0 (such as 500mm) is defined as the first stage. At this time, the distance between the end of the robot arm and the preset target is relatively far, and the weight function α(d) is defined as a fixed value α1, which reduces the calculation of the robot arm motion data, so that the robot arm can quickly locate the preset position, that is, the position where the distance d between the end of the robot arm and the preset target is d=d0; Subsequently, the stage when the distance d between the end of the robot arm and the preset target is less than the preset distance d0 is defined as the second stage. At this time, the distance between the end of the robot arm and the preset target is relatively close, and it is necessary to adjust the proportion of the first sub-path and the second sub-path in the fusion sub-path in real time based on the distance between the end of the robot arm and the preset target to improve the positioning accuracy; that is, in the second stage, the weight distribution function α(d) decreases as the distance between the end of the robot arm and the preset target d continues to decrease, as described above, that is, as the distance between the end of the robot arm and the preset target gets closer, the proportion of data captured by the second visual sensor set at the end of the robot arm becomes larger, which is conducive to improving the positioning accuracy of the robot arm.

[0107] In this example, the positioning of the robotic arm is divided into the first stage and the second stage. The first stage is mainly for rapid positioning, and the second stage is mainly for precise positioning. The two-stage positioning can take into account both positioning accuracy and positioning efficiency, effectively improving the effectiveness of the robotic arm positioning control.

[0108] In the above technical solution, the preset deviation may be a fixed value, which is used to indicate that when the deviation between the first sub-path and the second sub-path at time i exceeds the fixed threshold, the area is determined to be a sub-path to be adjusted.

[0109] In a better technical solution, the preset deviation includes:

[0110] The first deviation threshold δ1 is set based on the repeatability of the robotic arm;

[0111] The second deviation threshold δ2 is a threshold adjusted based on the target motion speed, where δ1 and δ2 satisfy:

[0112] δ2=δ1*(1+k*v);

[0113] Where, v is the target motion speed;

[0114] k is a speed influence coefficient. In a specific example, k may be a constant, and the value of k is a positive value.

[0115] When the deviation between the first path and the second path exceeds a larger value of a first deviation threshold and a second deviation threshold, it is determined that the deviation between the first path and the second path exceeds a preset deviation.

[0116] In the above technical solution, the first deviation threshold is a static threshold, which is set based on the repeatability of the robot arm. For example, when the repeatability of the robot arm is ±01.6 mm, the first deviation threshold is set to δ1=0.15 mm. The static threshold reflects the inherent error limit of the hardware performance of the robot arm; that is, this static threshold is used to ensure that the correction of the path deviation does not exceed the physical capability of the robot arm, and to avoid execution failure due to excessive adjustment (such as overshoot oscillation); the second deviation threshold is a dynamic threshold, which is adjusted in real time based on the target movement speed, that is, when the target moves at high speed, the second threshold automatically increases, allowing a greater path deviation tolerance, and giving priority to ensuring the tracking continuity of the robot arm.

[0117] During the specific execution process, the larger value of the two is taken as the final deviation threshold to ensure that the system automatically selects the optimal strategy between static high-precision requirements and dynamic environmental adaptability.

[0118] In a specific scenario, when the target is stationary, that is, v = 0, the second deviation threshold is equal to the first deviation threshold, and the system executes according to the fixed deviation threshold to avoid unnecessary path fusion; in a high-speed motion scenario, it is assumed that the first deviation threshold (δ1) is fixed at 0.1mm; when the target moves quickly (such as a workpiece on a conveyor belt), the second deviation threshold (δ2) increases with the speed. For example, when v = 1m / s, δ2 = 0.3mm, while the traditional single threshold (such as a fixed 0.1mm) will frequently trigger path corrections, causing the robot arm to shake. This solution allows for greater deviation tolerance, and the subsequent path is compensated by the prediction algorithm, and the overall tracking error is reduced (experimental data shows that the overall tracking error is reduced by 40%).

[0119] Another example of the present application discloses a control system for a robotic arm based on visual intelligent positioning, including: a first visual sensor 110, configured to obtain a first path based on the robotic arm and a preset target position; a second visual sensor 120, configured to obtain a second path based on the robotic arm and the preset target position; a storage module 130, storing a preset deviation between the first path and the second path; a judgment module 140, configured to judge whether the deviation between the first path and the second path exceeds the preset deviation; an adjustment module 150, when the deviation between the first path and the second path exceeds the preset deviation, adjusting the first path based on the second path to obtain a third path, or adjusting the second path based on the first path to obtain a fourth path; a path execution module 160, configured to control the robotic arm to execute the first path or the second path when the deviation between the first path and the second path does not exceed the preset deviation; or to control the robotic arm to execute the third path or the fourth path when the deviation between the first path and the second path exceeds the preset deviation.

[0120] The present application also provides an electronic device 20, please refer to Figure 3 , Figure 3 This is a schematic diagram of an electronic device 20 provided in an embodiment of the present application. In this embodiment, the electronic device 20 includes: a processor 21, a memory 22, and a bus 23. The memory 22 stores machine-readable instructions executable by the processor 21. When the electronic device 10 is running, the processor 21 and the memory 22 communicate via the bus 23. When the machine-readable instructions are executed by the processor 21, the control method of the robotic arm based on visual intelligent positioning provided in the embodiment of the present application is executed.

[0121] Optionally, an embodiment of the present application further provides a storage medium on which a computer program is stored. When the computer program is run by the processor 21, the control method of the robotic arm based on visual intelligent positioning provided in the embodiment of the present application is executed.

[0122] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0123] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0124] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A robotic arm control method based on visual intelligent positioning, characterized in that: include: Obtaining a first path based on a first visual sensor according to the position of the robotic arm and a preset target; According to the position of the robot arm and the preset target, a second path is obtained based on the second visual sensor; comparing the first path and the second path; if a deviation between the first path and the second path exceeds a preset deviation, adjusting the first path based on the second path to obtain a third path, or adjusting the second path based on the first path to obtain a fourth path; and controlling the robot arm based on the third path or the fourth path; If the deviation between the first path and the second path does not exceed a preset deviation, the robot arm is controlled based on the first path or the second path; The preset deviation includes: First deviation threshold , set based on the repeatability of the robot arm; Second deviation threshold , a threshold value adjusted based on the target motion speed or the density of environmental obstacles, where and satisfy: ； in, v is Target movement speed, k is the speed influence coefficient; When the deviation between the first path and the second path exceeds a larger value of a first deviation threshold and a second deviation threshold, it is determined that the deviation between the first path and the second path exceeds a preset deviation.

2. The method according to claim 1, characterized in that Comparing the first path with the second path includes: discretizing the first path and the second path into a time series or a spatial coordinate series; A minimum alignment distance between the discretized first path and the second path is calculated, and if the minimum alignment distance exceeds a deviation threshold, it is determined that the deviation between the first path and the second path exceeds a preset deviation.

3. The method according to claim 1, characterized in that Comparing the first path with the second path includes: Dividing the first path into a plurality of first sub-paths based on the time series; Dividing the second path into a plurality of second sub-paths based on the time series, wherein the number of the second sub-paths is equal to the number of the first sub-paths; Comparing the first subpath and the second subpath within the corresponding time zone, adjusting the subpath segments that exceed the preset deviation to obtain a third subpath; splicing the third sub-path with the unadjusted portion of the first path to obtain a third path; or The third sub-path and the unadjusted portion of the second path are spliced together to obtain a fourth path.

4. The method according to claim 1, wherein The first visual sensor is a global fixed camera, which is used to obtain global three-dimensional information of the robot arm workspace; The second visual sensor is a camera mounted on the end of the robotic arm, which is used to capture the local posture of the target in real time.

5. The method according to claim 3, characterized in that Based on the real-time distance between the end of the robotic arm and the preset target d Assign first path weight and the second path weight ; Then, in the time zone where the first subpath and the second subpath exceed the preset deviation, i At this moment, the third subpath is expressed as: ； in, For the i The first subpath at moment; For the i The second subpath of the moment; For the i The third subpath at moment .

6. The method according to claim 5, characterized in that The weight distribution function is expressed as: ； in, , is the weight range of the first visual sensor; , is the weight range of the second visual sensor; , which controls the rate at which weights decay with distance.

7. The method according to claim 5, characterized in that Weight distribution function Expressed as: ； in, , is the weight range of the first visual sensor; , is the weight range of the second visual sensor; , used to control the rate at which weight decays with distance; d 0 is the preset distance threshold.

8. A control system for a robotic arm based on visual intelligent positioning, characterized in that: include: a first visual sensor configured to obtain a first path based on the robotic arm and a preset target position; a second visual sensor configured to acquire a second path based on the robotic arm and a preset target position; a storage module storing a preset deviation between the first path and the second path; a determination module configured to determine whether a deviation between the first path and the second path exceeds a preset deviation; an adjustment module, configured to adjust the first path based on the second path to obtain a third path, or to adjust the second path based on the first path to obtain a fourth path, when a deviation between the first path and the second path exceeds a preset deviation; a path execution module, configured to control the robot arm to execute the first path or the second path when the deviation between the first path and the second path does not exceed a preset deviation; or when the deviation between the first path and the second path exceeds a preset deviation, controlling the robotic arm to execute the third path or the fourth path; The preset deviation includes: First deviation threshold , set based on the repeatability of the robot arm; Second deviation threshold , a threshold value adjusted based on the target motion speed or the density of environmental obstacles, where and satisfy: ； in, v is Target movement speed, k is the speed influence coefficient; When the deviation between the first path and the second path exceeds a larger value of a first deviation threshold and a second deviation threshold, it is determined that the deviation between the first path and the second path exceeds a preset deviation.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively coupled to the at least one processor, wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

Images