Mechanical arm control method and system based on visual intelligent positioning

Abstract

The invention discloses a mechanical arm control method and system based on visual intelligent positioning, and relates to the technical field of mechanical arm control. The mechanical arm control method based on visual intelligent positioning comprises the steps that a first path is obtained based on a first visual sensor according to a mechanical arm and a preset target position; according to the mechanical arm and the preset target position, a second path is obtained based on a second visual sensor; comparing the first path with the second path, if the deviation between the first path and the second path exceeds a preset deviation, adjusting the first path based on the second path to obtain a third path, or adjusting the second path based on the first path to obtain a fourth path; and the mechanical arm is controlled based on the third path or the fourth path. According to the technical scheme, through cooperative work of the visual sensor, the two independent paths are generated in the motion path planning stage of the mechanical arm, the path deviation is compared in real time, and the positioning robustness in the complex environment is improved.

Description

Technical Field

[0001] The present application relates to the field of robotic arm control technology, and in particular to a robotic arm control method and system based on visual intelligent positioning. Background Art

[0002] With the rapid development of industrial automation and intelligent manufacturing, robotic arms are widely used in assembly, welding, handling and other scenarios due to their high precision and high repeatability. However, in complex dynamic environments, the end positioning accuracy of robotic arms is easily affected by factors such as ambient light changes, target occlusion, and sensor noise, resulting in the problem of insufficient stability in traditional single visual positioning methods.

[0003] In existing technologies, the visual positioning of robotic arms mainly relies on a single sensor (such as a monocular camera, lidar, or structured light camera) to collect environmental information and plan the motion path through feature matching or deep learning algorithms. However, such methods are significantly limited in the following scenarios:

[0004] Dynamic lighting interference: When the ambient lighting changes suddenly (for example, the opening and closing of doors and windows in a workshop causes changes in the intensity of natural light), the stability of color feature extraction by the RGB camera decreases and the path planning error increases.

[0005] Mirror reflection interference: For highly reflective metal workpieces (such as automotive parts), the monocular vision system is prone to generate pseudo feature points due to mirror reflection, resulting in errors in the target object's pose calculation;

[0006] Partial occlusion problem: In the scenario of grabbing densely stacked materials, when the target object is partially obscured, the single-view sensor cannot reconstruct complete three-dimensional information, and the path planning failure rate increases.

[0007] In addition, some existing technologies propose to use a binocular vision system to calculate the depth information of the target object, but its dual camera data is only used for initial positioning. When a sensor fails due to environmental interference, the system's fault tolerance is significantly reduced. Summary of the invention

[0008] The technical solution of this application mainly provides a robot arm control method and system based on visual intelligent positioning. Through the collaborative work of the first visual sensor and the second visual sensor, two independent paths are generated in the robot arm motion path planning stage, and the path deviation is compared in real time. When a single sensor causes data anomalies due to environmental interference, the system can trigger a dynamic adjustment mechanism based on the path deviation threshold, effectively avoiding the positioning errors caused by local failure of traditional single sensors, and improving the positioning robustness in complex environments.

[0009] To achieve the above objectives, this application provides the following technical solutions:

[0010] One aspect of the present application provides a method for controlling a robotic arm based on visual intelligent positioning, comprising: obtaining a first path based on a first visual sensor according to a position between the robotic arm and a preset target;

[0011] According to the position of the robot arm and the preset target, a second path is obtained based on the second visual sensor;

[0012] Comparing the first path with the second path, if the deviation between the first path and the second path exceeds a preset deviation, adjusting the first path based on the second path to obtain a third path, or adjusting the second path based on the first path to obtain a fourth path; and controlling the robot arm based on the third path or the fourth path;

[0013] If the deviation between the first path and the second path does not exceed a preset deviation, the robot arm is controlled based on the first path or the second path.

[0014] Preferably, comparing the first path with the second path comprises:

[0015] discretizing the first path and the second path into a time series or a spatial coordinate series;

[0016] The minimum alignment distance between the discretized first path and the second path is calculated, and if the minimum alignment distance exceeds a deviation threshold, it is determined that the deviation between the first path and the second path exceeds a preset deviation.

[0017] Preferably, the preset deviation includes:

[0018] The first deviation threshold δ1 is set based on the repeatability of the robot arm;

[0019] The second deviation threshold δ2 is a threshold adjusted based on the target movement speed or the density of environmental obstacles, where δ1 and δ2 satisfy:

[0020] δ2=δ1*(1+k*v);

[0021] Among them, v is the target movement speed, k is the speed influence coefficient;

[0022] When the deviation between the first path and the second path exceeds a larger value of a first deviation threshold and a second deviation threshold, it is determined that the deviation between the first path and the second path exceeds a preset deviation.

[0023] Preferably, comparing the first path and the second path comprises:

[0024] Dividing the first path into a plurality of first sub-paths based on the time series;

[0025] Dividing the second path into a plurality of second sub-paths based on the time series, wherein the number of the second sub-paths is equal to the number of the first sub-paths;

[0026] Compare the first subpath and the second subpath in the corresponding time zone, adjust the subpath segment exceeding the preset deviation, and obtain the third subpath;

[0027] splicing the third sub-path with the unadjusted portion of the first path to obtain a third path; or

[0028] The third sub-path and the unadjusted portion of the second path are concatenated to obtain a fourth path.

[0029] Preferably, the first visual sensor is a global fixed camera, which is used to obtain global three-dimensional information of the robot arm workspace;

[0030] The second camera is a camera mounted at the end of the robotic arm and is used to capture the local posture of the target in real time.

[0031] The method according to claim 4, characterized in that the first path weight α(d) and the second path weight 1-α(d) are allocated based on the real-time distance d between the end of the robot arm and the preset target;

[0032] At the i-th moment in the time region where the first subpath and the second subpath exceed the preset deviation, the third subpath can be expressed as:

[0033]

[0034] Preferably, the weight distribution function α(d) is expressed as:

[0035] α(d)=α1+(α2-α1)·e -λd ;

[0036] Among them, α1∈[0.2, 0.4] is the lower limit of the weight of the first visual sensor;

[0037] α2∈[0.6, 0.8], is the upper limit of the weight of the second visual sensor;

[0038] λ∈[0.3, 0.7], which is used to control the rate at which the weight decays with distance.

[0039] Preferably, the weight distribution function α(d) is expressed as:

[0040]

[0041] Among them, α1∈[0.2, 0.4] is the lower limit of the weight of the first visual sensor;

[0042] α2∈[0.6, 0.8], is the upper limit of the weight of the second visual sensor;

[0043] λ∈[0.3, 0.7], used to control the rate at which the weight decays with distance;

[0044] d0 is the preset distance threshold.

[0045] The second aspect of the present application provides a control system for a robotic arm based on visual intelligent positioning, including:

[0046] A first visual sensor is configured to obtain a first path based on the robot arm and a preset target position;

[0047] a second visual sensor configured to acquire a second path based on the robotic arm and a preset target position;

[0048] A storage module storing a preset deviation between the first path and the second path;

[0049] A determination module, configured to determine whether a deviation between the first path and the second path exceeds a preset deviation;

[0050] an adjustment module, which adjusts the first path based on the second path to obtain a third path, or adjusts the second path based on the first path to obtain a fourth path, when a deviation between the first path and the second path exceeds a preset deviation;

[0051] The path execution module is configured to control the robot arm to execute the first path or the second path when the deviation between the first path and the second path does not exceed the preset deviation; or to control the robot arm to execute the third path or the fourth path when the deviation between the first path and the second path exceeds the preset deviation.

[0052] A third aspect of the present application provides an electronic device, comprising:

[0053] at least one processor; and

[0054] a memory communicatively coupled to the at least one processor, wherein:

[0055] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

[0056] Compared with the known public technologies, the technical solution provided by the present application has the following beneficial effects: through the collaborative work of the first visual sensor and the second visual sensor, two independent paths are generated in the robot arm motion path planning stage, and the path deviation is compared in real time. When a single sensor causes data anomalies due to environmental interference, the system can trigger a dynamic adjustment mechanism based on the path deviation threshold, effectively avoiding the positioning errors caused by local failure of traditional single sensors, and improving the positioning robustness in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0058] Figure 1 This is a workflow diagram of the robotic arm control method based on visual intelligent positioning disclosed in an embodiment of the present application.

[0059] Figure 2 This is a structural schematic diagram of a robotic arm control system based on visual intelligent positioning disclosed in an embodiment of the present application.

[0060] Figure 3 This is a structural schematic diagram of an electronic device for executing a robotic arm control method based on visual intelligent positioning disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0062] An embodiment of the present application provides a robotic arm control method based on visual intelligent positioning. In a specific example, the hardware configuration is as follows:

[0063] Robotic arm: six-degree-of-freedom collaborative robot (such as UR5e), with a gripper and force sensor at the end;

[0064] First visual sensor: a global fixed RGB-D camera (such as Azure Kinect DK) (global camera), installed on the top of the working area, covering a range of 3m×3m, used to obtain global 3D point cloud;

[0065] Second visual sensor: terminal embedded binocular camera (such as ZED 2i) (terminal camera), with a resolution of 3840×1080 and a frame rate of 60fps, to capture the local posture of the target in real time;

[0066] Control unit: Embedded industrial computer (NVIDIA Jetson AGX Orin), running path planning algorithm and judgment module, adjustment module, and path execution module.

[0067] Among them, the path generation and adjustment are carried out according to the following steps:

[0068] S100: Obtaining a first path based on a first visual sensor according to the robot arm and a preset target position;

[0069] S200: Obtaining a second path based on a second visual sensor according to the robot arm and a preset target position;

[0070] S300: Compare the first path and the second path:

[0071] If the deviation between the first path and the second path exceeds a preset deviation, the first path is adjusted based on the second path to obtain a third path, or the second path is adjusted based on the first path to obtain a fourth path; and the robot arm is controlled based on the third path or the fourth path;

[0072] If the deviation between the first path and the second path does not exceed a preset deviation, the robot arm is controlled based on the first path or the second path.

[0073] In the above technical solution, the first path is generated by calculating the angle sequence of each joint of the robot arm through inverse kinematics based on the three-dimensional coordinates of the target object obtained by the global camera, and generating a time-space path by discretizing the time series;

[0074] The second path is generated by collecting local images of the target object in real time based on the terminal camera, identifying key feature points using the YOLO target detection model, converting them to the base coordinate system through hand-eye calibration, and generating a path represented by a spatial coordinate sequence.

[0075] In addition, when the first path and the second path are compared in step 300 and the deviation between the first path and the second path exceeds the preset deviation, the following steps are mainly included:

[0076] Step 310: Divide the first path into a plurality of first sub-paths based on the time series;

[0077] Step 320: Divide the second path into a plurality of second sub-paths based on the time sequence, wherein the number of the second sub-paths is equal to the number of the first sub-paths;

[0078] Step S330: comparing the first subpath and the second subpath in the corresponding time zone, adjusting the subpath segments exceeding the preset deviation, and obtaining a third subpath;

[0079] Step S340: splicing the third sub-path with the unadjusted portion of the first path to obtain a third path; or splicing the third sub-path with the unadjusted portion of the second path to obtain a fourth path.

[0080] In the above technical solution, when the first path and the second path are compared in step S300 and the deviation between the first path and the second path does not exceed the preset deviation, the robot arm is controlled based on the first path or the second path; in the actual working process, the robot arm can first be controlled to quickly position the robot arm with the first path obtained by the global camera. As the robot arm continues to approach the preset target, the second path obtained by the end camera can be used as the main control to accurately position the robot arm.

[0081] In the above technical solution, in step S310 and step S320, the first path / second path is divided into a plurality of sub-paths and the sub-path segments exceeding the preset deviation are adjusted to obtain the third sub-path mainly by the following steps:

[0082] 1) discretizing the first path and the second path into a spatial coordinate sequence corresponding to the timestamp at fixed time intervals (e.g., every 0.1 second);

[0083] 2) Align the time series of the two paths and calculate the minimum alignment distance D min If D min If it exceeds the preset deviation D0 (e.g. 0.6 mm), the sub-section is marked as an area to be adjusted;

[0084] 3) In the area to be adjusted, with the current robot arm position as the starting point and the target position as the focus, a real-time obstacle map is constructed in combination with the end camera data to generate a collision-free optimized sub-path;

[0085] 4) The optimized subpath is directly connected to the unadjusted part of the original first path or the second path according to the timestamp; during the splicing process, several points are taken before and after the splicing point to generate a smooth transition curve to ensure the continuity of acceleration, and the transition path segment is generated with the splicing point as the control point to avoid sudden stop or shaking of the robot arm.

[0086] In the above 4), if the first visual sensor detects a change in the environmental structure (such as displacement of a support), the optimized sub-paths are spliced ​​based on the first path to obtain a third path; if the second visual sensor detects a dynamic obstacle or a change in the target posture within the sub-path segment, the optimized sub-paths are spliced ​​based on the second path to obtain a fourth path; if the aforementioned conditions do not exist, the weights of the first path and the second path are allocated based on the real-time distance d between the end of the robot arm and the preset target, and the first path and the second path are spliced ​​based on the weights of the first path and the second path to obtain a third path or a fourth path.

[0087] In a specific example, when the real-time distance between the end of the robot arm and the preset target is d, at a certain i-th moment when the first subpath and the second subpath exceed the preset deviation, the third subpath can be expressed as:

[0088]

[0089] Among them, α(d) is the first path weight, 1-α(d) is the second path weight;

[0090] is the first subpath at the i-th moment; is the second subpath at the i-th moment; is the third subpath at the i-th moment.

[0091] In the above example, α(d) can be a function that decreases as d decreases, indicating that as the distance between the end of the robot arm and the preset target decreases, the weight of the first path gradually decreases, while the weight of the second path gradually increases; that is, as the distance between the end of the robot arm and the preset target gets closer, the local image of the preset target acquired by the second visual sensor is mainly used, which can effectively improve the accuracy of robot arm positioning based on the preset target.

[0092] After the third sub-path is obtained by the above method, the third sub-path is spliced ​​with the unadjusted part of the first path / second path to obtain a third path / fourth path, and the robot arm is subsequently controlled based on the third path / fourth path.

[0093] In a specific example, the weight distribution function α(d) is expressed as:

[0094] α(d)=α1+(α2-α1)·e -λd ;

[0095] Among them, α1∈[0.2, 0.4] is the lower limit of the weight of the first visual sensor;

[0096] α2∈[0.6, 0.8], is the upper limit of the weight of the second visual sensor;

[0097] λ∈[0.3, 0.7], which is used to control the rate at which the weight decays with distance.

[0098] In the above example, as the distance d between the end of the robot arm and the preset target continues to decrease, the weight distribution function α(d) continues to decrease, which means that the proportion of global three-dimensional point cloud data captured by the first visual sensor continues to decrease, while the proportion of local posture of the target captured in real time by the second visual sensor that is getting closer to the preset target continues to increase. This is conducive to continuously adjusting the proportion of the first path and the second path in the fusion path according to the real-time movement of the robot arm, thereby improving the accuracy of the robot arm positioning.

[0099] In another example of the present application, the weight distribution function α(d) is expressed as:

[0100]

[0101] Among them, α1∈[0.2, 0.4] is the lower limit of the weight of the first visual sensor;

[0102] α2∈[0.6, 0.8], is the upper limit of the weight of the second visual sensor;

[0103] λ∈[0.3, 0.7], used to control the rate at which the weight decays with distance;

[0104] d0 is the preset distance threshold.

[0105] In the above example, the positioning of the robot arm is divided into two stages, the first stage is the coarse positioning stage, and the second stage is the fine positioning stage.

[0106] Among them, the stage when the distance d between the end of the robot arm and the preset target is greater than the preset distance threshold d0 (such as 500mm) is defined as the first stage. At this time, the distance between the end of the robot arm and the preset target is far, and the weight function α(d) is defined as a fixed value α1, which reduces the calculation of the robot arm motion data, so that the robot arm can quickly locate to the preset position, that is, the position where the distance d between the end of the robot arm and the preset target is d=d0; subsequently, the stage when the distance d between the end of the robot arm and the preset target is less than the preset distance d0 is defined as the second stage. At this time, the distance between the end of the robot arm and the preset target is close, and it is necessary to adjust the proportion of the first sub-path and the second sub-path in the fusion sub-path in real time based on the distance between the end of the robot arm and the preset target to improve the positioning accuracy; that is, in the second stage, the weight distribution function α(d) decreases as the distance between the end of the robot arm and the preset target d continues to decrease as described above, that is, as the distance between the end of the robot arm and the preset target is closer, the proportion of data captured by the second visual sensor set at the end of the robot arm is larger, which is conducive to improving the positioning accuracy of the robot arm.

[0107] In this example, the positioning of the robot arm is divided into the first stage and the second stage. The first stage is mainly for rapid positioning, and the second stage is mainly for precise positioning. The two-stage positioning can take into account both positioning accuracy and positioning efficiency, and effectively improve the effectiveness of the positioning control of the robot arm.

[0108] In the above technical solution, the preset deviation may be a fixed value, which is used to indicate that when the deviation between the first sub-path and the second sub-path at the i-th moment exceeds the fixed threshold, the region is determined to be a sub-path to be adjusted.

[0109] In a better technical solution, the preset deviation includes:

[0110] The first deviation threshold δ1 is set based on the repeatability of the robot arm;

[0111] The second deviation threshold δ2 is a threshold adjusted based on the target motion speed, where δ1 and δ2 satisfy:

[0112] δ2=δ1*(1+k*v);

[0113] Where, v is the target movement speed;

[0114] k is a speed influence coefficient. In a specific example, k may be a constant, and the value of k is a positive value.

[0115] When the deviation between the first path and the second path exceeds a larger value of a first deviation threshold and a second deviation threshold, it is determined that the deviation between the first path and the second path exceeds a preset deviation.

[0116] In the above technical solution, the first deviation threshold is a static threshold, which is set based on the repeatability of the robot arm. For example, when the repeatability of the robot arm is ±01.6 mm, the first deviation threshold is set to δ1=0.15 mm. The static threshold reflects the inherent error limit of the hardware performance of the robot arm. That is, this static threshold is used to ensure that the correction of the path deviation does not exceed the physical capability of the robot arm, thereby avoiding execution failure (such as overshoot oscillation) due to excessive adjustment. The second deviation threshold is a dynamic threshold, which is adjusted in real time based on the target movement speed. That is, when the target moves at high speed, the second threshold automatically increases to allow a greater tolerance for path deviations, with priority given to ensuring the tracking continuity of the robot arm.

[0117] During the specific execution process, the larger value of the two is taken as the final deviation threshold to ensure that the system automatically selects the optimal strategy between static high-precision requirements and dynamic environmental adaptability.

[0118] In a specific scenario, when the target is stationary, that is, v = 0, the second deviation threshold is equal to the first deviation threshold, and the system executes according to the fixed deviation threshold to avoid unnecessary path fusion; in the high-speed motion scenario, it is assumed that the first deviation threshold (δ1) is fixed at 0.1mm; when the target moves quickly (such as a workpiece on a conveyor belt), the second deviation threshold (δ2) increases with the speed. For example, when v = 1m / s, δ2 = 0.3mm, and the traditional single threshold (such as a fixed 0.1mm) will frequently trigger path correction, causing the robot arm to shake. This solution allows for greater deviation tolerance, and the subsequent path is compensated by the prediction algorithm, and the overall tracking error is reduced (experimental data shows that the overall tracking error is reduced by 40%).

[0119] Another example of the present application discloses a control system for a robotic arm based on visual intelligent positioning, including: a first visual sensor 110, configured to obtain a first path based on the robotic arm and a preset target position; a second visual sensor 120, configured to obtain a second path based on the robotic arm and the preset target position; a storage module 130, storing a preset deviation between the first path and the second path; a judgment module 140, configured to judge whether the deviation between the first path and the second path exceeds the preset deviation; an adjustment module 150, when the deviation between the first path and the second path exceeds the preset deviation, adjusts the first path based on the second path to obtain a third path, or adjusts the second path based on the first path to obtain a fourth path; a path execution module 160, configured to control the robotic arm to execute the first path or the second path when the deviation between the first path and the second path does not exceed the preset deviation; or controls the robotic arm to execute the third path or the fourth path when the deviation between the first path and the second path exceeds the preset deviation.

[0120] The present application embodiment also provides an electronic device 20, please refer to Figure 3 , Figure 3 Schematic diagram of an electronic device 20 provided in an embodiment of the present application. In this embodiment, the electronic device 20 includes: a processor 21, a memory 22 and a bus 23, the memory 22 stores machine-readable instructions executable by the processor 21, when the electronic device 10 is running, the processor 21 communicates with the memory 22 through the bus 23, and when the machine-readable instructions are executed by the processor 21, the control method of the robotic arm based on visual intelligent positioning provided in the embodiment of the present application is executed.

[0121] Optionally, an embodiment of the present application further provides a storage medium having a computer program stored thereon, and when the computer program is executed by the processor 21, the control method of the robotic arm based on visual intelligent positioning provided in the embodiment of the present application is executed.

[0122] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0123] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0124] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A robotic arm control method based on visual intelligent positioning, characterized in that: include: According to the position of the robot arm and the preset target, a first path is obtained based on a first visual sensor; According to the position of the robot arm and the preset target, a second path is obtained based on the second visual sensor; Comparing the first path with the second path, if the deviation between the first path and the second path exceeds a preset deviation, adjusting the first path based on the second path to obtain a third path, or adjusting the second path based on the first path to obtain a fourth path; and controlling the robot arm based on the third path or the fourth path; If the deviation between the first path and the second path does not exceed a preset deviation, the robot arm is controlled based on the first path or the second path.

2. The method according to claim 1, characterized in that: Comparing the first path with the second path includes: discretizing the first path and the second path into a time series or a spatial coordinate series; The minimum alignment distance between the discretized first path and the second path is calculated, and if the minimum alignment distance exceeds a deviation threshold, it is determined that the deviation between the first path and the second path exceeds a preset deviation.

3. The method according to claim 1, characterized in that The preset deviations include: The first deviation threshold δ1 is set based on the repeatability of the robot arm; The second deviation threshold δ2 is a threshold adjusted based on the target movement speed or the density of environmental obstacles, where δ1 and δ2 satisfy: δ2=δ1*(1+k*v); Among them, v is the target movement speed, k is the speed influence coefficient; When the deviation between the first path and the second path exceeds a larger value of a first deviation threshold and a second deviation threshold, it is determined that the deviation between the first path and the second path exceeds a preset deviation.

4. The method according to claim 1, characterized in that: Comparing the first path with the second path includes: Dividing the first path into a plurality of first sub-paths based on the time series; Dividing the second path into a plurality of second sub-paths based on the time series, wherein the number of the second sub-paths is equal to the number of the first sub-paths; Compare the first subpath and the second subpath in the corresponding time zone, adjust the subpath segment exceeding the preset deviation, and obtain the third subpath; splicing the third sub-path with the unadjusted portion of the first path to obtain a third path; or The third sub-path and the unadjusted portion of the second path are concatenated to obtain a fourth path.

5. The method according to claim 1, characterized in that: The first visual sensor is a global fixed camera, which is used to obtain global three-dimensional information of the robot arm workspace; The second camera is a camera mounted at the end of the robotic arm and is used to capture the local posture of the target in real time.

6. The method according to claim 4, characterized in that Allocate a first path weight α(d) and a second path weight 1-α(d) based on the real-time distance d between the end of the robotic arm and the preset target; At the i-th moment in the time region where the first subpath and the second subpath exceed the preset deviation, the third subpath can be expressed as:

7. The method according to claim 6, characterized in that The weight distribution function α(d) is expressed as: α(d)=α1+(α2-α1)·e -λd ; Among them, α1∈[0.2, 0.4] is the lower limit of the weight of the first visual sensor; α2∈[0.6, 0.8], is the upper limit of the weight of the second visual sensor; λ∈[0.3, 0.7], which is used to control the rate at which the weight decays with distance.

8. The method according to claim 6, characterized in that The weight distribution function α(d) is expressed as: Among them, α1∈[0.2, 0.4] is the lower limit of the weight of the first visual sensor; α2∈[0.6, 0.8], is the upper limit of the weight of the second visual sensor; λ∈[0.3, 0.7], used to control the rate at which the weight decays with distance; d0 is the preset distance threshold.

9. A control system for a robotic arm based on visual intelligent positioning, characterized in that: include: A first visual sensor is configured to obtain a first path based on the robot arm and a preset target position; a second visual sensor configured to acquire a second path based on the robotic arm and a preset target position; A storage module storing a preset deviation between the first path and the second path; A determination module, configured to determine whether a deviation between the first path and the second path exceeds a preset deviation; an adjustment module, which adjusts the first path based on the second path to obtain a third path, or adjusts the second path based on the first path to obtain a fourth path, when a deviation between the first path and the second path exceeds a preset deviation; The path execution module is configured to control the robot arm to execute the first path or the second path when the deviation between the first path and the second path does not exceed the preset deviation; or to control the robot arm to execute the third path or the fourth path when the deviation between the first path and the second path exceeds the preset deviation.

10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively coupled to the at least one processor, wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

Images