Mechanical arm visual ranging method and ranging system thereof

Abstract

The invention discloses a mechanical arm vision distance measurement method and a distance measurement system thereof, and relates to the technical field of intelligent vision perception, a camera is installed on a mechanical arm to collect images, and through an optimization matching algorithm, the images shot by the left eye and the right eye are rapidly compared to generate a clear and accurate disparity map; the motion state of the mechanical arm is detected in real time through a vibration compensation mechanism, a shot image is corrected, and it is ensured that the distance measuring process is stable and reliable; and in combination with the corrected disparity map, the accurate distance from the target object to the camera is obtained by utilizing a depth calculation formula, and the result is transmitted to a control system of the mechanical arm. The depth value is fed back to the mechanical arm controller in real time through depth calculation and linkage with a control system, the precision of dynamic path planning and action execution of the mechanical arm is improved, and the mechanical arm shows higher anti-interference capacity in a dynamic vibration environment.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent visual perception, and in particular to a robot arm visual ranging method and a ranging system thereof. Background Art

[0002] In the field of modern industrial automation, robotic arms are widely used in a variety of scenarios such as manufacturing, assembly, inspection and logistics. Their core task is to accurately perceive the environment and perform complex operations. To achieve this goal, robotic arms need to rely on high-precision visual ranging technology to obtain the distance and position information of the target object. However, the existing technology still has the following shortcomings in practical applications:

[0003] 1. Conventional binocular vision ranging methods rely on disparity map generation, while existing algorithms (such as block matching or semi-global methods based on cost aggregation) have high computational complexity and are difficult to meet real-time requirements. They also have low robustness to noise and occlusion problems.

[0004] 2. The vibration compensation mechanism is missing or its performance is insufficient. When the robot arm is running, it will inevitably cause vibration, which will have a negative impact on the quality of the captured image. Existing solutions mostly use simple processing such as low-pass filtering, which fails to dynamically correct the ranging deviation caused by vibration in real time.

[0005] Therefore, there is an urgent need for a robotic arm visual ranging method and a ranging system to solve the above problems. Summary of the invention

[0006] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0007] In view of the above problems existing in the prior art, the present invention is proposed.

[0008] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0009] A robot arm visual ranging method comprises the following steps:

[0010] Step 1: Collect images by installing a camera on the robotic arm, and quickly compare the images taken by the left and right eyes through an optimized matching algorithm to generate a clear and accurate disparity map;

[0011] Step 2: Use the vibration compensation mechanism to detect the motion state of the robotic arm in real time and correct the captured image to ensure that the ranging process is stable and reliable;

[0012] Step 3: Combined with the corrected disparity map, use the depth calculation formula to get the precise distance from the target object to the camera, and pass this result to the control system of the robotic arm to help it complete the task.

[0013] As a preferred solution of the robot arm visual ranging method described in the present invention, the generation process of the disparity map in step 1 is as follows:

[0014] S101: Image acquisition, using the camera on the robotic arm to collect left and right views, respectively I L (x,y) and I R (x, y) represents the grayscale value or color value of the left and right images at the pixel coordinates (x, y);

[0015] S102: Matching accuracy calculation, optimizing the matching of the left and right images, calculating the difference between each pair of pixels, and performing weighted summation to form the matching accuracy;

[0016] S103: Generate a disparity map Δ(x, y) according to the difference between the left and right views, where the disparity is defined as the relative displacement of each pair of matching points.

[0017] As a preferred solution of the robot arm visual ranging method described in the present invention, the matching accuracy calculation formula is:

[0018]

[0019] Where M, N represents the size of the image block, w ij Represents the matching weight coefficient, which is used to highlight important areas. i and j represent the pixel offset within the image block, which is usually used to describe the relative position of a pixel block in the left and right images.

[0020] The calculation formula of the disparity map Δ(x, y) is:

[0021]

[0022] Among them, the normalization function max(S(x,y)) is used to ensure that the scale of the disparity map is uniform.

[0023] As a preferred solution of the robot arm visual ranging method described in the present invention, the vibration compensation mechanism includes:

[0024] Vibration state monitoring, using sensors to detect the motion vibration state of the robot arm in real time, the vibration function V(t) is expressed as: V(t) = A·sin(2πft);

[0025] Among them, A represents the maximum amplitude of vibration, f represents the vibration frequency, and t represents time;

[0026] By correction factor Cv (x,y) represents the effect of the vibration state on the image;

[0027] The vibration correction factor C v (x, y) acts on the disparity map to obtain the corrected disparity Δ′(x, y) = Δ(x, y)·C v (x,y).

[0028] As a preferred solution of the robot arm visual ranging method described in the present invention, the calculation formula is:

[0029] Among them, σ represents the amplitude parameter that controls the vibration attenuation, and the exp function represents the exponential attenuation effect of vibration on image correction.

[0030] As a preferred solution of the robot arm visual ranging method described in the present invention, the depth calculation specifically includes the following steps:

[0031] S301: Calculate the depth d(x, y) from the target object to the camera by combining the corrected disparity map Δ'(x, y) and the fixed camera parameters;

[0032]

[0033] Among them, f c represents the focal length of the camera, B represents the baseline distance between the left and right cameras, and p represents the actual physical spacing of each pixel;

[0034] S302: The calculated depth d(x, y) will be transmitted to the control system of the robotic arm for adjusting the position, avoiding obstacles or other operation tasks.

[0035] As a preferred solution of the robot arm visual ranging method described in the present invention, when the corrected disparity Δ'(x, y)≤0 is obtained, the depth calculation is determined to be invalid, and a prompt is given to check the quality of the input image pair and recalculate the disparity;

[0036] If the disparity value Δ'(x, y)>0, and the depth value d(x, y) is within the preset range, the depth calculation is determined to be valid, and the subsequent steps are continued;

[0037] If the depth value d(x,y) exceeds the preset range, adjust the focal length f in the formula c Or baseline distance B, recalculate to ensure the results converge.

[0038] A distance measurement system applied to the above-mentioned robot arm visual distance measurement method, the system comprises:

[0039] The image acquisition and processing module uses a binocular camera installed on the robotic arm to synchronously collect image data of the left and right views and perform basic preprocessing on the images;

[0040] The disparity calculation and depth estimation module uses an optimized matching algorithm to generate a high-precision disparity map, and estimates the distance between the target object and the camera through the disparity map combined with the depth calculation formula;

[0041] The vibration compensation module detects the motion state of the robot arm in real time, records the three-axis acceleration data, calculates the vibration correction value based on the acceleration data, and corrects the image data;

[0042] The data verification and feedback module verifies the calculation results of each step and dynamically adjusts the system parameters according to the verification results;

[0043] The control and decision-making module sends motion control instructions to the control system of the robotic arm based on the calculated precise distance data of the target object.

[0044] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of the above-mentioned robot arm visual ranging method are implemented.

[0045] A computer-readable storage medium having a computer program stored thereon, characterized in that: when the computer program is executed by a processor, the steps of the above-mentioned robot arm visual ranging method are implemented.

[0046] Beneficial effects of the present invention:

[0047] 1. By calculating the disparity map Δ(x,y), the matching window weight is optimized, which significantly improves the accuracy of the disparity map. High-quality disparity data can be output even in complex lighting and noise environments. The vibration compensation formula is used to effectively correct the image deviation caused by the movement of the robotic arm, ensuring that the ranging results are stable and reliable.

[0048] 2. Through depth calculation and linkage with the control system, the depth value is fed back to the robot controller in real time, which improves the accuracy of the robot's dynamic path planning and action execution. The robot arm exhibits higher anti-interference ability in a dynamic vibration environment and has strong adaptability. It can be widely used in high-demand industrial scenarios such as assembly lines, high-precision detection and complex robot operating environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0050] Figure 1 The present invention is a flowchart of a robot arm visual ranging method proposed by the present invention. DETAILED DESCRIPTION

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0052] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0053] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0054] Reference Figure 1 , is an embodiment of the present invention, and provides a robot arm visual ranging method and ranging system thereof, the method comprising the following steps:

[0055] Step 1: Collect images by installing a camera on the robotic arm, and quickly compare the images taken by the left and right eyes through an optimized matching algorithm to generate a clear and accurate disparity map.

[0056] The specific steps are:

[0057] S101: Image acquisition, using the camera on the robotic arm to collect left and right views, respectively I L (x,y) and I R (x, y) represents the grayscale value or color value of the left and right images at the pixel coordinates (x, y);

[0058] S102: Matching accuracy calculation, optimizing the matching of the left and right images, calculating the difference between each pair of pixels, and performing weighted summation to form the matching accuracy;

[0059] The matching accuracy calculation formula is:

[0060]

[0061] Where M, N represents the size of the image block, w ij It represents the matching weight coefficient, which is used to highlight important areas. i and j represent the pixel offset within the image block, which is usually used to describe the relative position of a pixel block in the left and right images.

[0062] x+i and y+j represent the offset positions of i and j pixels to the right and downward based on the pixel coordinates (x, y). This is because when performing image block matching, the pixels of one image block need to be compared with another image block, and the pixel comparison is usually based on pixel offset.

[0063] S103: Generate a disparity map Δ(x, y) according to the difference between the left and right views. The disparity is defined as the relative displacement of each pair of matching points. The calculation formula of the disparity map Δ(x, y) is:

[0064]

[0065] Among them, the normalization function max(S(x,y)) is used to ensure that the scale of the disparity map is uniform.

[0066] Step 2: Use the vibration compensation mechanism to detect the motion state of the robotic arm in real time and correct the captured image to ensure that the ranging process is stable and reliable.

[0067] Specifically, the vibration compensation mechanism includes:

[0068] Vibration state monitoring, using sensors to detect the motion vibration state of the robot arm in real time, the vibration function V(t) is expressed as: V(t) = A·sin(2πft);

[0069] Among them, A represents the maximum amplitude of vibration, f represents the vibration frequency, and t represents time;

[0070] By correction factor C v (,y) represents the influence of the vibration state on the image;

[0071] Correction factor C v The calculation formula of (,y) is:

[0072]

[0073] Among them, σ represents the amplitude parameter that controls the vibration attenuation, and the exp function represents the exponential attenuation effect of vibration on image correction.

[0074] The vibration correction factor C v (x, y) acts on the disparity map to obtain the corrected disparity Δ′(x, y) = Δ(x, y)·C v(x,y).

[0075] Step 3: Combined with the corrected disparity map, use the depth calculation formula to get the precise distance from the target object to the camera, and pass this result to the control system of the robotic arm to help it complete the task.

[0076] If the corrected disparity Δ'(x,y)≤0, the depth calculation is considered invalid, prompting you to check the quality of the input image pair and recalculate the disparity;

[0077] If the disparity value Δ'(x, y)>0, and the depth value d(x, y) is within the preset range, the depth calculation is determined to be valid, and the subsequent steps are continued;

[0078] If the depth value d(x,y) exceeds the preset range, adjust the focal length f in the formula c Or baseline distance B, recalculate to ensure the results converge.

[0079] Depth calculation specifically includes the following steps:

[0080] S301: Calculate the depth d(x, y) from the target object to the camera by combining the corrected disparity map Δ'(x, y) and the fixed camera parameters;

[0081]

[0082] Among them, f c represents the focal length of the camera, B represents the baseline distance between the left and right cameras, and p represents the actual physical spacing of each pixel;

[0083] S302: The calculated depth d(x, y) will be transmitted to the control system of the robotic arm for adjusting the position, avoiding obstacles or other operation tasks.

[0084] A distance measurement system of a robot arm visual distance measurement method as above is characterized in that the system comprises:

[0085] The image acquisition and processing module uses a binocular camera installed on the robotic arm to synchronously collect image data of the left and right views and perform basic preprocessing on the images;

[0086] The disparity calculation and depth estimation module uses an optimized matching algorithm to generate a high-precision disparity map, and estimates the distance between the target object and the camera through the disparity map combined with the depth calculation formula;

[0087] The vibration compensation module detects the motion state of the robot arm in real time, records the three-axis acceleration data, calculates the vibration correction value based on the acceleration data, and corrects the image data;

[0088] The data verification and feedback module verifies the calculation results of each step and dynamically adjusts the system parameters according to the verification results;

[0089] The control and decision-making module sends motion control instructions to the control system of the robotic arm based on the calculated precise distance data of the target object.

[0090] In order to verify the beneficial effects of the present invention, a set of experimental data is used for verification. The experimental equipment includes:

[0091] Binocular vision camera (resolution 1920×1080, baseline length b=100mm), robotic arm controller, test conditions: 1. Vibration frequency: no vibration, low frequency (1Hz), medium frequency (5Hz), high frequency (10Hz). 2. Lighting conditions: normal light, strong light interference, low light.

[0092] The evaluation indicators after the test include: ranging error (Ed): the absolute difference between the real distance and the calculated distance, unit: mm; image matching time (Tm): the time to calculate the disparity map, unit: ms; image correction offset (Δv): the image position offset after vibration compensation, unit: pixel.

[0093] Table 1 is the data comparison table after the test

[0094]

[0095]

[0096] Combined with the data analysis, it can be seen that under vibration-free conditions, the ranging error is less than 1mm, proving that this solution has high precision in static scenes. Under vibration conditions, through the vibration compensation mechanism, the errors under low-frequency and medium-frequency interference are controlled at 1.2mm and 2.5mm respectively; even under high-frequency vibration conditions, the error is significantly lower than the typical value without compensation (usually 10+mm).

[0097] The corrected offset Δv after vibration compensation is controlled within 5 pixels in low-frequency and medium-frequency vibrations, and is up to 10 pixels in high-frequency vibrations, which greatly reduces the ranging error caused by image offset; the disparity map matching time Tm is 35ms under normal conditions, and the time delay after vibration compensation is controlled within 10ms, which can meet the real-time requirements; under strong light interference and low light conditions, the maximum ranging error is 8.9mm, which is still better than the average error of the prior art (15+mm), reflecting the strong anti-interference ability of the present invention.

[0098] In summary, the robot arm visual ranging method of the present invention can achieve high-precision and stable ranging under a variety of complex conditions and meet real-time requirements. Through the combination of vibration compensation and optimization calculation formula, the present invention provides strong technical support for the precise operation of the robot arm in industrial automation scenarios, and has significant practical value and promotion significance.

[0099] This embodiment also provides a computer device, which is applicable to a robot arm visual ranging method and a ranging system thereof, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to implement a robot arm visual ranging method and a ranging system thereof proposed in the above embodiment.

[0100] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0101] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, a robot arm visual ranging method and a ranging system thereof are implemented as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0102] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A robot arm visual ranging method, characterized in that: The following steps are involved: Step 1: Collect images by installing a camera on the robotic arm, and quickly compare the images taken by the left and right eyes through an optimized matching algorithm to generate a clear and accurate disparity map; Step 2: Use the vibration compensation mechanism to detect the motion state of the robotic arm in real time and correct the captured image to ensure that the ranging process is stable and reliable; Step 3: Combined with the corrected disparity map, use the depth calculation formula to get the precise distance from the target object to the camera, and pass this result to the control system of the robotic arm to help it complete the task.

2. A robot arm visual ranging method according to claim 1, characterized in that: The process of generating the disparity map in step 1 is specifically as follows: S101: Image acquisition, using the camera on the robotic arm to collect left and right views, respectively I L (x,y) and I R (x, y) represents the grayscale value or color value of the left and right images at the pixel coordinates (x, y); S102: Matching accuracy calculation, optimizing the matching of the left and right images, calculating the difference between each pair of pixels, and performing weighted summation to form the matching accuracy; S103: Generate a disparity map Δ(x, y) according to the difference between the left and right views, where the disparity is defined as the relative displacement of each pair of matching points.

3. A robot arm visual ranging method according to claim 2, characterized in that: The matching accuracy calculation formula is: Where M, N represents the size of the image block, w ij Represents the matching weight coefficient, which is used to highlight important areas. i and j represent the pixel offset within the image block, which is usually used to describe the relative position of a pixel block in the left and right images. The calculation formula of the disparity map Δ(x, y) is: Among them, the normalization function max(S(x,y)) is used to ensure that the scale of the disparity map is uniform.

4. A robot arm visual ranging method according to claim 3, characterized in that: The vibration compensation mechanism includes: Vibration state monitoring, using sensors to detect the motion vibration state of the robot arm in real time, the vibration function V(t) is expressed as: V(t) = A·sin(2πft); Among them, A represents the maximum amplitude of vibration, f represents the vibration frequency, and t represents time; By correction factor C v (x,y) represents the effect of the vibration state on the image; The vibration correction factor C v (x, y) acts on the disparity map to obtain the corrected disparity Δ'(x, y) = Δ(x, y)·C v (x,y).

5. A robot arm visual ranging method according to claim 4, characterized in that: The correction factor C v The calculation formula for (x, y) is: Number attenuation effect.

6. A robot arm visual ranging method according to claim 5, characterized in that: The depth calculation specifically includes the following steps: S301: Calculate the depth d(x, y) from the target object to the camera by combining the corrected disparity map Δ'(x, y) and the fixed camera parameters; Actual physical distance; S302: The calculated depth d(x, y) will be transmitted to the control system of the robotic arm for adjusting the position, avoiding obstacles or other operation tasks.

7. A robot arm visual ranging method according to claim 6, characterized in that: When the corrected disparity Δ'(x,y)≤0 is obtained, the depth calculation is determined to be invalid, and a prompt is given to check the quality of the input image pair and recalculate the disparity; If the disparity value Δ'(x, y)>0, and the depth value d(x, y) is within the preset range, the depth calculation is determined to be valid, and the subsequent steps are continued; If the depth value d(x,y) exceeds the preset range, adjust the focal length f in the formula c Or baseline distance B, recalculate to ensure the results converge.

8. The distance measurement system of the robot arm visual distance measurement method according to claim 7, characterized in that: The system includes: The image acquisition and processing module uses a binocular camera installed on the robotic arm to synchronously collect image data of the left and right views and perform basic preprocessing on the images; The disparity calculation and depth estimation module uses an optimized matching algorithm to generate a high-precision disparity map, and estimates the distance between the target object and the camera through the disparity map combined with the depth calculation formula; The vibration compensation module detects the motion state of the robot arm in real time, records the three-axis acceleration data, calculates the vibration correction value based on the acceleration data, and corrects the image data; The data verification and feedback module verifies the calculation results of each step and dynamically adjusts the system parameters according to the verification results; The control and decision-making module sends motion control instructions to the control system of the robotic arm based on the calculated precise distance data of the target object.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of a robot arm visual ranging method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a robot arm visual ranging method as described in any one of claims 1 to 7 are implemented.

Images