A non-calibrated vision dynamic target tracking method, device, equipment and medium

Abstract

The application discloses a kind of dynamic target tracking method, device, equipment and medium without calibration vision, and the first image and the second image are obtained by camera module in the application, and the coordinate conversion matrix of camera module and mechanical arm is calculated according to the first image and the second image, without the calibration process of traditional mode, improve efficiency and convenience;The target image of dynamic target is obtained by camera module, the first trajectory point sequence is obtained by image processing to target image, the second trajectory point sequence under the coordinate system of mechanical arm is obtained according to the first trajectory point sequence and the coordinate conversion matrix, real-time tracking is carried out to dynamic target according to the second trajectory point sequence, return the step of obtaining the target image of dynamic target by camera module, until reaching preset condition, can real-time iterative tracking, real-time correction tracking error, improve accuracy, the application can be widely used in machine vision technical field.

Description

Technical Field

[0001] This invention relates to the field of machine vision, and in particular to a method, apparatus, device, and medium for dynamic target tracking without calibration. Background Technology

[0002] Robots track and grasp target objects using vision, requiring precise transformation of the target coordinates acquired by the vision sensor into the robot's coordinate system. Current traditional robot vision solutions calibrate a fixed transformation relationship using the camera's intrinsic and extrinsic parameters and the robot arm's parameters. This calibration process is computationally complex and requires ensuring that the relative positions of the camera and robot remain unchanged after calibration. However, because robot working environments are often complex and the robot arm needs to repeatedly move, pause, and change direction rapidly, the positions of the robot arm and camera can easily change, such as due to camera loosening. This leads to inaccurate transformation relationships obtained from the original calibration, making it difficult to guarantee calibration and tracking accuracy. Summary of the Invention

[0003] In view of this, in order to solve the above-mentioned technical problems, the purpose of this invention is to provide a method, apparatus, device and medium for dynamic target tracking without calibration vision, so as to improve efficiency and accuracy.

[0004] The technical solution adopted in the embodiments of the present invention is as follows:

[0005] A calibration-free vision-based dynamic target tracking method, comprising:

[0006] A first image and a second image are acquired by a camera module, and a coordinate transformation matrix between the robotic arm and the camera module is calculated based on the first image and the second image. The camera module is installed on the robotic arm. The first image is acquired at a first position, and the second image is acquired after the robotic arm moves from the first position to the second position.

[0007] The camera module acquires a target image of the dynamic target, and performs image processing on the target image to obtain a first trajectory point sequence.

[0008] Based on the first trajectory point sequence and the coordinate transformation matrix, a second trajectory point sequence in the coordinate system of the robotic arm is obtained;

[0009] The dynamic target is tracked in real time according to the second trajectory point sequence, and the step of acquiring the target image of the dynamic target through the camera module is returned until the preset condition is met.

[0010] Furthermore, the step of acquiring a first image and a second image through the camera module, and calculating the coordinate transformation matrix between the robotic arm and the camera module based on the first image and the second image, includes:

[0011] A first image is acquired through a camera module. After the robotic arm is controlled to move to different second positions in different directions, a sub-image is acquired through the camera module. The second image includes several sub-images. The first image includes first image feature points, and the sub-images include second image feature points corresponding to the first image feature points.

[0012] The coordinate deviation is calculated based on the first image feature points and the second image feature points, and the moving speed vector is calculated based on the coordinate deviation.

[0013] The translational velocity and rotational velocity vector of the robotic arm as it moves from the first position to each of the second positions are obtained. Based on the translational velocity, the rotational velocity vector, and the movement velocity vector, a coordinate transformation matrix is ​​calculated.

[0014] Further, the step of calculating the coordinate deviation based on the first image feature point and the second image feature point includes:

[0015] Based on the first image feature point and the second image feature point, determine the movement trajectory of the image feature point;

[0016] The endpoints and the midpoint of the movement trajectory are determined from the movement trajectory, and the target point is determined according to the midpoint and the endpoints based on a preset rule;

[0017] The coordinate deviation is calculated based on the coordinate positions of the endpoint, the midpoint, and the target point in the first and second images.

[0018] Further, the step of calculating the coordinate transformation matrix based on the translational velocity, the rotational velocity vector, and the movement velocity vector includes:

[0019] Generate a matrix based on the translational velocity and the rotational velocity vector;

[0020] The coordinate transformation matrix is ​​obtained based on the ratio of the moving speed vector to the matrix.

[0021] Further, the step of performing image processing on the target image to obtain the first trajectory point sequence includes:

[0022] The target image is processed to obtain the target trajectory contour;

[0023] The target contour is obtained by filtering based on the target trajectory contour and preset specification parameters; the preset specification parameters include at least one of area, perimeter, and aspect ratio.

[0024] The target contour is subjected to erosion and dilation processing, and the erosion and dilation processing result is subjected to bone extraction processing to obtain a trajectory with a single pixel width.

[0025] Discretize the trajectory with the width of a single pixel to obtain a first trajectory point sequence.

[0026] Further, obtaining the second trajectory point sequence in the coordinate system of the robotic arm based on the first trajectory point sequence and the coordinate transformation matrix includes:

[0027] The first trajectory point sequence is transformed according to the coordinate transformation matrix to obtain the transformed trajectory point sequence;

[0028] A historical trajectory point sequence is obtained, and the overlapping parts in the transformed trajectory point sequence are removed based on the historical trajectory point sequence to obtain a second trajectory point sequence; the historical trajectory point sequence is the old transformed trajectory point sequence obtained in the previous time period.

[0029] Furthermore, the real-time tracking of the dynamic target based on the second trajectory point sequence includes:

[0030] The second trajectory point sequence is input into the robotic arm, which is then controlled to track the dynamic target in real time in a non-blocking mode.

[0031] This invention also provides a calibration-free vision-based dynamic target tracking device, comprising:

[0032] The first module is used to acquire a first image and a second image through a camera module, and to calculate the coordinate transformation matrix between the robotic arm and the camera module based on the first image and the second image; the camera module is set on the robotic arm, the first image is acquired at a first position, and the second image is acquired after the robotic arm moves from the first position to the second position;

[0033] The second module is used to acquire a target image of a dynamic target through the camera module, and to perform image processing on the target image to obtain a first trajectory point sequence.

[0034] The third module is used to obtain the second trajectory point sequence in the coordinate system of the robotic arm based on the first trajectory point sequence and the coordinate transformation matrix.

[0035] The fourth module is used to track the dynamic target in real time according to the second trajectory point sequence, and return the target image of the dynamic target obtained by the camera module until the preset conditions are met.

[0036] This invention also provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, a code set, or an instruction set. The at least one instruction, the at least one program, the code set, or the instruction set are loaded and executed by the processor to implement the method.

[0037] This invention also provides a computer-readable storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the method.

[0038] The beneficial effects of this invention are as follows: First and second images are acquired through a camera module, and the coordinate transformation matrix between the robotic arm and the camera module is calculated based on these images, eliminating the need for traditional calibration processes and improving efficiency and convenience. A target image of a dynamic target is acquired through the camera module, and image processing is performed on the target image to obtain a first trajectory point sequence. Based on the first trajectory point sequence and the coordinate transformation matrix, a second trajectory point sequence in the coordinate system of the robotic arm is obtained. The dynamic target is tracked in real time based on the second trajectory point sequence, returning to the step of acquiring the target image of the dynamic target through the camera module until a preset condition is met. This enables real-time iterative tracking, real-time correction of tracking errors, and improved accuracy. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating the steps of the dynamic target tracking method without calibration vision according to the present invention. Detailed Implementation

[0040] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0041] The terms "first," "second," "third," and "fourth," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0042] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0043] like Figure 1 As shown, this embodiment of the invention provides a dynamic target tracking method without calibration vision, including steps S100-S400:

[0044] S100. Acquire the first image and the second image through the camera module, and calculate the coordinate transformation matrix between the robotic arm and the camera module based on the first image and the second image.

[0045] In this embodiment of the invention, a ring-shaped light source is provided at the end of the robotic arm, which can be used to adjust the intensity of the light source. The camera module can be mounted on the robotic arm via a base and located at the central axis of the light source, so that the camera module can stably and clearly acquire images. It should be noted that the camera module has a coordinate system (or a base coordinate system), and the robotic arm has a coordinate system.

[0046] Optionally, when the robotic arm is in the first position, a first image is acquired through the camera module; after the robotic arm is controlled to move from the first position to the second position, a second image is acquired through the camera module.

[0047] Specifically, step S100 includes steps S110-S130:

[0048] S110. Acquire a first image through the camera module, control the robotic arm to move to different second positions in different directions, and then acquire a sub-image through the camera module.

[0049] It should be noted that the directions include the X, Y, and Z directions of the coordinate system. Other embodiments may include other numbers of directions without specific limitation. Moving to the corresponding second position in the three directions, i.e., a total of three second positions, can obtain three sub-images through the camera module. At this time, the second image includes three sub-images. The first image includes first image feature points, and the sub-images include second image feature points corresponding to the first image feature points. The first and second image feature points can be identified by image processing algorithms.

[0050] S120. Calculate the coordinate deviation based on the feature points of the first image and the feature points of the second image, and calculate the moving speed vector based on the coordinate deviation.

[0051] Optionally, the calculation of coordinate deviation based on the first image feature points and the second image feature points in step S120 includes steps S1201-S1203:

[0052] S1201. Determine the movement trajectory of the image feature points based on the first image feature points and the second image feature points.

[0053] Optionally, the selection of the first image feature point and the second image feature point is determined according to the actual situation and is not limited; based on the second image feature point obtained after the first image feature point is moved, the movement trajectory of the first image feature point and the image feature point represented by the first image feature point is determined.

[0054] S1202. Determine the endpoints and midpoints of the movement trajectory from the movement trajectory, and determine the target point based on the midpoints and endpoints according to preset rules.

[0055] Optionally, the endpoint and preset estimate can be adjusted according to the actual situation and are not specifically limited. For example, the endpoint can be the lower endpoint in the movement trajectory. Since the camera-robotic arm transformation matrix obtained from a feature point is a 2*6 non-square matrix, the preset rule is to select the lower endpoint in the movement trajectory, the midpoint of the movement trajectory, and generate the vertex of a third isosceles triangle based on the coordinates of these two points as the corner points of the isosceles triangle. The coordinates of this vertex are used as the target point.

[0056] S1203. Calculate the coordinate deviation based on the coordinate positions of the endpoint, midpoint, and target point in the first and second images.

[0057] Optionally, the coordinate deviation can be calculated based on the coordinate positions of the endpoint, midpoint, and target point in the first and second images.

[0058] Optionally, in step S120, the moving speed vector is calculated based on the coordinate deviation. Specifically, based on the coordinate deviation and the time taken to move from the first position to the second position, the speed vectors of the endpoint, midpoint, and target point can be calculated, thus obtaining the moving speed vector. in, The velocity vector at the endpoint. For the midpoint velocity vector, The velocity vector of the target point.

[0059] S130. Obtain the translational velocity and rotational velocity vector of the robotic arm as it moves from the first position to each of the second positions. Calculate the coordinate transformation matrix based on the translational velocity, rotational velocity vector, and movement velocity vector.

[0060] Optionally, step S130 includes steps S1301-S1302:

[0061] S1301. Generate a matrix based on the translational velocity and rotational velocity vector.

[0062] Specifically, the translational velocity and rotational velocity vector of the robotic arm as it moves from the first position to each of the second positions can be read from the robotic arm's control system. The translational velocity is denoted as (T). x ,T y ,T z The rotational velocity vector is denoted as (W). x W y W z ), where T x T represents the translational velocity component in the X direction. y T represents the translational velocity component in the Y direction. z W represents the translational velocity component in the Z direction. x W represents the rotational velocity component in the X direction. y W represents the rotational velocity component in the Y direction. z Generate a matrix for the rotational velocity component in the Z direction.

[0063] S1302. Based on the ratio of the moving speed vector to the matrix, obtain the coordinate transformation matrix.

[0064] Optionally, the formula for calculating the coordinate transformation matrix A is as follows. Therefore, the coordinate transformation matrix A can be calculated based on the ratio of the moving speed vector to the matrix. The coordinate transformation matrix A is a Jacobian matrix.

[0065]

[0066] S200. Acquire the target image of the dynamic target through the camera module, perform image processing on the target image, and obtain the first trajectory point sequence.

[0067] Specifically, the camera module acquires target images of dynamic targets in motion.

[0068] Optionally, step S200, which involves image processing of the target image to obtain the first trajectory point sequence, includes steps S210-S240:

[0069] S210. Perform recognition processing on the target image to obtain the target trajectory contour.

[0070] Specifically, the target image is processed by image processing algorithms to obtain the target trajectory contour.

[0071] S220. The target contour is obtained by filtering based on the target trajectory contour and preset specification parameters.

[0072] Optionally, the preset specification parameters include at least one of area, perimeter, and aspect ratio. For example, the target trajectory contour is filtered based on its outline, area, perimeter, and aspect ratio, and the filtering result is used as the target trajectory contour.

[0073] S230. Perform erosion and dilation processing on the target contour, and perform bone extraction processing on the erosion and dilation processing result to obtain a trajectory with a single pixel width.

[0074] S240. Discretize the trajectory with a single pixel width to obtain the first trajectory point sequence.

[0075] In this embodiment of the invention, the target contour is subjected to erosion and dilation processing, and the erosion and dilation processing result is subjected to bone extraction processing through a bone extraction algorithm to obtain a trajectory with a single pixel width. Then, the trajectory with a single pixel width is discretized to obtain a first trajectory point sequence.

[0076] S300. Based on the first trajectory point sequence and the coordinate transformation matrix, obtain the second trajectory point sequence in the coordinate system of the robotic arm.

[0077] S400: Track the dynamic target in real time according to the second trajectory point sequence, and return to the step of acquiring the target image of the dynamic target through the camera module until the preset conditions are met.

[0078] Specifically, step S300 includes steps S310-S320:

[0079] S310. The first trajectory point sequence is transformed according to the coordinate transformation matrix to obtain the transformed trajectory point sequence.

[0080] Specifically, the coordinate transformation matrix is ​​multiplied by the first trajectory point sequence to obtain the transformed trajectory point sequence in the robotic arm coordinate system.

[0081] S320. Obtain the historical trajectory point sequence, and remove the overlapping parts in the transformed trajectory point sequence based on the historical trajectory point sequence to obtain the second trajectory point sequence.

[0082] It should be noted that the historical trajectory point sequence is the old transformed trajectory point sequence obtained in the previous time period. For example, in step S400, the step of returning to the step of acquiring the target image of the dynamic target through the camera module is performed once. In each iteration, a new coordinate transformation matrix and the corresponding transformed trajectory point sequence are obtained. This is recorded as a time period. Therefore, each time period has a corresponding transformed trajectory point sequence. Assuming that the current iteration is the second iteration, the old transformed trajectory point sequence obtained in the previous time period is the transformed trajectory point sequence obtained in the first iteration. If the current iteration is the first iteration, the old transformed trajectory point sequence obtained in the previous time period can be a preset empty sequence.

[0083] Optionally, the overlapping portion is determined by comparing the historical trajectory point sequence with the transformed trajectory point sequence, and then the overlapping portion in the transformed trajectory point sequence is removed to obtain the second trajectory point sequence.

[0084] It should be noted that the S400 performs real-time tracking of the dynamic target based on the second trajectory point sequence. Specifically, the second trajectory point sequence is input to the robotic arm (such as its control system), which controls the robotic arm to track the dynamic target in real-time in a non-blocking mode. It should be noted that in the non-blocking mode, the robotic arm's control system issues a movement command and executes the next command directly without waiting for status feedback.

[0085] In this embodiment of the invention, step S400 returns to the step of acquiring the target image of the dynamic target through the camera module, that is, step S200 is re-executed to acquire a new target image, thereby finally determining a new coordinate transformation matrix and a new second trajectory point sequence to achieve iteration. In this way, the coordinate transformation relationship error caused by interference factors that may occur during the movement of the robotic arm, such as the camera module becoming loose or the relative position between the camera module and the robotic arm, is corrected through real-time iterative updates. This improves the accuracy and real-time performance of real-time tracking, compresses the visual processing cycle (i.e., the cycle from image processing of the target image to obtaining the second trajectory point sequence) to less than ten milliseconds, and the processing of the visual processing cycle and the robotic arm tracking are performed simultaneously in two threads to improve the real-time performance of tracking, enabling the robotic arm to work efficiently, easily, and accurately in complex environments.

[0086] It should be noted that the preset conditions include, but are not limited to, knowing that the target image of the dynamic target cannot be obtained through the camera module, that is, the dynamic target is outside the field of view of the camera module. At this time, the control robot arm and camera module enter the standby state.

[0087] This invention also provides a calibration-free vision-based dynamic target tracking device, comprising:

[0088] The first module is used to acquire a first image and a second image through the camera module, and to calculate the coordinate transformation matrix between the robotic arm and the camera module based on the first image and the second image; the camera module is set on the robotic arm, the first image is acquired at a first position, and the second image is acquired after the robotic arm moves from the first position to the second position;

[0089] The second module is used to acquire target images of dynamic targets through the camera module, perform image processing on the target images, and obtain the first trajectory point sequence.

[0090] The third module is used to obtain the second trajectory point sequence in the coordinate system of the robotic arm based on the first trajectory point sequence and the coordinate transformation matrix.

[0091] The fourth module is used to track the dynamic target in real time based on the second trajectory point sequence, and return the steps of acquiring the target image of the dynamic target through the camera module until the preset conditions are met.

[0092] The content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0093] This invention also provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, a code set, or an instruction set. The processor loads and executes the at least one instruction, at least one program, a code set, or an instruction set to implement the calibration-free vision-based dynamic target tracking method of the aforementioned embodiments. The electronic device of this invention includes, but is not limited to, mobile phones, tablet computers, computers, and in-vehicle computers.

[0094] The content of the above method embodiments is applicable to this device embodiment. The specific functions implemented in this device embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0095] This invention also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the calibration-free vision dynamic target tracking method of the foregoing embodiments.

[0096] This invention also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the calibration-free vision-based dynamic target tracking method of the foregoing embodiments.

[0097] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0098] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0099] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs. Additionally, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.

[0100] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them; those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features, without causing the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for dynamic target tracking without calibration vision, characterized in that, include: The first image and the second image are acquired by the camera module, and the coordinate transformation matrix between the robotic arm and the camera module is calculated based on the first image and the second image. The camera module is mounted on the robotic arm. The first image is acquired at a first position, and the second image is acquired after the robotic arm moves from the first position to the second position. The camera module acquires a target image of the dynamic target, and performs image processing on the target image to obtain a first trajectory point sequence. Based on the first trajectory point sequence and the coordinate transformation matrix, a second trajectory point sequence in the coordinate system of the robotic arm is obtained; The dynamic target is tracked in real time according to the second trajectory point sequence, and the step of acquiring the target image of the dynamic target through the camera module is returned until the preset condition is met. The step of acquiring a first image and a second image through a camera module, and calculating the coordinate transformation matrix between the robotic arm and the camera module based on the first image and the second image, includes: A first image is acquired through a camera module. After the robotic arm is controlled to move to different second positions in different directions, a sub-image is acquired through the camera module. The second image includes several sub-images. The first image includes first image feature points, and the sub-images include second image feature points corresponding to the first image feature points. The coordinate deviation is calculated based on the first image feature points and the second image feature points, and the moving speed vector is calculated based on the coordinate deviation. The translational velocity and rotational velocity vector of the robotic arm as it moves from the first position to each of the second positions are obtained. Based on the translational velocity, the rotational velocity vector, and the movement velocity vector, a coordinate transformation matrix is ​​calculated.

2. The dynamic target tracking method without calibration vision according to claim 1, characterized in that: The step of calculating the coordinate deviation based on the first image feature points and the second image feature points includes: Based on the first image feature point and the second image feature point, determine the movement trajectory of the image feature point; The endpoints and the midpoint of the movement trajectory are determined from the movement trajectory, and the target point is determined according to the midpoint and the endpoints based on a preset rule; The coordinate deviation is calculated based on the coordinate positions of the endpoint, the midpoint, and the target point in the first and second images.

3. The dynamic target tracking method without calibration vision according to claim 1, characterized in that: The step of calculating the coordinate transformation matrix based on the translational velocity, the rotational velocity vector, and the movement velocity vector includes: Generate a matrix based on the translational velocity and the rotational velocity vector; The coordinate transformation matrix is ​​obtained based on the ratio of the moving speed vector to the matrix.

4. The dynamic target tracking method without calibration vision according to any one of claims 1-3, characterized in that: The step of image processing the target image to obtain the first trajectory point sequence includes: The target image is processed to obtain the target trajectory contour; The target contour is obtained by filtering based on the target trajectory contour and preset specification parameters; the preset specification parameters include at least one of area, perimeter, and aspect ratio. The target contour is subjected to erosion and dilation processing, and the erosion and dilation processing result is subjected to bone extraction processing to obtain a trajectory with a single pixel width. Discretize the trajectory with the width of a single pixel to obtain a first trajectory point sequence.

5. The dynamic target tracking method without calibration vision according to any one of claims 1-3, characterized in that: The step of obtaining the second trajectory point sequence in the coordinate system of the robotic arm based on the first trajectory point sequence and the coordinate transformation matrix includes: The first trajectory point sequence is transformed according to the coordinate transformation matrix to obtain the transformed trajectory point sequence; A historical trajectory point sequence is obtained, and the overlapping parts in the transformed trajectory point sequence are removed based on the historical trajectory point sequence to obtain a second trajectory point sequence; the historical trajectory point sequence is the old transformed trajectory point sequence obtained in the previous time period.

6. The dynamic target tracking method without calibration vision according to any one of claims 1-3, characterized in that: The real-time tracking of the dynamic target based on the second trajectory point sequence includes: The second trajectory point sequence is input into the robotic arm, which is then controlled to track the dynamic target in real time in a non-blocking mode.

7. An apparatus for implementing the calibration-free vision dynamic target tracking method as described in any one of claims 1-6, characterized in that, include: The first module is used to acquire a first image and a second image through a camera module, and to calculate the coordinate transformation matrix between the robotic arm and the camera module based on the first image and the second image; the camera module is set on the robotic arm, the first image is acquired at a first position, and the second image is acquired after the robotic arm moves from the first position to the second position; The second module is used to acquire a target image of a dynamic target through the camera module, and to perform image processing on the target image to obtain a first trajectory point sequence. The third module is used to obtain the second trajectory point sequence in the coordinate system of the robotic arm based on the first trajectory point sequence and the coordinate transformation matrix. The fourth module is used to track the dynamic target in real time according to the second trajectory point sequence, and return the target image of the dynamic target obtained by the camera module until the preset conditions are met.

8. An electronic device, characterized in that: The electronic device includes a processor and a memory, the memory storing at least one instruction, at least one program, a code set, or an instruction set, the at least one instruction, the at least one program, the code set, or the instruction set being loaded and executed by the processor to implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the method as described in any one of claims 1-6.

Images