High-frequency dynamic tracking method based on multiple sensors, automatic robot, equipment and storage medium

Abstract

The invention discloses a high-frequency dynamic tracking method based on multiple sensors, an automatic robot, equipment and a storage medium, and relates to the technical field of dynamic tracking, and the high-frequency dynamic tracking method based on multiple sensors comprises the steps: obtaining the distance between a base and a target according to a first sensor, controlling the automatic robot to advance to a target position according to the distance; when the automatic robot reaches the target position, acquiring an initial coordinate of a target point location according to the second sensor, and performing coordinate conversion on the initial coordinate to obtain a target coordinate of the target point location; and controlling the mechanical arm to track the target point location according to the target coordinate. According to the invention, through cooperative work of various sensors, the accuracy of dynamic tracking is improved.

Description

Technical Field

[0001] The present application relates to the field of dynamic tracking technology, and in particular to a high-frequency dynamic tracking method, an automatic robot, a device, and a storage medium based on multiple sensors. Background Art

[0002] High-frequency motion tracking systems are widely used in industrial automation, robotic navigation, motion analysis, and other fields. Traditional motion tracking technologies typically rely on fixed cameras or lidar, but these methods suffer from response delays and insufficient accuracy in high-speed motion scenarios. Therefore, improving the accuracy of motion tracking remains an unresolved issue.

[0003] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a high-frequency dynamic tracking method, automatic robot, equipment and storage medium based on multiple sensors, aiming to solve the technical problem of how to improve the accuracy of dynamic tracking.

[0005] To achieve the above objectives, the present application proposes a high-frequency dynamic tracking method based on multiple sensors, which is applied to an automatic robot comprising: a base, a robotic arm, a first sensor, and a second sensor, wherein the method comprises:

[0006] acquiring a distance between the base and the target according to the first sensor, and controlling the automatic robot to move to a target position according to the distance;

[0007] When the automatic robot reaches the target position, the initial coordinates of the target point are obtained according to the second sensor, and the initial coordinates are converted to obtain the target coordinates of the target point;

[0008] The robotic arm is controlled to track the target point according to the target coordinates.

[0009] In one embodiment, the step of controlling the automatic robot to move to a target position according to the distance comprises:

[0010] When the distance is greater than the preset target distance, the moving speed of the automatic robot is increased, and the distance between the base and the target is shortened until the automatic robot reaches the target position;

[0011] When the distance is less than the preset target distance, the travel speed of the automatic robot is reduced and the distance between the base and the target is increased until the automatic robot reaches the target position.

[0012] In one embodiment, the step of performing coordinate conversion on the initial coordinates to obtain target coordinates of the target point includes:

[0013] Calculate the hand-eye transformation matrix;

[0014] Coordinate transformation is performed based on the hand-eye transformation matrix and the initial coordinates to obtain the target coordinates of the target point.

[0015] In one embodiment, the step of calculating the hand-eye conversion matrix includes:

[0016] Acquire a first coordinate and a second coordinate of a preset point in a calibration plate coordinate system, wherein the preset point is a point on the calibration plate, and the calibration plate is located at the end of the robotic arm of the automatic robot;

[0017] A first conversion matrix and a second conversion matrix are obtained according to the posture of the automatic robot and preset parameters, wherein the first conversion matrix is ​​a conversion matrix from the robot arm base coordinate system to the robot arm end coordinate system, and the second conversion matrix is ​​a conversion matrix from the calibration plate coordinate system to the camera coordinate system;

[0018] A hand-eye transformation matrix is ​​calculated based on the first coordinates, the second coordinates, the first transformation matrix, and the second transformation matrix.

[0019] In one embodiment, the step of obtaining the first coordinate and the second coordinate of the preset point in the calibration plate coordinate system includes:

[0020] In any position of the robot arm, obtain the first coordinate of the preset point in the calibration plate coordinate system;

[0021] The robot arm is controlled to transform into a new posture, and in the new posture, the second coordinate of the preset point in the calibration plate coordinate system is obtained.

[0022] In one embodiment, the step of obtaining the first transformation matrix and the second transformation matrix according to the posture of the automatic robot and preset parameters includes:

[0023] Obtaining joint angle data of the robot arm according to the posture of the automatic robot;

[0024] Obtain the rotation matrix and translation vector based on the joint angle data and preset parameters;

[0025] A first conversion matrix and a second conversion matrix are obtained according to the rotation matrix and the translation vector.

[0026] In one embodiment, the step of calculating the hand-eye transformation matrix according to the first coordinates, the second coordinates, the first transformation matrix, and the second transformation matrix includes:

[0027] According to the first coordinates, obtain first result coordinates through the first transformation matrix, the second transformation matrix, and the hand-eye transformation matrix in sequence;

[0028] According to the second coordinate, obtain a second result coordinate through the first transformation matrix, the second transformation matrix, and the hand-eye transformation matrix in sequence;

[0029] A hand-eye conversion matrix is ​​calculated based on the equality of the first result coordinates and the second result coordinates.

[0030] In addition, to achieve the above-mentioned purpose, the present application also proposes a multi-sensor based high-frequency dynamic tracking automatic robot, the multi-sensor based high-frequency dynamic tracking automatic robot comprising: a base, a robotic arm, a first sensor and a second sensor, the automatic robot comprising:

[0031] an accompanying module, configured to obtain a distance between the base and a target according to the first sensor, and control the automatic robot to move to a target position according to the distance;

[0032] a conversion module, configured to, when the automatic robot reaches the target position, obtain the initial coordinates of the target point according to the second sensor, and perform coordinate conversion on the initial coordinates to obtain the target coordinates of the target point;

[0033] A tracking module is used to control the robotic arm to track the target point according to the target coordinates.

[0034] In addition, to achieve the above-mentioned purpose, the present application also proposes a high-frequency dynamic tracking device based on multiple sensors, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program is configured to implement the steps of the high-frequency dynamic tracking method based on multiple sensors as described above.

[0035] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the high-frequency dynamic tracking method based on multiple sensors as described above are implemented.

[0036] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the multi-sensor based high-frequency dynamic tracking method as described above.

[0037] The present application provides a high-frequency dynamic tracking method based on multiple sensors. The present application obtains the distance between the base and the target according to the first sensor, and controls the automatic robot to move to the target position according to the distance; when the automatic robot reaches the target position, the initial coordinates of the target point are obtained according to the second sensor, and the initial coordinates are converted to obtain the target coordinates of the target point; the robotic arm is controlled to track the target point according to the target coordinates.

[0038] In summary, the present application improves the accuracy of dynamic tracking by using multiple sensor data to track target points and performing coordinate conversion on the target points. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0041] Figure 1 A flowchart of the first embodiment of the high-frequency dynamic tracking method based on multiple sensors of the present application is provided;

[0042] Figure 2 Schematic diagram of an automatic robot provided in Example 1 of the multi-sensor-based high-frequency dynamic tracking method of this application;

[0043] Figure 3 Schematic diagram of an automatic robot tracking a target provided in Example 1 of the multi-sensor-based high-frequency dynamic tracking method of this application;

[0044] Figure 4 A flowchart of the second embodiment of the high-frequency dynamic tracking method based on multiple sensors of the present application is provided;

[0045] Figure 5 A schematic diagram of coordinate conversion provided for the second embodiment of the high-frequency dynamic tracking method based on multiple sensors of this application;

[0046] Figure 6 This is a schematic diagram of the module structure of a multi-sensor high-frequency dynamic tracking automatic robot according to an embodiment of the present application;

[0047] Figure 7 Schematic diagram of the device structure of the hardware operating environment involved in the high-frequency dynamic tracking method based on multiple sensors in the embodiment of the present application.

[0048] The purpose, features and advantages of this application will be further explained with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0050] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0051] The main solution of this application is to obtain the distance between the base and the target based on the first sensor, and control the automatic robot to move to the target position based on the distance; when the automatic robot reaches the target position, the initial coordinates of the target point are obtained based on the second sensor, and the initial coordinates are converted to obtain the target coordinates of the target point; according to the target coordinates, the robotic arm is controlled to track the target point.

[0052] Currently, high-frequency motion tracking systems are widely used in industrial automation, robotic navigation, motion analysis, and other fields. Traditional motion tracking technologies typically rely on fixed cameras or lidar, but these methods suffer from response delays and insufficient accuracy in high-speed motion scenarios. Therefore, improving the accuracy of motion tracking remains an unresolved issue.

[0053] This application improves the accuracy of dynamic tracking by using multiple sensor data to track target points and performing coordinate conversion on the target points.

[0054] Based on this, the embodiment of the present application provides a high-frequency dynamic tracking method based on multiple sensors, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the high-frequency dynamic tracking method based on multiple sensors of the present application.

[0055] In this embodiment, the multi-sensor based high-frequency dynamic tracking method includes steps S10 to S30:

[0056] Step S10: obtaining the distance between the base and the target according to the first sensor, and controlling the automatic robot to move to the target position according to the distance;

[0057] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or an electronic device capable of performing the above functions, a multi-sensor high-frequency motion tracking device, etc. The following uses a multi-sensor high-frequency motion tracking device as an example to illustrate this embodiment and the following embodiments.

[0058] It should be noted that an automatic robot is a robot that can move on its own and work through a robotic arm. Figure 2 , Figure 2 This is a schematic diagram of an autonomous robot. The two robotic arms in the figure correspond to the robotic arms of the autonomous robot. A camera module includes a traveling camera and a hole-finding camera. The traveling camera corresponds to the first sensor, and the hole-finding camera corresponds to the second sensor. The battery and wireless charging receiver are installed inside the autonomous robot for charging and storing power. Four universal wheels and two differential wheels are installed on the base of the autonomous robot to support the rapid movement of the autonomous robot. The first sensor can be the traveling camera, which is used to track the target so that the relative distance between the autonomous robot and the target remains unchanged. Figure 3 , Figure 3 This diagram shows an autonomous robot tracking a target. The robot includes a companion camera, which corresponds to the robot's first sensor and is responsible for tracking the target on the signal. The target position is the relative position of the autonomous robot and the target. The two robotic arms correspond to the robot's mechanical arms, and the hole-finding camera corresponds to the robot's second sensor. Therefore, when the target moves, the autonomous robot is controlled to move, maintaining a constant relative distance from the target.

[0059] In a feasible manner, the step of controlling the automatic robot to move to a target position according to the distance includes:

[0060] When the distance is greater than the preset target distance, the moving speed of the automatic robot is increased, and the distance between the base and the target is shortened until the automatic robot reaches the target position;

[0061] When the distance is less than the preset target distance, the travel speed of the automatic robot is reduced and the distance between the base and the target is increased until the automatic robot reaches the target position.

[0062] It is understandable that the accompanying camera recognizes the accompanying target in real time. The accompanying target is set at the bottom edge of the automobile reporting fixture and is at the same height as the accompanying camera to ensure the consistency of its recognition position. Since the reporting fixtures of the automobile production line are affected by uncertain working conditions such as production rhythm, on-site line stoppage, emergency line stoppage and emergency re-line, it is necessary to ensure that the accompanying camera recognizes the accompanying target in real time so that it is in a relatively static state with the automobile reporting fixture. When the target moves, the distance between the accompanying camera and the target will change. Therefore, it is necessary to judge the relationship between the current distance to the target and the preset distance. If the current distance is greater than the preset target distance, it is necessary to increase the travel speed of the automatic robot and shorten the distance between the base and the target so that the automatic robot can move to the target position. If the current distance is less than the preset target distance, it is necessary to reduce the travel speed of the automatic robot and increase the distance between the base and the target so that the automatic robot can move to the target position.

[0063] Step S20: When the automatic robot reaches the target position, the initial coordinates of the target point are acquired according to the second sensor, and the initial coordinates are converted to obtain the target coordinates of the target point;

[0064] It should be noted that the second sensor can be a hole-finding camera, and the target point can be a screw hole. Figure 3 , Figure 3 The center hole camera is the robot's secondary sensor, used to track screw hole locations on vehicles. After obtaining the initial coordinates of the screw hole locations, these coordinates are not in the robot's coordinate system. Therefore, the screw hole coordinates must be converted to the robot's coordinate system to facilitate tracking.

[0065] Step S30: controlling the robotic arm to track the target point according to the target coordinates.

[0066] It is understandable that after obtaining the target coordinates, the robotic arm can be controlled to accurately track the target point in order to complete the task.

[0067] This embodiment obtains the distance between the base and the target based on the first sensor, and controls the automatic robot to move to the target position based on the distance; when the automatic robot reaches the target position, the initial coordinates of the target point are obtained based on the second sensor, and the initial coordinates are converted to obtain the target coordinates of the target point; according to the target coordinates, the robotic arm is controlled to track the target point.

[0068] This application improves the accuracy of dynamic tracking by using multiple sensor data to track target points and performing coordinate conversion on the target points.

[0069] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 4 , step S20 further includes steps S201 to S202:

[0070] Step S201: Calculate the hand-eye conversion matrix;

[0071] It should be noted that the hand-eye conversion matrix is ​​a matrix obtained by calculation that can convert coordinates in other coordinate systems into the coordinate system of the end of the robotic arm.

[0072] In one feasible manner, the step of calculating the hand-eye conversion matrix includes:

[0073] Acquire a first coordinate and a second coordinate of a preset point in a calibration plate coordinate system, wherein the preset point is a point on the calibration plate, and the calibration plate is located at the end of the robotic arm of the automatic robot;

[0074] A first conversion matrix and a second conversion matrix are obtained according to the posture of the automatic robot and preset parameters, wherein the first conversion matrix is ​​a conversion matrix from the robot arm base coordinate system to the robot arm end coordinate system, and the second conversion matrix is ​​a conversion matrix from the calibration plate coordinate system to the camera coordinate system;

[0075] A hand-eye transformation matrix is ​​calculated based on the first coordinates, the second coordinates, the first transformation matrix, and the second transformation matrix.

[0076] It should be noted that in this embodiment, there is a camera coordinate system O c , calibration plate coordinate system O w , the robot arm end coordinate system O e , robot arm base coordinate system O b , you can refer to Figure 5 , Figure 5 This is a diagram of coordinate conversion. In the figure, cal.object is the calibration plate. The points on the calibration plate are the points that the robot needs to track. camera is the camera, which is a sensor used to obtain the position coordinates of the points on the calibration plate. tool is the end of the robot arm, and base is the base of the robot arm. Therefore, the conversion relationship between the calibration plate coordinate system and the camera coordinate system in the figure is The conversion relationship X from the camera coordinate system to the robot arm base coordinate system, and the conversion relationship from the robot arm base coordinate system to the robot arm end coordinate system The transformation relationship X from the camera coordinate system to the robot arm base coordinate system is the hand-eye calibration matrix that needs to be solved.

[0077] In a feasible manner, the step of obtaining the first coordinate and the second coordinate of the preset point in the calibration plate coordinate system includes:

[0078] In any position of the robot arm, obtain the first coordinate of the preset point in the calibration plate coordinate system;

[0079] The robot arm is controlled to transform into a new posture, and in the new posture, the second coordinate of the preset point in the calibration plate coordinate system is obtained.

[0080] In a feasible manner, the step of calculating the hand-eye conversion matrix according to the first coordinate, the second coordinate, the first conversion matrix, and the second conversion matrix includes:

[0081] According to the first coordinates, obtain first result coordinates through the first transformation matrix, the second transformation matrix, and the hand-eye transformation matrix in sequence;

[0082] According to the second coordinate, obtain a second result coordinate through the first transformation matrix, the second transformation matrix, and the hand-eye transformation matrix in sequence;

[0083] A hand-eye conversion matrix is ​​calculated based on the equality of the first result coordinates and the second result coordinates.

[0084] It is understandable that the calibration plate is fixed at the end of the robot arm. In a certain position, the coordinate value of the point on the calibration plate in the calibration plate coordinate system is P1. After the coordinate system conversion relationship is converted, the point on the calibration plate can be converted to the coordinate P3 in the robot arm end coordinate system. The conversion relationship is as follows:

[0085]

[0086] Then the robot arm changes its posture and can obtain another set of formulas with the same form as above, namely:

[0087]

[0088] Convert the above two formulas into the following form:

[0089]

[0090] Further conversion yields:

[0091]

[0092] Thus, the hand-eye calibration matrix can be obtained. According to the posture of the automatic robot and the preset parameters, as well as

[0093] In a feasible manner, the step of obtaining the first conversion matrix and the second conversion matrix according to the posture of the automatic robot and preset parameters includes:

[0094] Obtaining joint angle data of the robot arm according to the posture of the automatic robot;

[0095] Obtain the rotation matrix and translation vector based on the joint angle data and preset parameters;

[0096] A first conversion matrix and a second conversion matrix are obtained according to the rotation matrix and the translation vector.

[0097] It is understandable that the posture of the automatic robot is different, and the resulting transformation matrix is ​​also different. Therefore, the joint angle data of the robot arm can be obtained according to the posture of the automatic robot, and the rotation matrix and translation vector can be obtained together with the preset parameters to obtain the first transformation matrix and the second transformation matrix.

[0098] Step S202: performing coordinate transformation based on the hand-eye transformation matrix and the initial coordinates to obtain target coordinates of the target point.

[0099] It can be understood that after obtaining the hand-eye conversion matrix, the initial coordinates are multiplied by the hand-eye conversion matrix according to the matrix multiplication rule to obtain the target coordinates of the target point.

[0100] This embodiment calculates a hand-eye conversion matrix; coordinate conversion is performed based on the hand-eye conversion matrix and the initial coordinates to obtain the target coordinates of the target point. By calculating the hand-eye conversion matrix, this embodiment can accurately obtain the position of the target point, thereby enabling the robotic arm to accurately track the target point, improving the accuracy of dynamic tracking.

[0101] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the high-frequency dynamic tracking method based on multiple sensors of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0102] This application also provides a high-frequency dynamic tracking automatic robot based on multiple sensors, please refer to Figure 6 The multi-sensor based high-frequency dynamic tracking automatic robot includes: a base, a robotic arm, a first sensor and a second sensor, and the automatic robot includes:

[0103] The accompanying module 10 is used to obtain the distance between the base and the target according to the first sensor, and control the automatic robot to move to the target position according to the distance;

[0104] a conversion module 20, configured to obtain the initial coordinates of the target point according to the second sensor when the automatic robot reaches the target position, and perform coordinate conversion on the initial coordinates to obtain the target coordinates of the target point;

[0105] The tracking module 30 is used to control the robotic arm to track the target point according to the target coordinates.

[0106] This embodiment obtains the distance between the base and the target based on the first sensor, and controls the automatic robot to move to the target position based on the distance; when the automatic robot reaches the target position, the initial coordinates of the target point are obtained based on the second sensor, and the initial coordinates are converted to obtain the target coordinates of the target point; according to the target coordinates, the robotic arm is controlled to track the target point.

[0107] This application improves the accuracy of dynamic tracking by using multiple sensor data to track target points and performing coordinate conversion on the target points.

[0108] In one embodiment, the accompanying module 10 is also used to increase the travel speed of the automatic robot and shorten the distance between the base and the target when the distance is greater than the preset target distance, until the automatic robot reaches the target position; when the distance is less than the preset target distance, reduce the travel speed of the automatic robot and increase the distance between the base and the target until the automatic robot reaches the target position.

[0109] In one embodiment, the conversion module 20 is further configured to calculate a hand-eye conversion matrix; and perform coordinate conversion based on the hand-eye conversion matrix and the initial coordinates to obtain target coordinates of the target point.

[0110] In one embodiment, the conversion module 20 is also used to obtain the first coordinate and the second coordinate of a preset point in the calibration plate coordinate system, where the preset point is a point on the calibration plate, and the calibration plate is located at the end of the robotic arm of the automatic robot; the first conversion matrix and the second conversion matrix are obtained according to the posture of the automatic robot and the preset parameters, the first conversion matrix is ​​the conversion matrix from the robotic arm base coordinate system to the robotic arm end coordinate system, and the second conversion matrix is ​​the conversion matrix from the calibration plate coordinate system to the camera coordinate system; the hand-eye conversion matrix is ​​calculated according to the first coordinate, the second coordinate, the first conversion matrix and the second conversion matrix.

[0111] In one embodiment, the conversion module 20 is also used to obtain the first coordinate of the preset point in the calibration plate coordinate system at any posture of the robotic arm; control the robotic arm to transform into a new posture, and obtain the second coordinate of the preset point in the calibration plate coordinate system at the new posture.

[0112] In one embodiment, the conversion module 20 is also used to obtain the joint angle data of the robot arm according to the posture of the automatic robot; obtain the rotation matrix and translation vector according to the joint angle data and preset parameters; and obtain the first conversion matrix and the second conversion matrix according to the rotation matrix and the translation vector.

[0113] In one embodiment, the conversion module 20 is further used to obtain a first result coordinate according to the first coordinate, in sequence through a first conversion matrix, a second conversion matrix, and a hand-eye conversion matrix; to obtain a second result coordinate according to the second coordinate, in sequence through a first conversion matrix, a second conversion matrix, and a hand-eye conversion matrix; and to obtain a hand-eye conversion matrix based on the equality of the first result coordinate and the second result coordinate.

[0114] The multi-sensor-based high-frequency dynamic tracking device provided in this application, which employs the multi-sensor-based high-frequency dynamic tracking method in the above-mentioned embodiment, can solve the technical problem of how to improve the accuracy of dynamic tracking. Compared with the prior art, the beneficial effects of the multi-sensor-based high-frequency dynamic tracking device provided in this application are the same as those of the multi-sensor-based high-frequency dynamic tracking method provided in the above-mentioned embodiment, and the other technical features of the multi-sensor-based high-frequency dynamic tracking device are the same as those disclosed in the above-mentioned embodiment method, and are not further described here.

[0115] The present application provides a high-frequency dynamic tracking device based on multiple sensors, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the high-frequency dynamic tracking method based on multiple sensors in the above-mentioned embodiment one.

[0116] Reference below Figure 7 , which shows a schematic diagram of the structure of a multi-sensor based high-frequency dynamic tracking device suitable for implementing the embodiments of the present application. The multi-sensor based high-frequency dynamic tracking device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7 The multi-sensor based high-frequency dynamic tracking device shown is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present application.

[0117] like Figure 7 As shown, the high-frequency dynamic tracking device based on multiple sensors may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the ROM (Read Only Memory) 1002 or the program loaded from the storage device 1003 to the RAM (Random Access Memory) 1004. Various programs and data required for the operation of the high-frequency dynamic tracking device based on multiple sensors are also stored in the RAM 1004. The processing device 1001, the ROM 1002 and the RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the multi-sensor-based high-frequency dynamic tracking device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a multi-sensor-based high-frequency dynamic tracking device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or have instead.

[0118] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0119] The multi-sensor-based high-frequency dynamic tracking device provided in this application, which employs the multi-sensor-based high-frequency dynamic tracking method of the above-mentioned embodiment, can solve the technical problem of how to improve the accuracy of dynamic tracking. Compared with the prior art, the beneficial effects of the multi-sensor-based high-frequency dynamic tracking device provided in this application are the same as the beneficial effects of the multi-sensor-based high-frequency dynamic tracking method provided in the above-mentioned embodiment, and the other technical features of the multi-sensor-based high-frequency dynamic tracking device are the same as those disclosed in the method of the above-mentioned embodiment, and are not further described here.

[0120] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0121] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0122] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, wherein the computer-readable program instructions are used to execute the multi-sensor-based high-frequency dynamic tracking method in the above-mentioned embodiment.

[0123] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0124] The computer-readable storage medium may be included in the multi-sensor-based high-frequency dynamic tracking device; or may exist independently without being assembled into the multi-sensor-based high-frequency dynamic tracking device.

[0125] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the high-frequency dynamic tracking device based on multiple sensors, the high-frequency dynamic tracking device based on multiple sensors: obtains the distance between the base and the target according to the first sensor, and controls the automatic robot to move to the target position according to the distance; when the automatic robot reaches the target position, obtains the initial coordinates of the target point according to the second sensor, and performs coordinate conversion on the initial coordinates to obtain the target coordinates of the target point; and controls the robotic arm to track the target point according to the target coordinates.

[0126] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0127] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0128] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0129] The computer-readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned multi-sensor-based high-frequency dynamic tracking method, thereby solving the technical problem of improving the accuracy of dynamic tracking. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the multi-sensor-based high-frequency dynamic tracking method provided in the aforementioned embodiment, and are not further elaborated here.

[0130] The present application also provides a computer program product, comprising a computer program, which implements the steps of the above-mentioned multi-sensor-based high-frequency dynamic tracking method when executed by a processor.

[0131] The computer program product provided in this application can solve the technical problem of how to improve the accuracy of dynamic tracking. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the multi-sensor high-frequency dynamic tracking method provided in the above embodiment, and will not be repeated here.

[0132] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A high-frequency dynamic tracking method based on multiple sensors, characterized in that: The method is applied to an automatic robot, which includes a base, a robotic arm, a first sensor, and a second sensor. The method includes: acquiring a distance between the base and the target according to the first sensor, and controlling the automatic robot to move to a target position according to the distance; When the automatic robot reaches the target position, the initial coordinates of the target point are obtained according to the second sensor, and the initial coordinates are converted to obtain the target coordinates of the target point; The robotic arm is controlled to track the target point according to the target coordinates.

2. The method according to claim 1, wherein The step of controlling the automatic robot to move to a target position according to the distance comprises: When the distance is greater than the preset target distance, the moving speed of the automatic robot is increased, and the distance between the base and the target is shortened until the automatic robot reaches the target position; When the distance is less than the preset target distance, the travel speed of the automatic robot is reduced and the distance between the base and the target is increased until the automatic robot reaches the target position.

3. The method according to claim 1, wherein The step of performing coordinate conversion on the initial coordinates to obtain the target coordinates of the target point comprises: Calculate the hand-eye transformation matrix; Coordinate transformation is performed based on the hand-eye transformation matrix and the initial coordinates to obtain the target coordinates of the target point.

4. The method according to claim 3, wherein The step of calculating the hand-eye conversion matrix comprises: Acquire a first coordinate and a second coordinate of a preset point in a calibration plate coordinate system, wherein the preset point is a point on the calibration plate, and the calibration plate is located at the end of the robotic arm of the automatic robot; A first conversion matrix and a second conversion matrix are obtained according to the posture of the automatic robot and preset parameters, wherein the first conversion matrix is ​​a conversion matrix from the robot arm base coordinate system to the robot arm end coordinate system, and the second conversion matrix is ​​a conversion matrix from the calibration plate coordinate system to the camera coordinate system; A hand-eye transformation matrix is ​​calculated based on the first coordinates, the second coordinates, the first transformation matrix, and the second transformation matrix.

5. The method according to claim 4, wherein The step of obtaining the first coordinate and the second coordinate of the preset point in the calibration plate coordinate system includes: In any position of the robot arm, obtain the first coordinate of the preset point in the calibration plate coordinate system; The robot arm is controlled to transform into a new posture, and in the new posture, the second coordinate of the preset point in the calibration plate coordinate system is obtained.

6. The method according to claim 4, wherein The step of obtaining the first conversion matrix and the second conversion matrix according to the posture of the automatic robot and the preset parameters includes: Obtaining joint angle data of the robot arm according to the posture of the automatic robot; Obtain the rotation matrix and translation vector based on the joint angle data and preset parameters; A first conversion matrix and a second conversion matrix are obtained according to the rotation matrix and the translation vector.

7. The method according to claim 4, wherein The step of calculating the hand-eye conversion matrix according to the first coordinates, the second coordinates, the first conversion matrix, and the second conversion matrix includes: According to the first coordinates, obtain first result coordinates through the first transformation matrix, the second transformation matrix, and the hand-eye transformation matrix in sequence; According to the second coordinate, obtain a second result coordinate through the first transformation matrix, the second transformation matrix, and the hand-eye transformation matrix in sequence; A hand-eye conversion matrix is ​​calculated based on the equality of the first result coordinates and the second result coordinates.

8. A high-frequency dynamic tracking automatic robot based on multiple sensors, characterized in that: The automatic robot comprises: a base, a robotic arm, a first sensor and a second sensor, and the automatic robot comprises: an accompanying module, configured to obtain a distance between the base and a target according to the first sensor, and control the automatic robot to move to a target position according to the distance; a conversion module, configured to, when the automatic robot reaches the target position, obtain the initial coordinates of the target point according to the second sensor, and perform coordinate conversion on the initial coordinates to obtain the target coordinates of the target point; A tracking module is used to control the robotic arm to track the target point according to the target coordinates.

9. A high-frequency dynamic tracking device based on multiple sensors, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the multi-sensor based high-frequency dynamic tracking method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the high-frequency dynamic tracking method based on multiple sensors according to any one of claims 1 to 7 is implemented.

Images