Conversion relationship calibration method and device, equipment, storage medium and program product

Abstract

This application provides a method, apparatus, device, storage medium, and program product for calibrating transformation relationships. Embodiments of this application can be applied to various scenarios such as cloud technology, artificial intelligence, smart transportation, and vehicle-mounted systems. The method includes: determining an initial transformation relationship between a visual coordinate system and a control coordinate system using a first pose parameter of an image acquisition device mounted on a moving object and a second pose parameter of the moving object's centroid; controlling the moving object to move and recording a first motion trajectory of the moving object in the visual coordinate system; transforming the first motion trajectory using the initial transformation relationship to obtain a second motion trajectory in the control coordinate system; determining compensation parameters for the initial transformation relationship by registering the trajectory points of the first and second motion trajectories; and performing error compensation on the initial transformation relationship based on the compensation parameters to obtain a target transformation relationship. This application improves the accuracy of transformation relationship calibration.

Description

Technical Field

[0001] This application relates to automatic control technology, and more particularly to a method, apparatus, equipment, storage medium, and program product for calibrating conversion relationships. Background Technology

[0002] Transformation calibration is the process of determining the transformation relationship between the visual coordinate system and the control coordinate system of moving objects such as robots and autonomous vehicles. This enables the control system of the moving object to issue correct control commands based on the image data captured by the moving object. In related technologies, the accuracy of transformation calibration between the visual and control coordinate systems is relatively low. Summary of the Invention

[0003] This application provides a method, apparatus, device, computer-readable storage medium, and program product for calibrating conversion relationships, which can improve the accuracy of conversion relationship calibration.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides a method for identifying conversion relationships, including: Using the first pose parameter of the image acquisition device set on the moving object and the second pose parameter of the centroid of the moving object, the initial transformation relationship between the visual coordinate system and the control coordinate system of the moving object is determined; Control the moving object to move, and record the first motion trajectory of the moving object in the visual coordinate system; The first motion trajectory is transformed using the initial transformation relationship to obtain the second motion trajectory of the moving object in the control coordinate system; By registering the trajectory points of the first motion trajectory and the second motion trajectory, compensation parameters are determined for the initial transformation relationship; Based on the compensation parameters, error compensation is performed on the initial transformation relationship to obtain the target transformation relationship between the visual coordinate system and the control coordinate system.

[0005] This application provides a conversion relationship calibration device, including: The relationship determination module is used to determine the initial transformation relationship between the visual coordinate system and the control coordinate system of the moving object by using the first pose parameter of the image acquisition device set on the moving object and the second pose parameter of the centroid of the moving object. A motion control module is used to control the motion object to perform its motion; A trajectory recording module is used to record the first motion trajectory of the moving object in the visual coordinate system; The trajectory conversion module is used to convert the first motion trajectory using the initial conversion relationship to obtain the second motion trajectory of the moving object in the control coordinate system; The parameter determination module is used to determine compensation parameters for the initial transformation relationship by registering the trajectory points of the first motion trajectory and the second motion trajectory. The relationship determination module is further configured to perform error compensation on the initial transformation relationship based on the compensation parameters, so as to obtain the target transformation relationship between the visual coordinate system and the control coordinate system.

[0006] In some embodiments of this application, the parameter determination module is further configured to perform coordinate system alignment processing on the first motion trajectory and the second motion trajectory to obtain a first trajectory to be matched and a second trajectory to be matched. For each trajectory point in the first trajectory to be matched, a corresponding matching trajectory point is determined from the second trajectory to be matched; and a compensation parameter is determined based on the coordinates of each trajectory point in the first trajectory to be matched and the coordinates of the matching trajectory point.

[0007] In some embodiments of this application, the parameter determination module is further configured to: acquire first time information of each trajectory point of the first trajectory to be matched, and second time information of each trajectory point of the second trajectory to be matched; determine the second time information whose difference from the first time information is less than a time threshold as candidate time information; filter the trajectory points in the second trajectory to be matched that correspond to the candidate time information to obtain candidate trajectory points; and filter the candidate trajectory points to obtain the matching trajectory points for each trajectory point in the first trajectory to be matched.

[0008] In some embodiments of this application, the parameter determination module is further configured to perform distance calculation for each trajectory point in the first trajectory to be matched and the candidate trajectory point to obtain the trajectory distance corresponding to the candidate trajectory point; and determine the candidate trajectory point corresponding to the smallest trajectory distance as the matching trajectory point of each trajectory point in the first trajectory to be matched.

[0009] In some embodiments of this application, the parameter determination module is further configured to: after aligning the first motion trajectory and the second motion trajectory to obtain a first trajectory to be matched and a second trajectory to be matched, calculate a first trajectory center for the first trajectory to be matched and calculate a second trajectory center for the second trajectory to be matched; update the first trajectory to be matched using the first trajectory center to obtain an updated first trajectory, and update the second trajectory to be matched using the second trajectory center to obtain an updated second trajectory; for each updated trajectory point in the updated first trajectory, determine an updated matching trajectory point from the updated second trajectory; and determine the compensation parameter based on the coordinates of each updated trajectory point and the coordinates of the updated matching trajectory point.

[0010] In some embodiments of this application, the parameter determination module is further configured to determine the rotation angle and translation distance between the visual coordinate system and the control coordinate system; adjust the orientation of the first motion trajectory according to the rotation angle and the translation distance to obtain the first trajectory to be matched, and determine the second motion trajectory as the second trajectory to be matched; or, determine the first motion trajectory as the first trajectory to be matched, and adjust the orientation of the second motion trajectory according to the rotation angle and the translation distance to obtain the second trajectory to be matched.

[0011] In some embodiments of this application, the parameter determination module is further configured to determine the orientation of the starting point of the first motion trajectory and the included angle of the first coordinate axis of the visual coordinate system before determining the rotation angle and translation distance between the visual coordinate system and the control coordinate system, thereby obtaining a trajectory included angle; the first coordinate axis is a coordinate axis parallel to the horizontal plane in the motion scene; the second motion trajectory is rotated around the second coordinate axis of the control coordinate system by the trajectory included angle to obtain an optimized second motion trajectory; the second coordinate axis is a coordinate axis perpendicular to the horizontal plane in the motion scene; the optimized second motion trajectory is determined as the second trajectory to be matched; based on the rotation angle and the translation distance, the orientation of the optimized second motion trajectory is adjusted to obtain the second trajectory to be matched.

[0012] In some embodiments of this application, the relationship determination module is further configured to decompose the initial rotation and translation matrix into an initial translation vector and an initial rotation matrix; update the initial rotation matrix using the compensation matrix to obtain an updated rotation matrix; integrate the initial translation vector and the updated rotation matrix into an updated rotation and translation matrix, and determine the updated rotation and translation matrix as the target transformation relationship.

[0013] In some embodiments of this application, the parameter determination module is further configured to perform an i-th singular value decomposition on the coordinates of each trajectory point in the first trajectory to be matched and the coordinates of the matching trajectory point to obtain an i-th rotation matrix and an i-th translation vector. S is the total number of iterations; using the i-th rotation matrix and the i-th translation vector, each trajectory point in the first trajectory to be matched is updated to obtain the i-th updated trajectory point; when the difference between the distance between the i-th updated trajectory point and the matching trajectory point and the distance between the (i-1)-th updated trajectory point and the matching trajectory point is less than the difference threshold, the cumulative result from the first rotation matrix to the i-th rotation matrix is ​​determined as the compensation parameter.

[0014] In some embodiments of this application, the trajectory recording module is further configured to record a third motion trajectory of the moving object in the control coordinate system; The relationship determination module is further configured to determine the target transformation relationship between the visual coordinate system and the control coordinate system based on the alignment of the first motion trajectory and the third motion trajectory.

[0015] In some embodiments of this application, the motion control module is further configured to control the moving object to perform translational motion.

[0016] This application provides an electronic device, including: Memory, used to store executable instructions; The processor, when executing executable instructions stored in the memory, implements the conversion relationship calibration method provided in the embodiments of this application.

[0017] This application provides a computer-readable storage medium storing executable instructions for inducing a processor to execute and implement the conversion relationship calibration method provided in this application.

[0018] This application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the conversion relationship calibration method provided in this application. The embodiments of this application have the following beneficial effects: The electronic device first performs a coarse calibration of the transformation relationship between the visual coordinate system and the control coordinate system based on the first pose parameter and the second pose parameter to obtain the initial transformation relationship. Then, it controls the moving object to generate a first motion trajectory in the visual coordinate system, and combines the initial transformation relationship to transform the first motion trajectory into a second motion trajectory in the control coordinate system. Finally, it registers the trajectory points of the first motion trajectory and the trajectory points of the second motion trajectory. Based on the registered trajectory points, it determines the compensation parameter for the initial transformation relationship, and uses the compensation parameter to eliminate the error of the initial transformation relationship to obtain a more accurate target transformation relationship. In this way, the accuracy of the transformation relationship calibration is improved. Attached Figure Description

[0019] Figure 1 This is a schematic diagram illustrating the transformation relationship between the robot's visual coordinate system and control coordinate system, determined based on hand-eye calibration. Figure 2 This is a schematic diagram of a calibration frame; Figure 3 This is a schematic diagram of the architecture of the conversion relationship calibration system provided in the embodiments of this application; Figure 4 This is provided by the embodiments of this application. Figure 3 A schematic diagram of the terminal structure in the diagram; Figure 5 This is a flowchart illustrating a conversion relationship calibration method provided in an embodiment of this application; Figure 6 This is a comparative schematic diagram of the first and second motion trajectories provided in the embodiments of this application; Figure 7 This is another flowchart illustrating the conversion relationship calibration method provided in the embodiments of this application; Figure 8 This is another flowchart illustrating the conversion relationship calibration method provided in the embodiments of this application; Figure 9A This is a schematic diagram of a quadruped robot provided in an embodiment of this application; Figure 9B This is another schematic diagram of the quadruped robot provided in the embodiments of this application; Figure 10 This is a schematic diagram of the coordinate system dependency provided in the embodiments of this application; Figure 11 This is a schematic diagram illustrating the error between the center of mass trajectory and the camera trajectory of the quadruped robot during its movement, as provided in an embodiment of this application. Figure 12A This is a schematic diagram comparing the centroid trajectory and the camera trajectory obtained based on the modified transformation relationship provided in an embodiment of this application; Figure 12BThis is another comparative schematic diagram of the centroid trajectory and the camera trajectory obtained based on the modified transformation relationship provided in the embodiments of this application; Figure 13 This is a schematic diagram of the point cloud registration process provided in the embodiments of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0022] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0024] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0025] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0026] 1) Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0027] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0028] 2) Computer Vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing and measuring targets, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0029] 3) A robot is an intelligent machine capable of semi-autonomous or fully autonomous operation. Robots possess basic characteristics such as perception, decision-making, and execution, and can assist or even replace humans in completing heavy and complex tasks.

[0030] When a robot works fully autonomously, it usually needs to use image acquisition devices (such as cameras) to perceive the external environment. The control system generates control commands based on the external environment to control the robot's movement.

[0031] 4) The coordinate system transformation relationship is the parameter that aligns the robot's visual coordinate system and control coordinate system. The robot's visual coordinate system and control coordinate system usually move together with the robot. Therefore, before the robot is put into use, the transformation relationship between the visual coordinate system and the control coordinate system needs to be calibrated. In this way, during the operation of the machine, the control system can align the visual coordinate system and the control coordinate system through the transformation relationship to issue correct commands.

[0032] The transformation relationship between coordinate systems can be represented by rotation and translation matrices. These matrices can be decomposed into a rotation matrix and a translation vector.

[0033] 5) Point cloud registration refers to the process of merging point clouds collected from different perspectives into a unified perspective to form a complete point cloud dataset. Point cloud registration can be achieved using algorithms such as ICP (Iterative Closer Point) and NDT (Normal Distribution Transformation).

[0034] 6) SVD (Singular Value Decomposition) is used to decompose the coordinates of two points to obtain rotation and translation matrices.

[0035] 7) Calibration object refers to a planar object with obvious characteristics during the calibration process, such as a calibration board or a checkerboard. Knowing its specific size and position, the camera's extrinsic parameters can be calculated based on the image's pixel coordinates and the camera's intrinsic parameters.

[0036] 8) Registration of trajectory points refers to the process of searching for the trajectory point that is closest to a trajectory point in another motion trajectory, given a trajectory point in a certain motion trajectory.

[0037] Transformation calibration is the process of determining the transformation relationship between the visual coordinate system and the control coordinate system of moving objects such as robots and autonomous vehicles, so that the control system of the moving object can issue correct control commands based on the image data captured by the moving object.

[0038] In related technologies, there are two main methods for determining the conversion relationship: one is based on hand-eye calibration to determine the conversion relationship, and the other is based on calibration frame calibration to determine the conversion relationship.

[0039] Figure 1This is a schematic diagram illustrating the transformation relationship between the robot's visual coordinate system and control coordinate system based on hand-eye calibration. During hand-eye calibration, the robot body 1-1 and camera 1-2 are first fixed in designated positions, and calibration object 1-3 is fixed to the robot's end effector 1-4. Then, the robot is controlled to perform a specified action, and the pose of the end effector 1-4 and the corresponding image are recorded. Next, the coordinate transformation relationship between camera 1-2 and calibration object 1-3 is calculated using the calibrated camera intrinsic parameters and projection principles (which can be represented as...). ), and the known poses of the ends 1-4 (which can represent The relationship between the fixed end and the calibration plate Constraint relationships can be constructed. The constraint relationship can be shown in equation (1):

[0040] in, To represent a constant, one can... By using two or more constraint relationships, it is possible to define... That is, the transformation relationship between the visual coordinate system and the control coordinate system. The more constraint relationships there are, the better the calibration results. The more precise it is, the better.

[0041] Figure 2 This is a schematic diagram of a calibration frame. See also... Figure 2 On calibration frame 2-1, there are many calibration plates of known size and relative positions. Using bundle adjustment, the transformation between the vision coordinate system and the control coordinate system can be calibrated. The calibration steps are as follows: The robot body is fixed at a certain position on calibration frame 2-1, and the position and image of each calibration plate are recorded; the position of the robot body is changed, and the position and image of each calibration plate are recorded again; the transformation relationship between the known calibration plates and the robot is then established. The pixel coordinates are calculated using reprojection, and a loss function is constructed based on the difference between the reprojection and the actual pixel coordinates. The transformation relationship between the visual coordinate system and the control coordinate system is then solved by optimizing the loss function. The loss function is shown in equation (2).

[0042] in, Let J be the pixel coordinates of the j-th feature point in the i-th measurement. Let be the true coordinates of the j-th feature point, M be the intrinsic parameter matrix of the camera, and R be the extrinsic parameter matrix of the camera. Since each position is in the world coordinate system with the robot's center of mass as the reference, the solved R represents the transformation relationship between the visual coordinate system and the control coordinate system. The process of transforming the calibration board to the world coordinate system is omitted here.

[0043] As can be seen from the above, both hand-eye calibration and calibration frame-based methods for determining the transformation relationship between the robot's visual and control coordinate systems require manual operation by the operator. Manual operation may introduce errors, affecting the accuracy of the transformation relationship calibration. Furthermore, both calibration methods are static, making it difficult to detect minute angular errors during the calibration process, which also impact the accuracy of the transformation relationship calibration. Therefore, related technologies suffer from low accuracy in calibrating the transformation relationship between the visual and control coordinate systems.

[0044] In addition, when calibration is performed manually by staff, the operation steps are cumbersome and the calibration time is longer, which also makes the conversion relationship calibration inefficient.

[0045] This application provides a method, apparatus, device, computer-readable storage medium, and program product for conversion relation calibration, which can improve the accuracy of conversion relation calibration. The following describes exemplary applications of the electronic device for conversion relation calibration provided in this application. The electronic device provided in this application can be implemented as various types of terminals such as laptops, tablets, desktop computers, set-top boxes, and mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), or as a server, or even as a server cluster composed of multiple servers in real time. The following will describe exemplary applications when the electronic device is implemented as a terminal.

[0046] See Figure 3 , Figure 3 This is a schematic diagram of the architecture of the conversion relationship calibration system provided in the embodiments of this application. In order to support a conversion relationship calibration application, in the conversion relationship calibration system 100, the terminal 400 is connected to the moving object 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0047] The terminal 400 uses the first pose parameter of the image acquisition device set on the moving object 200 and the second pose parameter of the centroid of the moving object 200 to determine the initial transformation relationship between the visual coordinate system and the control coordinate system of the moving object 200; controls the moving object 200 to move and records the first motion trajectory of the moving object 200 in the visual coordinate system; transforms the first motion trajectory using the initial transformation relationship to obtain the second motion trajectory of the moving object 200 in the control coordinate system; registers the trajectory points of the first motion trajectory and the second motion trajectory to determine compensation parameters for the initial transformation relationship; performs error compensation on the initial transformation relationship based on the compensation parameters to obtain the target transformation relationship between the visual coordinate system and the control coordinate system, thus completing the transformation relationship calibration process.

[0048] In some embodiments, the terminal 400 may be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, etc., but is not limited thereto. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application embodiment does not impose any restrictions.

[0049] See Figure 4 , Figure 4 This is provided by the embodiments of this application. Figure 3 A schematic diagram of the structure of a terminal (one implementation of an electronic device) in the diagram. Figure 4 The terminal 400 shown includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 4 The general labeled all buses as Bus System 440.

[0050] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0051] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0052] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0053] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0054] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0055] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks; The network communication module 452 is used to reach other computing devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc. Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 associated with user interface 430 (e.g., a display screen, a speaker, etc.). The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0056] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 4 A conversion relationship calibration device 455 stored in memory 450 is shown. This device can be software in the form of programs or plug-ins, and includes the following software modules: a relationship determination module 4551, a motion control module 4552, a trajectory recording module 4553, a trajectory conversion module 4554, and a parameter determination module 4555. These modules are logically connected and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0057] In some embodiments, the terminal or server can implement the conversion relationship calibration method provided in this application by running a computer program. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), i.e., a program that needs to be installed in the operating system to run, such as a robot calibration APP; it can also be a mini-program, i.e., a program that only needs to be downloaded to a browser environment to run; or it can be a mini-program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module, or plugin.

[0058] The embodiments of this application can be applied to various scenarios such as cloud technology, artificial intelligence, smart transportation, and vehicle-mounted systems. Below, the conversion relationship calibration method provided by the embodiments of this application will be described in conjunction with exemplary applications and implementations of the electronic devices provided in the embodiments of this application.

[0059] See Figure 5 , Figure 5 This is a flowchart illustrating a transformation relationship calibration method provided in an embodiment of this application, which will be combined with... Figure 5 The steps shown are explained.

[0060] S101. Using the first pose parameter of the image acquisition device set on the moving object and the second pose parameter of the centroid of the moving object, determine the initial transformation relationship between the visual coordinate system and the control coordinate system of the moving object.

[0061] This embodiment is developed in a scenario where the transformation relationship between the visual coordinate system and the control coordinate system of a moving object is calibrated, so that the control system of the moving object can align the visual coordinate system and the control coordinate system according to the calibrated transformation relationship. The moving object acquires visual information through an image acquisition device mounted on it, such as taking photos or videos of the surrounding environment. After the transformation relationship calibration process begins, the pose of the image acquisition device mounted on the moving object is first measured to obtain the first pose parameter, and the pose of the center of mass of the moving object is measured to obtain the second pose parameter. The electronic device uses the first pose parameter and the second pose parameter to solve for the transformation relationship between the visual coordinate system and the control coordinate system, and determines the transformation relationship obtained from this solution as the initial transformation relationship.

[0062] It is understood that the moving object can be a quadruped robot, a bipedal robot, or even a car with autonomous driving capabilities, etc., and this application embodiment does not limit it. The image acquisition device mounted on the moving object can be any one or more of a monocular camera, a binocular camera, and a fisheye camera. The image acquisition device can be a color camera or an infrared camera, and this application embodiment does not limit it.

[0063] It should be noted that the first pose parameter includes the position and orientation information of the image acquisition device, while the second position parameter includes the position and orientation information of the moving object's center of mass. The position information can be described using coordinate points, and the orientation information can be described using direction angles.

[0064] In this embodiment, the electronic device can obtain the first pose parameter by matching features of an existing calibration object or map, and then solving the camera extrinsic parameters using the photogrammetric principle (e.g., bundle adjustment) of the image acquisition device. The second pose parameter can be obtained by aligning the centroid of the moving object with a specific position in the visual coordinate system, and then locating the centroid.

[0065] It should be noted that due to the placement of calibration objects or the accuracy of the map, the pose measurement of the image acquisition device may contain measurement errors, resulting in inaccurate first pose parameters. Simultaneously, the positioning of the centroid of the moving object also presents errors; for example, the tilt of the moving object may cause three-axis rotation errors, leading to inaccurate second pose parameters. The initial transformation relationship determined based on inaccurate first and second pose parameters is highly likely to deviate from the true transformation relationship between the visual coordinate system and the control coordinate system. In other words, the process of solving the initial transformation relationship in this step is equivalent to performing a coarse calibration of the visual and control coordinate systems.

[0066] In some embodiments, the electronic device can obtain an initial transformation relationship by comparing a first pose parameter and a second pose parameter. In other embodiments, the electronic device can also perform singular value decomposition on the first pose parameter and the second pose parameter to obtain the initial transformation relationship.

[0067] It is understood that the initial transformation relationship can be represented by rotation matrix alone, or by rotation matrix and translation vector together. This application does not limit this.

[0068] S102. Control the moving object to move and record the first motion trajectory of the moving object in the visual coordinate system.

[0069] After determining the initial transformation relationship, the electronic device issues motion commands to the moving object to control its movement within its environment. During the movement, the electronic device uses visual positioning technology to continuously locate the moving object in the visual coordinate system, obtains the object's trajectory points, connects these trajectory points in time sequence to obtain the object's first motion trajectory in the visual coordinate system, and records this first motion trajectory.

[0070] It is understandable that visual positioning technology can be achieved by using n 3D coordinate points of a given scene and the 2D projection coordinates of these coordinate points in the image acquired by the image acquisition device, thereby solving for the position of the scene and the image acquisition device.

[0071] In this embodiment of the application, the electronic device can control the moving object to perform translational movement on the horizontal plane of the scene. For example, it can control the moving object to move in a straight line on the horizontal plane of the scene, or first move in a straight line and then turn, or move in an arc, etc.

[0072] S103. The first motion trajectory is transformed using the initial transformation relationship to obtain the second motion trajectory of the moving object in the control coordinate system.

[0073] The electronic device uses the initial transformation relationship to transform and calculate each trajectory point of the first motion trajectory to obtain the trajectory points in the control coordinate system. By connecting the trajectory points in the control coordinate system according to the time sequence, the second motion trajectory of the moving object in the control coordinate system can be obtained.

[0074] It should be noted that, due to the error between the initial transformation relationship and the actual transformation relationship between the visual coordinate system and the control coordinate system, there will be an error between the second motion trajectory of the moving object in the control coordinate system and the first motion trajectory in the visual coordinate system. Moreover, this error will become more and more significant as the moving object travels further.

[0075] For example, Figure 6 This is a comparative schematic diagram of the first and second motion trajectories provided in the embodiments of this application. From Figure 6 It can be seen that in the same coordinate system (the coordinate system formed by the X-axis, Y-axis, and Z-axis), the first motion trajectory 6-1 and the second motion trajectory 6-2 are basically coincident at the beginning of the motion. However, as the motion distance increases, the error between the first motion trajectory 6-1 and the second motion trajectory 6-2 gradually increases and they can no longer coincide. Therefore, it is evident that there is an error in the initial transformation relationship that needs to be compensated.

[0076] S104. By registering the trajectory points of the first motion trajectory and the second motion trajectory, compensation parameters are determined for the initial transformation relationship.

[0077] The electronic device registers each trajectory point in the first motion trajectory with a trajectory point in the second motion trajectory, obtaining matching point pairs between the first and second motion trajectories. Then, using the coordinates of the trajectory points in the matching point pairs, compensation parameters are determined based on the initial transformation relationship. In essence, registering the trajectory points of the first and second motion trajectories involves searching for the closest trajectory point in the second motion trajectory for each trajectory point in the first trajectory. Thus, the matching point pairs corresponding to each trajectory point in the first trajectory contain the two closest trajectory points in both the first and second motion trajectories.

[0078] In some embodiments, the electronic device can solve for the rotation matrix based on the coordinates of the matching point pairs, and then use the rotation matrix as a compensation parameter to perform error compensation on the initial transformation relationship to obtain the target transformation relationship. In other embodiments, the electronic device can also directly calculate the translation component required to align the matching point pairs, and use the translation component as a compensation parameter to perform error compensation on the initial transformation relationship. This embodiment is not limited to these embodiments.

[0079] S105. Based on the compensation parameters, perform error compensation on the initial transformation relationship to obtain the target transformation relationship between the visual coordinate system and the control coordinate system.

[0080] Finally, the electronic device uses the calculated compensation parameters to update the initial transformation relationship in order to cancel out the error of the initial transformation relationship and achieve error compensation for the initial transformation relationship. The transformation relationship obtained after the update is a more accurate target transformation relationship between the visual coordinate system and the control coordinate system.

[0081] It is understandable that, compared with related technologies, the conversion relationship between the robot's visual coordinate system and control coordinate system determined by hand-eye calibration, and the conversion relationship between the robot's visual coordinate system and control coordinate system determined by calibration frame, both suffer from low accuracy in conversion relationship calibration. In this embodiment, a coarse calibration of the conversion relationship between the visual coordinate system and control coordinate system is first performed based on the first pose parameter and the second pose parameter to obtain an initial conversion relationship. Then, the moving object is controlled to generate a first motion trajectory in the visual coordinate system, and combined with the initial conversion relationship, the first motion trajectory is converted to obtain a second motion trajectory in the control coordinate system. Finally, the trajectory points of the first motion trajectory and the trajectory points of the second motion trajectory are registered. Based on the registered trajectory points, compensation parameters are determined for the initial conversion relationship. The compensation parameters are used to eliminate the error of the initial conversion relationship, resulting in a more accurate target conversion relationship. In this way, the accuracy of the conversion relationship calibration is improved.

[0082] based on Figure 5See Figure 7 , Figure 7 This is another flowchart illustrating the transformation relationship calibration method provided in this application. In some embodiments of this application, by registering the trajectory points of the first motion trajectory and the second motion trajectory, compensation parameters are determined for the initial transformation relationship. The specific implementation process of S104 may include: S1041-S1043, as follows: S1041. Align the first motion trajectory and the second motion trajectory in the coordinate system to obtain the first trajectory to be matched and the second trajectory to be matched.

[0083] Because the visual coordinate system moves with the image acquisition device of the moving object, and the control coordinate system moves with the center of mass of the moving object, differences arise in the orientation and height of the visual and control coordinate systems. Consequently, the orientation and height of the first and second motion trajectories also differ, making direct trajectory point matching impossible. Therefore, when the electronic device begins registering trajectory points for the first and second motion trajectories, it first performs coordinate system alignment processing on the first and second motion trajectories. That is, it aligns the visual and control coordinate systems to transform the first and second motion trajectories into the same coordinate system. The transformation result of the first motion trajectory is determined as the first trajectory to be matched, and the transformation result of the second motion trajectory is determined as the second trajectory to be matched.

[0084] In some embodiments, the electronic device obtains a first trajectory to be matched and a second trajectory to be matched by uniformly transforming the first motion trajectory and the second motion trajectory into a new coordinate system. In other embodiments, the electronic device may also determine the difference between the visual coordinate system and the control coordinate system, and then adjust the first motion trajectory according to the difference, determining the adjusted first motion trajectory as the first trajectory to be matched and directly determining the second motion trajectory as the second trajectory to be matched; or, it may adjust the second motion trajectory according to the difference, determining the adjusted second motion trajectory as the second trajectory to be matched and directly determining the first motion trajectory as the first trajectory to be matched.

[0085] S1042. For each trajectory point in the first trajectory to be matched, determine the corresponding matching trajectory point from the second trajectory to be matched.

[0086] The electronic device matches each trajectory point in the first trajectory to be matched with each trajectory point in the second trajectory to be matched, and determines the trajectory points in the second trajectory that match each trajectory point in the first trajectory to be matched as the matching trajectory points of each trajectory point in the first trajectory to be matched.

[0087] In some embodiments, the electronic device performs distance calculations for each trajectory point in the first trajectory to be matched and each trajectory point in the second trajectory to be matched, and determines the trajectory point in the second trajectory to be matched that is closest to each trajectory point in the first trajectory to be matched as the matching trajectory point.

[0088] In other embodiments, the electronic device can compare the time information of each trajectory point in the first trajectory to be matched with the time information of each trajectory point in the second trajectory to be matched, and determine the trajectory points in the second trajectory that have the same time information as each trajectory point in the first trajectory to be matched as the matching trajectory points. In this case, the electronic device uses time information to speed up the process of determining the matching trajectory points, thereby reducing the time required to determine the matching trajectory points for each trajectory point in the first trajectory to be matched, and thus speeding up the calibration efficiency of the conversion relationship.

[0089] S1043. Determine the compensation parameters based on the coordinates of each trajectory point in the first trajectory to be matched and the coordinates of the matching trajectory point.

[0090] In some embodiments, the electronic device performs singular value decomposition using the coordinates of each trajectory point in the first trajectory to be matched and the coordinates of the matching trajectory point. The singular value decomposition determines the rotation matrix and translation vector between the coordinates of each trajectory point and the coordinates of its corresponding matching trajectory point, and any one or more of the rotation matrix or translation vector are determined as compensation parameters.

[0091] In other embodiments, the electronic device may also perform difference calculations on the coordinates of each trajectory point in the first trajectory to be matched and the coordinates of the matched trajectory point, and determine the calculated difference as a compensation parameter.

[0092] In this embodiment, the electronic device first performs coordinate system alignment processing on the first motion trajectory and the second motion trajectory to eliminate the differences between the first motion trajectory and the second motion trajectory caused by the coordinate system definition, resulting in a first trajectory to be matched and a second trajectory to be matched with only calibration error remaining. Then, for each trajectory point in the first trajectory to be matched, the corresponding matching trajectory point is determined from the second trajectory to be matched. The compensation parameter is then determined by the coordinates of each trajectory point in the first trajectory to be matched and the coordinates of the matching trajectory point, so that the compensation parameter can be more accurate, thereby improving the accuracy of the conversion relationship calibration.

[0093] In some embodiments of this application, the specific implementation process of determining the corresponding matching trajectory point from the second trajectory to be matched for each trajectory point in the first trajectory to be matched, i.e., S1042, may include: S1042a-S1042d, as follows: S1042a. Obtain the first time information of each trajectory point of the first trajectory to be matched, and the second time information of each trajectory point of the second trajectory to be matched.

[0094] Understandably, the first time information is the generation time of each trajectory point in the first trajectory to be matched, which is obtained when the first motion trajectory of the moving object is recorded. The second time information is generated based on the first time information. For example, an electronic device can directly copy the first time information as the second time information for each trajectory point in the second trajectory to be matched (since the second motion trajectory is converted from the first motion trajectory, the time information should also be the same).

[0095] S1042b: The second time information whose difference from the first time information is less than the time threshold is determined as the candidate time information.

[0096] The electronic device calculates the difference between the first and second time information, then compares this difference with a time threshold. Second time information whose difference from the first time information is less than the time threshold is selected as candidate time information. In other words, the electronic device selects second time information that is sufficiently close to the first time information of each trajectory point in the second trajectory to be matched, using these as candidate time information.

[0097] S1042c: Filter the trajectory points in the second trajectory to be matched that correspond to the candidate time information to obtain candidate trajectory points.

[0098] Since the candidate time information is a part of the second time information of the trajectory points in the second trajectory to be matched, the electronic device can filter out the trajectory points corresponding to the candidate time information from the second trajectory to be matched based on the correspondence between the second time information and each trajectory point in the second trajectory to be matched, and determine these trajectory points as candidate trajectory points. It is understandable that the number of candidate trajectory points is not unique.

[0099] S1042d: From the candidate trajectory points, filter for each trajectory point in the first trajectory to be matched to obtain the matching trajectory points.

[0100] In some embodiments, the electronic device may determine the trajectory point with the smallest distance from each trajectory point in the first trajectory to be matched as the matching trajectory point. In other embodiments, the electronic device may also determine the trajectory point with the same pose as each trajectory point in the first trajectory to be matched as the matching trajectory point; the embodiments of this application are not limited thereto.

[0101] In this embodiment, the electronic device first uses time information to perform preliminary registration between each trajectory point of the first trajectory to be matched and each trajectory point of the second trajectory to be matched. Then, it registers each trajectory point of the first trajectory to be matched with candidate trajectory points to select the most suitable matching trajectory point. Since the candidate trajectory points are only a part of the trajectory points of the second trajectory to be matched, the electronic device uses time information to reduce the number of trajectory points to be registered with each trajectory point of the first trajectory to be matched, thereby reducing the computational load during point cloud registration and improving the efficiency of conversion relationship calibration.

[0102] In some embodiments of this application, the specific implementation process of filtering matching trajectory points from candidate trajectory points for each trajectory point in the first trajectory to be matched, i.e., S1042d, may include: S201-S202, as follows: S201. Calculate the distance between each trajectory point in the first trajectory to be matched and the candidate trajectory point to obtain the trajectory distance corresponding to the candidate trajectory point.

[0103] S202. The candidate trajectory points corresponding to the smallest trajectory distance are determined as the matching trajectory points for each trajectory point in the first trajectory to be matched.

[0104] The electronic device can calculate the Euclidean distance between each trajectory point in the first trajectory to be matched and a candidate matching point, and determine the calculated Euclidean distance as the trajectory distance. Alternatively, the electronic device can calculate the Chebyshev distance between each trajectory point in the first trajectory to be matched and a candidate trajectory point, and determine the calculated Chebyshev distance as the trajectory distance. Then, the electronic device compares the trajectory distances corresponding to different candidate trajectory points, determines the smallest trajectory distance, and selects the candidate trajectory point corresponding to the smallest trajectory distance from among the different candidate trajectory points as the matching trajectory point.

[0105] In this embodiment of the application, the electronic device can filter out matching trajectory points from the candidate trajectory points by the distance between each trajectory point in the first trajectory to be matched and the candidate trajectory points, so as to solve the compensation parameters based on the coordinates of each trajectory point in the first trajectory to be matched and the coordinates of the matching trajectory points.

[0106] In some embodiments of this application, after aligning the first motion trajectory and the second motion trajectory to obtain the first trajectory to be matched and the second trajectory to be matched, i.e. after S1041, the method may further include: S1044-S1047, as follows: S1044. Calculate the center of the first trajectory for the first trajectory to be matched, and calculate the center of the second trajectory for the second trajectory to be matched.

[0107] In some embodiments, the electronic device can obtain the first trajectory center of the first trajectory to be matched by averaging the coordinates of each trajectory point of the first trajectory to be matched. Then, the electronic device can determine the second trajectory center of the second trajectory to be matched using the same method as for determining the first trajectory center.

[0108] In other embodiments, the electronic device may also connect the coordinates of the various trajectory points of the first trajectory to be matched into a geometric figure, and determine the geometric center point of the geometric figure as the first trajectory center of the first trajectory to be matched. In this case, the electronic device can use a similar method to determine the second trajectory center of the second trajectory to be matched.

[0109] S1045. Update the first trajectory to be matched using the center of the first trajectory to obtain the updated first trajectory, and update the second trajectory to be matched using the center of the second trajectory to obtain the updated second trajectory.

[0110] Once the center of the first trajectory is known, the electronic device can perform decentralized updates on each trajectory point of the first trajectory to be matched, thereby obtaining a new trajectory for the first trajectory to be matched. This new trajectory is the updated first trajectory. Similarly, the electronic device can obtain the updated second trajectory corresponding to the second trajectory to be matched in a similar manner.

[0111] In this embodiment, the electronic device can calculate the coordinate difference between the coordinates of each trajectory point of the first trajectory to be matched and the coordinates of the center of the first trajectory, and use the obtained coordinate difference to update the coordinates of each trajectory point in the first trajectory to be matched, thereby realizing decentralized updating of the trajectory point information of the first trajectory to be matched. For example, when the coordinates of the trajectory point of the first trajectory to be matched are P1 and the coordinates of the center of the first trajectory are X1, the electronic device can update the original P1 by P1-X1.

[0112] Understandably, updating the first trajectory to be matched using the center of the first trajectory is essentially removing the coordinate components of the center of the first trajectory from each trajectory point of the first trajectory to be matched. This results in each trajectory point of the updated first trajectory revolving around the origin of the coordinate system, effectively moving each trajectory point of the first trajectory to be matched towards the coordinate center. Similarly, each trajectory point of the updated second trajectory also revolves around the origin of the coordinate system. This method of determining the updated first and second trajectories ignores translational changes between the trajectory points of the updated first and second trajectories, better reflecting rotational changes between them. This facilitates a better solution for the rotational error between the visual coordinate system and the control coordinate system (the emphasis on rotational error stems from factors such as the placement of the calibration object and the tilt of the moving object causing three-axis rotational errors, which can lead to a slight rotation between the control coordinate system and the ideal situation. As the moving object travels further, its position shifts increasingly, requiring localization and elimination).

[0113] S1046. For each updated trajectory point in the updated first trajectory, determine the updated matching trajectory point from the updated second trajectory.

[0114] S1047. Based on the coordinates of each updated trajectory point and the coordinates of the updated matching trajectory points, determine the compensation parameters.

[0115] It should be noted that the implementation process of S1046-S1047 is similar to that of S1042-S1043, and will not be repeated here.

[0116] In this embodiment, the electronic device can calculate the corresponding trajectory center for the first trajectory to be matched and the second trajectory to be matched, respectively, that is, obtain the first trajectory center and the second trajectory center. Then, the first trajectory center is used to update the first trajectory to be matched, and the second trajectory center is used to update the second trajectory to be matched, so that the updated first trajectory and the updated second trajectory can better reflect the rotational changes of the trajectory points, so as to better solve the rotational transformation between the visual coordinate system and the control coordinate system and eliminate the small rotational error of the control coordinate system.

[0117] In some embodiments of this application, the first motion trajectory and the second motion trajectory are aligned in coordinate systems to obtain the first trajectory to be matched and the second trajectory to be matched. The specific implementation process of S1041 may include: S1041a, and S1041b or S1041c, as follows: S1041a. Determine the rotation angle and translation distance between the visual coordinate system and the control coordinate system.

[0118] The electronic device can first determine the angle that the coordinate axes of the visual coordinate system (either the X-axis or the Y-axis) have rotated from the start of the moving object to the current moment, as well as the distance that the origin has translated (generally, the directions of the coordinate axes of the visual coordinate system and the control coordinate system are determined at the start of the movement). At the same time, it can determine the angle that the coordinate axes of the control coordinate system have rotated from the start of the moving object to the current moment, as well as the distance that the origin has translated. Then, the difference between the angle rotated by the coordinate axes of the visual coordinate system and the coordinate axes of the control coordinate system is determined as the rotation angle between the visual coordinate system and the control coordinate system. The difference between the distance translated by the origin of the visual coordinate system and the distance translated by the origin of the control coordinate system is determined as the translation distance between the visual coordinate system and the control coordinate system.

[0119] S1041b: Based on the rotation angle and translation distance, the orientation of the first motion trajectory is adjusted to obtain the first trajectory to be matched, and the second motion trajectory is determined as the second trajectory to be matched.

[0120] The electronic device rotates the first motion trajectory by a rotation angle and translates it by a translation distance, thereby aligning the visual coordinate system of the first motion trajectory with the control coordinate system of the second motion trajectory. This adjusts the orientation of the first motion trajectory and identifies the adjusted first motion trajectory as the first trajectory to be matched. Simultaneously, the electronic device directly identifies the second motion trajectory as the second trajectory to be matched.

[0121] S1041c, The first motion trajectory is determined as the first trajectory to be matched, and the orientation of the second motion trajectory is adjusted according to the rotation angle and translation distance to obtain the second trajectory to be matched.

[0122] It should be noted that the process of adjusting the orientation of the second motion trajectory based on the rotation angle and translation distance in this step is the same as that in S1041b, and will not be repeated here.

[0123] In this embodiment, the electronic device can first determine the rotation angle and translation distance between the visual coordinate system and the control coordinate system, and then perform orientation adjustment for either the first motion trajectory or the second motion trajectory so that the coordinate systems of the first motion trajectory and the second motion trajectory are aligned.

[0124] In some embodiments of this application, before determining the rotation angle and translation distance between the visual coordinate system and the controlled coordinate system, i.e., before S1041a, the method may further include: S1041d-S1041e, as follows: S1041d: Determine the orientation of the starting point of the first motion trajectory and the angle between the first coordinate axis of the visual coordinate system to obtain the trajectory angle.

[0125] The orientation of the starting point of the first motion trajectory is actually the initial orientation of the moving object. Therefore, in this embodiment of the application, the electronic device actually determines the angle between the initial orientation of the moving object and the first coordinate axis of the visual coordinate system, and determines the angle as the trajectory angle.

[0126] It is understandable that the first axis of the visual coordinate system is the axis parallel to the horizontal plane in the moving scene, which can be either the X-axis or the Y-axis.

[0127] S1041e. Rotate the second motion trajectory around the second coordinate axis of the control coordinate system by the included angle to obtain the optimized second motion trajectory.

[0128] Understandably, the second coordinate axis is a coordinate axis perpendicular to the horizontal plane in the motion scene, which can be the Z-axis. In other words, the electronic device can select the Z-axis as the center and rotate the second motion trajectory by an angle. By rotating the second motion trajectory, the orientation error caused by the initial orientation of the moving object being different from that of the first coordinate axis can be eliminated, resulting in an optimized second motion trajectory.

[0129] In this case, the process of determining the second motion trajectory as the second trajectory to be matched, i.e., the implementation process of S1041b, can be changed to: determining the optimized second motion trajectory as the second trajectory to be matched.

[0130] Furthermore, the process of adjusting the orientation of the second motion trajectory based on the rotation angle and the translation distance to obtain the second trajectory to be matched, i.e., the implementation process of S1041c, can be changed to: adjusting the orientation of the optimized second motion trajectory based on the rotation angle and the translation distance to obtain the second trajectory to be matched.

[0131] In this embodiment of the application, the electronic device can also eliminate the error caused by the initial orientation of the moving object being different from the first coordinate axis for the second motion trajectory, and use the optimized second motion trajectory to determine the second trajectory to be matched, so as to determine a more accurate matching trajectory point for each trajectory point in the first trajectory to be matched.

[0132] based on Figure 5 See Figure 8 , Figure 8 This is another flowchart illustrating the transformation relationship calibration method provided in the embodiments of this application. In some embodiments of this application, the initial transformation relationship includes: an initial rotation and translation matrix, and the compensation parameter includes: a compensation matrix. At this time, error compensation is performed on the initial transformation relationship based on the compensation parameter to obtain the target transformation relationship between the visual coordinate system and the control coordinate system. The specific implementation process of S105 can include: S1051-S1053, as follows: S1051. Decompose the initial rotation and translation matrix into an initial translation vector and an initial rotation matrix.

[0133] S1052. Using the compensation matrix, update the initial rotation matrix to obtain the updated rotation matrix.

[0134] The electronic device decomposes the initial rotation and translation matrix to obtain the initial translation vector and the initial rotation matrix. Then, the electronic device compensates for the initial rotation matrix by multiplying the compensation matrix with the initial rotation matrix to obtain the updated rotation matrix, or by weighting the compensation matrix and the initial rotation matrix to compensate for the initial rotation matrix to obtain the updated rotation matrix.

[0135] It should be noted that in this embodiment, the electronic device does not need to update the initial translation vector, because the initial translation vector is based on distance measurement, which is generally quite accurate, and small errors will not have a significant impact on the control of the moving object.

[0136] S1053. Integrate the initial translation vector and the updated rotation matrix into an updated rotation and translation matrix, and determine the updated rotation and translation matrix as the target transformation relationship.

[0137] Finally, the electronic device merges the initial translation vector and the updated rotation matrix into an updated rotation-translation matrix, and uses this updated rotation-translation matrix as the target transformation relationship between the visual coordinate system and the control coordinate system. It can be seen that the accuracy of the target transformation relationship is higher than that of the initial transformation relationship.

[0138] In this embodiment, the electronic device uses a compensation matrix to update the initial rotation matrix to eliminate the three-axis rotation error caused by the tilt of the moving object, thereby obtaining a more accurate updated rotation matrix and a more accurate target transformation relationship based on the updated rotation matrix.

[0139] In some embodiments of this application, the compensation parameters are determined based on the coordinates of each trajectory point in the first trajectory to be matched and the coordinates of the matching trajectory point. The specific implementation process of S1043 may include: S1043a-S1043c, as follows: S1043a. Perform the i-th singular value decomposition on the coordinates of each trajectory point in the first trajectory to be matched and the coordinates of the matching trajectory point to obtain the i-th rotation matrix and the i-th translation vector.

[0140] S1043b: Using the i-th rotation matrix and the i-th translation vector, update each trajectory point in the first trajectory point to be matched to obtain the i-th updated trajectory point.

[0141] in, S is the total number of iterations.

[0142] The electronic device rotates each trajectory point in the first trajectory point to be matched according to the i-th rotation matrix and translates it according to the i-th translation vector, and determines the resulting new trajectory point as the i-th updated trajectory point.

[0143] S1043c: When the difference between the distance between the i-th updated trajectory point and the matching trajectory point and the distance between the (i-1)-th updated trajectory point and the matching trajectory point is less than the difference threshold, the cumulative result from the first rotation matrix to the i-th rotation matrix is ​​determined as the compensation parameter.

[0144] The electronic device calculates the distance between the i-th updated trajectory point and the matching trajectory point, and also calculates the distance between the (i-1)-th updated trajectory point and the matching trajectory point. It then calculates the difference between these two distances and compares the difference with a threshold value to determine if the distance between the i-th updated trajectory point and the matching trajectory point has converged. When the distance between the i-th updated trajectory point and the matching trajectory point is less than the threshold value, it indicates that the distance between them has converged. At this point, the electronic device accumulates all the solved rotation matrices, starting from the first rotation matrix and accumulating up to the i-th rotation matrix. The accumulated result is then used as the final compensation parameter.

[0145] In this embodiment, the electronic device determines whether each trajectory point of the first trajectory to be matched and the matching trajectory point are correct matching point pairs by iterating the nearest point. If they are correct, the rotation matrix in the iteration process is determined as a compensation parameter so that the initial transformation relationship can be compensated according to the compensation parameter in the future.

[0146] In some embodiments of this application, after controlling the moving object to move and recording the first motion trajectory of the moving object in the visual coordinate system, i.e. after S102, the method may further include: S106-S107, as follows: S106. Record the third motion trajectory of the moving object in the control coordinate system.

[0147] When a moving object is in motion, the electronic device records its trajectory in the control coordinate system, resulting in a third motion trajectory. It should be noted that while both the second and third motion trajectories are motion trajectories of the object in the control coordinate system, the second trajectory is calculated based on the first trajectory, while the third trajectory is directly recorded by the electronic device.

[0148] S107. Based on the alignment of the first motion trajectory and the third motion trajectory, determine the target transformation relationship between the visual coordinate system and the control coordinate system.

[0149] The electronic device directly aligns the first motion trajectory and the third motion trajectory, that is, it makes the first motion trajectory and the second motion trajectory coincide. In this way, the rotation angle and translation distance between the visual coordinate system and the control coordinate system can be directly determined, thereby obtaining the target transformation relationship between the visual coordinate system and the control coordinate system.

[0150] In this embodiment of the application, the electronic device can also omit the coarse calibration process during the calibration of the transformation relationship, that is, omit the process of calculating the initial transformation relationship, and directly determine the difference between the visual coordinate system and the control coordinate system by aligning the first motion trajectory and the second motion trajectory, thereby enabling the target transformation relationship to be determined more quickly.

[0151] In some embodiments of this application, the specific implementation process of controlling the motion of the moving object, i.e., S102, may include: S1021, as follows: S1021. Control the moving object to perform translational motion.

[0152] In this embodiment, since both the centroid and the image acquisition device perform rigid motion on the moving object, it can be assumed that the trajectory of the moving object in the visual coordinate system and the trajectory of the moving object in the control coordinate system should remain parallel along the Z-axis. Furthermore, on the plane formed by the X and Y axes, the angle between the trajectory projections and the angle between the direction of motion of the cloud object in the visual coordinate system and the X-axis should be consistent. To satisfy this constraint, the electronic device should control the moving object to perform translational motion during the conversion relationship calibration process.

[0153] In this embodiment of the application, the electronic device can control the moving object to perform translational motion so that the first motion trajectory and the second motion trajectory remain parallel in the Z-axis direction. This allows the compensation parameters determined by point cloud registration based on the first motion trajectory and the second motion trajectory to be directly used to compensate the initial rotation matrix in the initial transformation relationship.

[0154] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0155] The embodiments of this application are implemented in the scenario of calibrating the visual coordinate system and control coordinate system of a quadruped robot (moving object). Figure 9A This is a schematic diagram of a quadruped robot provided in an embodiment of this application. Figure 9A The image shows the side view of the quadruped robot 11-1. Figure 9B This is another schematic diagram of the quadruped robot provided in the embodiments of this application. Figure 11 Part B shows the front view of the quadruped robot 11-1.

[0156] First, this application will introduce the definition of the coordinate system for a quadruped robot.

[0157] The world coordinate system (T_world) is defined based on the real physical world and according to the needs of the task, and conforms to the right-hand screw rule. The visual coordinate system (T_vision) is used for visual navigation. Its origin is fixed on a known map plane, and its Z-axis is perpendicular to the map plane, conforming to the right-hand screw rule. Generally, the map is fixed to the real world, allowing the visual coordinate system and the world coordinate system to coincide. The camera coordinate system (T_camera) has the optical center of the camera (image acquisition device) as its origin, with the Z-axis coinciding with the optical axis. It is used to describe the camera's pose and conforms to the right-hand screw rule. The quadruped robot's center-of-mass coordinate system (T_robot) has its origin at the quadruped robot's center of mass (usually the center), with the X-axis pointing towards the quadruped robot's orientation and the Z-axis perpendicular to the quadruped robot's motion plane. It is used to describe the quadruped robot's pose and conforms to the right-hand screw rule. The control coordinate system (T_control) is determined by the initial centroid coordinate system and is fixed to the world coordinate system. It is used to send position-based control commands to the quadruped robot. The origin is the projection point of the initial position of the centroid coordinate system onto the XY plane of the world coordinate system, and the initial attitude plane of the three axes is the centroid coordinate system.

[0158] For example, Figure 10 This is a schematic diagram illustrating the coordinate system dependency provided in an embodiment of this application. See also... Figure 10 The transformation relationship between the visual coordinate system 10-1 and the camera coordinate system 10-2 is obtained through measurement. The camera coordinate system 10-2 can change with the movement of the quadruped robot's camera; for example, when the orientation of the quadruped robot's camera changes, the camera coordinate system 10-2 will also rotate, transforming into the camera coordinate system 10-21. The control coordinate system 10-3 can be obtained by initializing the quadruped robot's centroid coordinate system 10-4. The centroid coordinate system 10-4 also changes with the movement of the quadruped robot's center of mass; for example, when the orientation of the quadruped robot's center of mass changes, the centroid coordinate system 10-4 will also rotate, transforming into the centroid coordinate system 10-41. Therefore, in the process of calibrating the transformation relationship between the visual coordinate system 10-1 and the control coordinate system 10-3, only the transformation relationship between the camera coordinate system 10-2 and the centroid coordinate system 10-4 is unknown and needs to be solved.

[0159] In the calibration process of a quadruped robot, the calibration board is first identified and measured by the monocular camera of the quadruped robot, or the map stored locally is identified and measured. The camera extrinsic parameters (i.e., Tvision_camera) in the visual coordinate system are calculated by the principle of central projection. It can be seen that the estimation of the camera pose depends on the measurement accuracy and the map accuracy.

[0160] Since the control system needs to control the quadruped robot, it needs to describe the quadruped robot's pose (i.e., Tcontrol_robot) in the control coordinate system. Therefore, the relationship in equation (3) can be obtained from the definition of the coordinate system:

[0161] Since the visual coordinate system and the world coordinate system have coincided, Tworld can be replaced with Tvision, thus transforming equation (3) into equation (4):

[0162] It should be noted that Tcontrol_vision can be initialized by the pose Tvision_robot_init of the quadruped robot's center of mass at the initial moment, and Tvision_robot_init can be obtained from equation (5):

[0163] Where Tvision_robot_init represents the initial pose of the camera in visual coordinates. Therefore, it can be seen that only during the entire transformation process... It is unknown and needs to be calibrated.

[0164] The electronic device can obtain tcamera_robot by measuring the positional relationship between the camera positioned at the end of the quadruped robot and the robot's center of mass. (It should be noted that...) This can be represented using a rotation-translation matrix, thus decomposing into a translation vector tcamera_robot and a rotation matrix Rcamera_robot. This is because distance measurements are generally quite accurate, and even small errors will not significantly affect the subsequent control of the quadruped robot. However, calibrating Rcamera_robot requires measuring the pose of the camera and the quadruped robot's center of mass.

[0165] The electronic device controls the quadruped robot to call the monocular camera, and takes a picture to measure the camera pose (first pose parameter) Tvision_camera in the visual coordinate system. At the same time, it controls the quadruped robot's center of mass to align with a certain position in the visual coordinate system to obtain the center of mass pose (second pose parameter) Tvision_robot. Based on Tvision_camera and Tvision_robot, the transformation relationship Tcamera_robot (initial transformation relationship) can be calculated.

[0166] However, due to measurement errors, accuracy errors of the calibration object and map, or positioning errors of the quadruped robot's center of mass, or three-axis rotation errors caused by the quadruped robot's body tilt, the control coordinate system will rotate slightly compared to the ideal situation. As the quadruped robot continues to move, the position of the quadruped robot in the control coordinate system will shift more and more, resulting in errors in the center of mass trajectory and camera trajectory.

[0167] For example, Figure 11 This is a schematic diagram illustrating the error between the center of mass trajectory and the camera trajectory of the quadruped robot during its movement, as provided in the embodiments of this application. Figure 11 The left view showing the errors of the centroid trajectory 11-1 and camera trajectory 11-2 is provided. It can be seen that due to the rotation error of the control coordinate system, the centroid trajectory 11-1 is not parallel to the ground. Furthermore, combined with... Figure 6 It can be seen that the centroid trajectory 11-1 and the camera trajectory 11-2 are not parallel, indicating that there is an error in the transformation relationship Tcamera_robot.

[0168] In response, the electronic device updates the transformation relationship Tcamera_robot based on point cloud registration. Below is a detailed process of updating the transformation relationship Tcamera_robot through point cloud registration: 1. The electronic device sets the initial conversion relationship Tcamera_robot (initial conversion relationship).

[0169] 2. The electronic device determines the angle between the initial orientation of the quadruped robot in the visual coordinate system and the X-axis (first coordinate axis) of the visual coordinate system. (The angle between the trajectories).

[0170] 3. Electronic equipment controls the quadruped robot to perform a translational movement.

[0171] The reason for performing translational motion is that, in the real world, the camera and center of mass of a quadruped robot are both fixed to the body and undergo rigid motion. Therefore, the camera trajectory and the center of mass trajectory should remain parallel in the Z-axis direction, and the angle between their projected trajectories in the XY plane should be consistent with the angle between the quadruped robot's motion direction in the visual coordinate system and the X-axis. To satisfy this constraint, the quadruped robot can only be controlled to perform translational motion.

[0172] 4. The electronic device obtains the camera trajectory (first motion trajectory) in the visual coordinate system and the centroid trajectory (second motion trajectory) in the control coordinate system.

[0173] The centroid trajectory in the control coordinate system is obtained by rotating and translating the camera trajectory in the visual coordinate system according to Tcamera_robot.

[0174] 5. The electronic device will align the control coordinate system and the visual coordinate system (coordinate system alignment process) to obtain a point cloud where the centroid trajectory and the camera trajectory are in the same coordinate system.

[0175] 6. The electronic device rotates the center of mass trajectory along the Z-axis. This eliminates the error in the initial orientation of the center of mass trajectory.

[0176] 7. The electronic device uses the camera trajectory as the target trajectory (the first trajectory to be matched) and the centroid trajectory as the current trajectory (the second trajectory to be matched) to perform point cloud registration, and obtains the rotation matrix R (compensation parameter).

[0177] 8. The electronic device compensates the rotation matrix R into the initial transformation relation Tcamera_robot, resulting in Rcamera_robot = R. Rcamera_robot, based on the updated Rcamera_robot (obtaining the updated rotation matrix), the corrected transformation relationship between the visual coordinate system and the control coordinate system (target transformation relationship).

[0178] If the centroid trajectory obtained by re-transforming the camera trajectory using the final transformation relationship can coincide with the camera trajectory.

[0179] Figure 12A This is a schematic diagram comparing the centroid trajectory and the camera trajectory obtained based on the modified transformation relationship, as provided in an embodiment of this application. Figure 12A As can be seen, both camera trajectory 12-1 and centroid trajectory 12-2 remain horizontal with respect to the ground. Figure 12B This is another comparative schematic diagram of the centroid trajectory and camera trajectory obtained based on the modified transformation relationship provided in an embodiment of this application. Figure 12BAs can be seen, the camera trajectory 12-1 and the centroid trajectory 12-2 are parallel to each other and nearly coincident. Therefore, the corrected transformation relationship is more accurate.

[0180] Figure 13 This is a schematic diagram of the point cloud registration process provided in the embodiments of this application. See also... Figure 13 The point cloud registration process includes: S301. Initialize the electronic device with the number of iteration steps i, convergence condition (difference threshold), and maximum number of iterations.

[0181] S302. The electronic device inputs two sets of point clouds, referred to as target point cloud P1 (the trajectory points to be matched with the first trajectory) and current point cloud P2 (the trajectory points to be matched with the second trajectory).

[0182] S303 and the electronic device respectively calculate the coordinate mean values ​​X1 (first trajectory center) and X2 (second trajectory center) of P1 and P2, and set P1=P1-X1 and P2=P2-X2.

[0183] S304. The electronic device finds the point in P2 that is closest to each point in P1 (matching trajectory point) to form a point pair Pi.

[0184] S305. Electronic equipment uses SVD decomposition to obtain the rotation matrix Ri (the i-th rotation matrix) and the translation vector ti (the i-th translation vector).

[0185] S306. The electronic device updates P1 using the rotation matrix Ri and the translation vector ti.

[0186] S307. The electronic device calculates the average distance between P1 and P2 during the i-th iteration.

[0187] S308. The electronic device determines whether the difference between the average distance of the i-th time and the average distance of the (i-1)-th time is less than the convergence condition. If yes, proceed to S309; ​​otherwise, proceed to S310.

[0188] S309, Electronic device returns compensation matrix R.

[0189] S310. The electronic device determines whether the iteration step number i is greater than the maximum number of iterations. If yes, execute S309; ​​otherwise, increment i by 1 and execute S304.

[0190] The following description continues to illustrate the exemplary structure of the conversion relationship calibration device 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 4 As shown, the software module stored in the conversion relationship calibration device 455 of the memory 450 may include: The relationship determination module 4551 is used to determine the initial transformation relationship between the visual coordinate system and the control coordinate system of the moving object by using the first pose parameter of the image acquisition device set on the moving object and the second pose parameter of the centroid of the moving object. Motion control module 4552 is used to control the motion object to perform motion; The trajectory recording module 4553 is used to record the first motion trajectory of the moving object in the visual coordinate system; The trajectory conversion module 4554 is used to convert the first motion trajectory using the initial conversion relationship to obtain the second motion trajectory of the moving object in the control coordinate system; The parameter determination module 4555 is used to determine compensation parameters for the initial transformation relationship by registering the trajectory points of the first motion trajectory and the second motion trajectory. The relationship determination module 4551 is further configured to perform error compensation on the initial transformation relationship based on the compensation parameters, so as to obtain the target transformation relationship between the visual coordinate system and the control coordinate system.

[0191] In some embodiments of this application, the parameter determination module 4555 is further configured to perform coordinate system alignment processing on the first motion trajectory and the second motion trajectory to obtain a first trajectory to be matched and a second trajectory to be matched. For each trajectory point in the first trajectory to be matched, a corresponding matching trajectory point is determined from the second trajectory to be matched; and a compensation parameter is determined based on the coordinates of each trajectory point in the first trajectory to be matched and the coordinates of the matching trajectory point.

[0192] In some embodiments of this application, the parameter determination module 4555 is further configured to acquire first time information of each trajectory point of the first trajectory to be matched, and second time information of each trajectory point of the second trajectory to be matched; determine the second time information whose difference from the first time information is less than a time threshold as candidate time information; filter the trajectory points in the second trajectory to be matched that correspond to the candidate time information to obtain candidate trajectory points; and filter the candidate trajectory points to obtain the matching trajectory points for each trajectory point in the first trajectory to be matched from the candidate trajectory points.

[0193] In some embodiments of this application, the parameter determination module 4555 is further configured to perform distance calculation for each trajectory point in the first trajectory to be matched and the candidate trajectory point to obtain the trajectory distance corresponding to the candidate trajectory point; and determine the candidate trajectory point corresponding to the smallest trajectory distance as the matching trajectory point of each trajectory point in the first trajectory to be matched.

[0194] In some embodiments of this application, the parameter determination module 4555 is further configured to: after performing coordinate system alignment processing on the first motion trajectory and the second motion trajectory to obtain a first trajectory to be matched and a second trajectory to be matched, calculate a first trajectory center for the first trajectory to be matched and calculate a second trajectory center for the second trajectory to be matched; update the first trajectory to be matched using the first trajectory center to obtain an updated first trajectory, and update the second trajectory to be matched using the second trajectory center to obtain an updated second trajectory; for each updated trajectory point in the updated first trajectory, determine an updated matching trajectory point from the updated second trajectory; and determine the compensation parameter based on the coordinates of each updated trajectory point and the coordinates of the updated matching trajectory point.

[0195] In some embodiments of this application, the parameter determination module 4555 is further configured to determine the rotation angle and translation distance between the visual coordinate system and the control coordinate system; adjust the orientation of the first motion trajectory according to the rotation angle and the translation distance to obtain the first trajectory to be matched, and determine the second motion trajectory as the second trajectory to be matched; or, determine the first motion trajectory as the first trajectory to be matched, and adjust the orientation of the second motion trajectory according to the rotation angle and the translation distance to obtain the second trajectory to be matched.

[0196] In some embodiments of this application, the parameter determination module 4555 is further configured to determine the orientation of the starting point of the first motion trajectory and the included angle of the first coordinate axis of the visual coordinate system before determining the rotation angle and translation distance between the visual coordinate system and the control coordinate system, thereby obtaining a trajectory included angle; the first coordinate axis is a coordinate axis parallel to the horizontal plane in the motion scene; the second motion trajectory is rotated around the second coordinate axis of the control coordinate system by the trajectory included angle to obtain an optimized second motion trajectory; the second coordinate axis is a coordinate axis perpendicular to the horizontal plane in the motion scene; the optimized second motion trajectory is determined as the second trajectory to be matched; based on the rotation angle and the translation distance, the orientation of the optimized second motion trajectory is adjusted to obtain the second trajectory to be matched.

[0197] In some embodiments of this application, the relationship determination module 4551 is further configured to decompose the initial rotation and translation matrix into an initial translation vector and an initial rotation matrix; update the initial rotation matrix using the compensation matrix to obtain an updated rotation matrix; integrate the initial translation vector and the updated rotation matrix into an updated rotation and translation matrix, and determine the updated rotation and translation matrix as the target transformation relationship.

[0198] In some embodiments of this application, the parameter determination module 4555 is further configured to perform an i-th singular value decomposition on the coordinates of each trajectory point in the first trajectory to be matched and the coordinates of the matching trajectory point to obtain an i-th rotation matrix and an i-th translation vector. S is the total number of iterations; using the i-th rotation matrix and the i-th translation vector, each trajectory point in the first trajectory to be matched is updated to obtain the i-th updated trajectory point; when the difference between the distance between the i-th updated trajectory point and the matching trajectory point and the distance between the (i-1)-th updated trajectory point and the matching trajectory point is less than the difference threshold, the cumulative result from the first rotation matrix to the i-th rotation matrix is ​​determined as the compensation parameter.

[0199] In some embodiments of this application, the trajectory recording module 4553 is further configured to record a third motion trajectory of the moving object in the control coordinate system; The relationship determination module 4551 is further configured to determine the target transformation relationship between the visual coordinate system and the control coordinate system based on the alignment of the first motion trajectory and the third motion trajectory.

[0200] In some embodiments of this application, the motion control module 4552 is also used to control the moving object to perform translational motion.

[0201] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the relationship identification method described above in this application.

[0202] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to execute the conversion relationship marking method provided in this application. For example, ... Figure 5 The transformation relationship calibration method is shown.

[0203] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0204] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0205] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0206] As an example, executable instructions can be deployed to execute on a single computing device (electronic device), or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0207] In summary, through the embodiments of this application, a coarse calibration of the transformation relationship between the visual coordinate system and the control coordinate system is first performed based on the first pose parameter and the second pose parameter to obtain an initial transformation relationship. Then, the moving object is controlled to generate a first motion trajectory in the visual coordinate system, and combined with the initial transformation relationship, the first motion trajectory is transformed into a second motion trajectory in the control coordinate system. Finally, the trajectory points of the first motion trajectory and the trajectory points of the second motion trajectory are registered. Based on the registered trajectory points, compensation parameters are determined for the initial transformation relationship. The compensation parameters are used to eliminate the error of the initial transformation relationship, resulting in a more accurate target transformation relationship. Thus, the accuracy of the transformation relationship calibration is improved. The use of time information reduces the number of trajectory points to be registered with each trajectory point in the first trajectory point to be matched, thereby reducing the computational load during point cloud registration and improving the efficiency of the transformation relationship calibration. The first trajectory center is used to update the first trajectory to be matched, and the second trajectory center is used to update the second trajectory to be matched, so that the updated first trajectory and updated second trajectory can better reflect the rotational changes of the trajectory points, so as to better solve the rotational transformation between the visual coordinate system and the control coordinate system and control the small rotational error of the coordinate system.

[0208] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for calibrating transformation relationships, characterized in that, The method includes: Using the first pose parameter of the image acquisition device set on the moving object and the second pose parameter of the centroid of the moving object, the initial transformation relationship between the visual coordinate system and the control coordinate system of the moving object is determined; Control the moving object to move, and record the first motion trajectory of the moving object in the visual coordinate system, and determine the first motion trajectory as the first trajectory to be matched; The first motion trajectory is transformed using the initial transformation relationship to obtain the second motion trajectory of the moving object in the control coordinate system; The second motion trajectory is rotated around the second coordinate axis of the control coordinate system by a trajectory angle to obtain the optimized second motion trajectory; the second coordinate axis is a coordinate axis perpendicular to the horizontal plane in the motion scene; the trajectory angle is the angle between the orientation of the starting point of the first motion trajectory and the first coordinate axis of the visual coordinate system; the first coordinate axis is a coordinate axis parallel to the horizontal plane in the motion scene. Based on the rotation angle and translation distance between the visual coordinate system and the control coordinate system, the orientation of the optimized second motion trajectory is adjusted to obtain the second trajectory to be matched. For each trajectory point in the first trajectory to be matched, a corresponding matching trajectory point is determined from the second trajectory to be matched; based on the coordinates of each trajectory point in the first trajectory to be matched and the coordinates of the matching trajectory point, a compensation parameter is determined; Based on the compensation parameters, error compensation is performed on the initial transformation relationship to obtain the target transformation relationship between the visual coordinate system and the control coordinate system.

2. The method according to claim 1, characterized in that, The step of determining the corresponding matching trajectory point from the second trajectory to be matched for each trajectory point in the first trajectory to be matched includes: Obtain the first time information of each trajectory point of the first trajectory to be matched, and the second time information of each trajectory point of the second trajectory to be matched; Second time information whose difference from the first time information is less than the time threshold is determined as candidate time information; The trajectory points in the second trajectory to be matched that correspond to the candidate time information are filtered to obtain candidate trajectory points; The matching trajectory points are obtained by filtering each trajectory point in the first trajectory to be matched from the candidate trajectory points.

3. The method according to claim 2, characterized in that, The step of filtering the candidate trajectory points to obtain the matching trajectory points for each trajectory point in the first trajectory to be matched includes: For each trajectory point in the first trajectory to be matched and the candidate trajectory point, the distance is calculated to obtain the trajectory distance corresponding to the candidate trajectory point; The candidate trajectory point corresponding to the smallest trajectory distance is determined as the matching trajectory point for each trajectory point in the first trajectory to be matched.

4. The method according to any one of claims 2 to 3, characterized in that, The method, after determining the first motion trajectory as the first trajectory to be matched, and adjusting the orientation of the optimized second motion trajectory based on the rotation angle and the translation distance to obtain the second trajectory to be matched, includes: Calculate the center of a first trajectory for the first trajectory to be matched, and calculate the center of a second trajectory for the second trajectory to be matched; The first trajectory to be matched is updated using the first trajectory center to obtain an updated first trajectory, and the second trajectory to be matched is updated using the second trajectory center to obtain an updated second trajectory; For each updated trajectory point in the updated first trajectory, an updated matching trajectory point is determined from the updated second trajectory; The compensation parameters are determined based on the coordinates of each updated trajectory point and the coordinates of the updated matching trajectory point.

5. The method according to any one of claims 2 to 3, characterized in that, The method further includes: Based on the rotation angle and the translation distance, the orientation of the first motion trajectory is adjusted to obtain the first trajectory to be matched, and the second motion trajectory is determined as the second trajectory to be matched; or, The first motion trajectory is determined as the first trajectory to be matched, and the orientation of the second motion trajectory is adjusted according to the rotation angle and the translation distance to obtain the second trajectory to be matched.

6. The method according to claim 5, characterized in that, Determining the second motion trajectory as the second trajectory to be matched includes: The optimized second motion trajectory is determined as the second trajectory to be matched; The step of adjusting the orientation of the second motion trajectory based on the rotation angle and the translation distance to obtain the second trajectory to be matched includes: Based on the rotation angle and the translation distance, the orientation of the optimized second motion trajectory is adjusted to obtain the second trajectory to be matched.

7. The method according to any one of claims 1 to 3, characterized in that, The initial transformation relationship includes an initial rotation and translation matrix, and the compensation parameters include a compensation matrix. The step of performing error compensation on the initial transformation relationship based on the compensation parameters to obtain the target transformation relationship between the visual coordinate system and the control coordinate system includes: The initial rotation and translation matrix is ​​decomposed into an initial translation vector and an initial rotation matrix; The initial rotation matrix is ​​updated using the compensation matrix to obtain the updated rotation matrix; The initial translation vector and the updated rotation matrix are integrated into an updated rotation-translation matrix, and the updated rotation-translation matrix is ​​determined as the target transformation relationship.

8. The method according to any one of claims 2 to 3, characterized in that, The step of determining compensation parameters based on the coordinates of each trajectory point in the first trajectory to be matched and the coordinates of the matched trajectory point includes: For the coordinates of each trajectory point in the first trajectory to be matched and the coordinates of the matching trajectory point, perform the i-th singular value decomposition to obtain the i-th rotation matrix and the i-th translation vector; S is the total number of iterations; Using the i-th rotation matrix and the i-th translation vector, each trajectory point in the first trajectory to be matched is updated to obtain the i-th updated trajectory point; When the difference between the distance between the i-th updated trajectory point and the matching trajectory point and the distance between the (i-1)-th updated trajectory point and the matching trajectory point is less than the difference threshold, the cumulative result from the first rotation matrix to the i-th rotation matrix is ​​determined as the compensation parameter.

9. The method according to any one of claims 1 to 3, characterized in that, After controlling the moving object to move and recording the first motion trajectory of the moving object in the visual coordinate system, the method further includes: Record the third motion trajectory of the moving object in the control coordinate system; Based on the alignment of the first motion trajectory and the third motion trajectory, the target transformation relationship between the visual coordinate system and the control coordinate system is determined.

10. The method according to any one of claims 1 to 3, characterized in that, Controlling the movement of the moving object includes: Control the moving object to perform translational motion.

11. A conversion relationship calibration device, characterized in that, The device includes: The relationship determination module is used to determine the initial transformation relationship between the visual coordinate system and the control coordinate system of the moving object by using the first pose parameter of the image acquisition device set on the moving object and the second pose parameter of the centroid of the moving object. A motion control module is used to control the motion object to perform its motion; A trajectory recording module is used to record the first motion trajectory of the moving object in the visual coordinate system; The parameter determination module is used to determine the first motion trajectory as the first trajectory to be matched; The trajectory conversion module is used to convert the first motion trajectory using the initial conversion relationship to obtain the second motion trajectory of the moving object in the control coordinate system; The parameter determination module is used to rotate the second motion trajectory around the second coordinate axis of the control coordinate system by a trajectory angle to obtain an optimized second motion trajectory; the second coordinate axis is a coordinate axis perpendicular to the horizontal plane in the motion scene; the trajectory angle is the angle between the orientation of the starting point of the first motion trajectory and the first coordinate axis of the visual coordinate system; the first coordinate axis is a coordinate axis parallel to the horizontal plane in the motion scene; based on the rotation angle and translation distance between the visual coordinate system and the control coordinate system, the orientation of the optimized second motion trajectory is adjusted to obtain a second trajectory to be matched; for each trajectory point in the first trajectory to be matched, a corresponding matching trajectory point is determined from the second trajectory to be matched; based on the coordinates of each trajectory point in the first trajectory to be matched and the coordinates of the matching trajectory point, compensation parameters are determined; The relationship determination module is further configured to perform error compensation on the initial transformation relationship based on the compensation parameters, so as to obtain the target transformation relationship between the visual coordinate system and the control coordinate system.

12. The apparatus according to claim 11, characterized in that, The parameter determination module is further configured to acquire first time information of each trajectory point of the first trajectory to be matched, and second time information of each trajectory point of the second trajectory to be matched; and to determine the second time information whose difference from the first time information is less than a time threshold as candidate time information. The trajectory points in the second trajectory to be matched that correspond to the candidate time information are filtered to obtain candidate trajectory points; The matching trajectory points are obtained by filtering each trajectory point in the first trajectory to be matched from the candidate trajectory points.

13. The apparatus according to claim 12, characterized in that, The parameter determination module is further configured to perform distance calculations between each trajectory point in the first trajectory to be matched and the candidate trajectory point, and obtain the trajectory distance corresponding to the candidate trajectory point; The candidate trajectory point corresponding to the smallest trajectory distance is determined as the matching trajectory point for each trajectory point in the first trajectory to be matched.

14. An electronic device, characterized in that, The electronic device includes: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the conversion relationship calibration method according to any one of claims 1 to 10.

15. A computer-readable storage medium storing executable instructions, characterized in that, When the executable instructions are executed by the processor, they implement the conversion relationship calibration method according to any one of claims 1 to 10.

16. A computer program product comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the conversion relationship calibration method according to any one of claims 1 to 10.

Images