Calibration method and system of electronic device, calibration device and storage medium

Abstract

This application provides a calibration method, system, calibration device, and storage medium for an electronic device. The calibration method includes: obtaining a first pose relationship, which is the relative pose between a first camera and a second camera when the electronic device is not placed on a robotic arm, wherein the first and second cameras are fixed to the robotic arm; calibrating the relative pose between a first sensor and the second camera within the electronic device when the electronic device is fixed to the robotic arm, to obtain a second pose relationship; and determining the relative pose between the first sensor and the first camera based on the first and second pose relationships. The first camera does not need to photograph the calibration chart through the electronic device's display module; instead, it calculates the relative pose between the sensor and the first camera through coordinate system transfer relationships, improving calibration accuracy.

Description

Technical Field

[0001] This application belongs to the field of XR equipment calibration, and particularly relates to calibration methods, systems, calibration equipment and storage media for electronic equipment. Background Technology

[0002] In head-mounted extended reality (XR) devices, to ensure that the virtual image displayed by the XR device does not drift as the wearer's head moves, it is usually necessary to pre-calibrate the relative pose between the sensors on the XR device and the human eye. In automated calibration lines, a human-eye-like camera can be used to replace the human eye position to calibrate the relative pose between the sensors and the human-eye-like camera within the XR device.

[0003] In existing calibration methods, the images used for calibration are calibration charts taken by a human eye camera through the optical lens of an XR device. The inherent refraction and distortion of the optical lens will cause severe deformation of the captured calibration chart, thus affecting the calibration accuracy. Summary of the Invention

[0004] In view of this, embodiments of this application provide a calibration method, system, calibration device, and storage medium for electronic devices, which can accurately calibrate the relative pose between the sensors in the electronic device and the first camera without the first camera needing to photograph the calibration chart through the optical lens of the electronic device.

[0005] A first aspect of this application provides a calibration method for an electronic device, comprising: Obtain the first pose relationship, which is the relative pose between the first camera and the second camera when no electronic device is placed on the robotic arm, and the first camera and the second camera are fixed on the robotic arm. When fixing the electronic device on the robotic arm, the relative pose between the first sensor and the second camera inside the electronic device is calibrated to obtain a second pose relationship. The relative pose between the first sensor and the first camera is determined based on the first pose relationship and the second pose relationship.

[0006] In one embodiment, obtaining the first pose relation includes: When the electronic device is not fixed to the robotic arm, the robotic arm is controlled to move relative to the calibration map card so that the calibration map card is located within the target depth of field range of the first camera and the second camera. The first camera and the second camera are controlled to simultaneously capture images of the calibration chart to obtain a first image and a second image. The first pose relationship is determined based on the first image and the second image.

[0007] In one embodiment, the first sensor is a motion sensor, and the step of calibrating the relative pose between the first sensor and the second camera within the electronic device to obtain a second pose relationship includes: Control the movement of the robotic arm; Acquire a first image sequence and motion data of the electronic device. The first image sequence is an image sequence obtained by the second camera capturing a calibration chart during the movement of the robotic arm. The motion data is data collected by the motion sensor during the movement of the robotic arm. The second pose relationship is determined based on the motion data and the first image sequence.

[0008] In one embodiment, the first sensor is a real-time localization and mapping camera, and the step of calibrating the relative pose between the first sensor and the second camera within the electronic device to obtain a second pose relationship includes: Control the movement of the robotic arm; Acquire a second image sequence and a first image sequence, wherein the second image sequence is an image sequence obtained by the real-time localization and mapping camera capturing images of the calibration map during the movement of the robotic arm, and the first image sequence is an image sequence obtained by the second camera capturing images of the calibration map during the movement of the robotic arm; The second pose relationship is determined based on the second image sequence and the first image sequence.

[0009] In one embodiment, the electronic device further includes a second sensor, wherein one of the first and second sensors is a motion sensor and the other is a real-time localization and mapping camera, and the calibration method further includes: When the electronic device is fixed on the robotic arm, the relative pose between the second sensor and the second camera is calibrated to obtain a third pose relationship. The relative pose between the second sensor and the first camera is determined based on the first pose relationship and the third pose relationship.

[0010] In one embodiment, the sampling times of the electronic device, the first camera, and the second camera are based on the same time reference.

[0011] In one embodiment, after determining the relative pose between the second sensor and the first camera based on the first pose relationship and the third pose relationship, the method further includes: Obtain the verification image of the calibration map card captured by the real-time positioning and mapping camera; Based on the real-time positioning and mapping, the relative pose between the camera and the first camera is constructed, and the preset calibration points in the verification image are projected onto the virtual image captured by the first camera. Determine the pixel error between the projected position of the calibration point in the virtual image and the theoretical position of the calibration point in the virtual image; If the pixel error is within a set threshold range, the relative pose between the first sensor and the first camera and the relative pose between the second sensor and the first camera are verified. If the pixel error is not within the set threshold range, a recalibration prompt message will be output.

[0012] A second aspect of this application provides a calibration apparatus for an electronic device, comprising: The first calibration module is used to obtain the first pose relationship, which is the relative pose between the first camera and the second camera when no electronic device is placed on the robotic arm. The first camera and the second camera are fixed on the robotic arm. The second calibration module is used to calibrate the relative pose between the first sensor and the second camera in the electronic device when the electronic device is fixed on the robotic arm, so as to obtain a second pose relationship. The output module is used to determine the relative pose between the first sensor and the first camera based on the first pose relationship and the second pose relationship.

[0013] A third aspect of this application provides a calibration device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the calibration method of the electronic device as described in the first aspect above.

[0014] A fourth aspect of this application provides a calibration system for an electronic device, comprising: Robotic arms are used to provide multi-degree-of-freedom motion; A clamp, mounted on the robotic arm, is used to secure the electronic device, the first camera, and the second camera. The calibration chart is placed within a preset field of view in front of the robotic arm; And the calibration equipment as described in the third aspect above.

[0015] In one embodiment, the calibration system further includes a clock synchronization controller, which is connected to the electronic device, the first camera, and the second camera, respectively. The clock synchronization controller is used to output a hardware trigger signal to the electronic device, the first camera and the second camera, and the hardware trigger signal is used to trigger the sampling time of the electronic device, the first camera and the second camera to be unified to the same time base.

[0016] A fifth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the calibration method for an electronic device as described in the first aspect above.

[0017] A sixth aspect of this application provides a computer program product that, when run on a calibration device, causes the calibration device to perform the calibration method of the electronic device described in any of the first aspects.

[0018] The beneficial effects of this application embodiment compared to the prior art are as follows: By introducing a second camera as a bridge for spatial coordinate system transformation, the calibration process is divided into a pre-calibration stage and an online calibration stage. In the pre-calibration stage, the first camera does not need to photograph the calibration chart through the display module of the electronic device, eliminating the influence of optical distortion and obtaining a clear, distortion-free image, thereby obtaining a high-precision relative pose relationship between the first and second cameras. In the online calibration stage, the relative pose between the second camera and the first sensor is calibrated, avoiding errors caused by the first camera photographing through the display module, significantly improving calibration accuracy. Finally, the relative pose between the sensor and the first camera is calculated through coordinate system transfer relationships, further improving calibration accuracy. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0020] Figure 1 This is a schematic diagram of a calibration system for an electronic device provided in an embodiment of this application; Figure 2 This is a schematic diagram illustrating the implementation process of a calibration method for an electronic device provided in an embodiment of this application; Figure 3 This is a schematic diagram of the calibration device provided in the embodiments of this application. Detailed Implementation

[0021] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0022] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0023] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0024] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0025] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0026] Furthermore, in the description of this application, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0027] One embodiment of this application provides a calibration system for electronic devices, used to calibrate sensors of electronic devices such as augmented reality (AR) glasses, virtual reality (VR) glasses, and head-mounted displays (HMDs).

[0028] like Figure 1As shown, the calibration system for the electronic device includes a calibration device 10, a robotic arm 20, a fixture 30, and a calibration chart 40. The robotic arm 20 provides multi-degree-of-freedom motion, such as three-degree-of-freedom translational motion or six-degree-of-freedom translational and rotational motion, thereby providing multi-angle and multi-dimensional dynamic excitation for the calibration of the electronic device. The fixture 30 is mounted on the robotic arm 20, for example, at its end. The fixture 30 has slots for fixing the electronic device 50, the first camera 60, and the second camera 70. The first camera 60 is a human-eye-like camera used to simulate a human eye, and the second camera 70 is used for auxiliary positioning. For electronic devices with two display modules (left and right eye), each display module needs to be calibrated separately, requiring two first cameras 60 and two second cameras 70. The calibration chart 40 can be an AprilGrid array calibration board or a checkerboard calibration board, placed within a preset field of view in front of the robotic arm. The calibration device 10 is used to control the movement of the robotic arm 20 and to acquire images captured by the first camera 60 and the second camera 70, so as to perform the calibration method of the electronic device as described below.

[0029] In one embodiment, the calibration system for the electronic device further includes a clock synchronization controller, which is connected to the electronic device 50, the first camera 60, and the second camera 70, respectively. The clock synchronization controller outputs hardware trigger signals to the electronic device 50, the first camera 60, and the second camera 70. The hardware trigger signals can be pulse signals or pulse width modulation (PWM) signals, used to trigger the sampling times of the electronic device 50, the first camera 60, and the second camera 70 to be unified to the same time base, thereby achieving hardware-level time synchronization and ensuring that the data from heterogeneous sensors are aligned under the same time base, thus improving calibration accuracy.

[0030] The calibration method for the electronic equipment provided in this application will be described below in conjunction with the calibration system for the aforementioned electronic equipment.

[0031] like Figure 2 As shown, an embodiment of this application provides a calibration method for an electronic device including steps S201 to S203.

[0032] S201: Obtain the first pose relationship, which is the relative pose between the first camera and the second camera when no electronic device is placed on the robotic arm, and the first camera and the second camera are fixed on the robotic arm.

[0033] In one embodiment, when no electronic device is fixed on the robotic arm, the robotic arm is controlled to move relative to the calibration map card so that the calibration map card is located within the target depth of field range of the first camera and the second camera (e.g., the calibration map card is adjusted until the calibration map card moves to the optimal depth of field range of the first camera and the second camera). The first camera and the second camera are controlled to simultaneously capture images of the calibration map card to obtain a first image and a second image. The first pose relationship is determined based on the first image and the second image.

[0034] Specifically, for the first image, the pose transformation matrix of the first camera relative to the coordinate system of the calibration chart is obtained based on the coordinates of multiple feature points (e.g., corner points) in the first image and the physical coordinates of the corresponding feature points on the calibration chart in the coordinate system of the calibration chart. This matrix represents the extrinsic parameters of the first camera. Similarly, for the second image, the pose transformation matrix of the second camera relative to the coordinate system of the calibration chart is obtained based on the coordinates of multiple feature points in the second image and the physical coordinates of the corresponding feature points on the calibration chart in the coordinate system of the calibration chart. This matrix represents the extrinsic parameters of the second camera. Based on the extrinsic parameters of the first and second cameras, the first pose relationship between the first and second cameras is obtained, which is the coordinate transformation matrix between the coordinate systems of the first and second cameras.

[0035] It is understandable that the first image and the second image can be one or multiple images. For example, the robotic arm is controlled to move to multiple positions, and at each position, the first and second cameras simultaneously capture images of the calibration chart, resulting in a set of first and second images. The relative pose between the first and second cameras at that position is then calculated. Subsequently, the relative poses between the first and second cameras at multiple positions are averaged, or an optimization algorithm is used to fit the relative poses between the first and second cameras at multiple positions to obtain the first pose relationship.

[0036] In another embodiment, when a set cycle is reached (e.g., the calibration device is turned on daily or a preset number of electronic devices have been calibrated), the first camera and the second camera are controlled to simultaneously capture images of the calibration chart, obtaining a first image and a second image. The first pose relationship is determined based on the first image and the second image and stored. When the set cycle has not been reached, the first pose relationship obtained from the previous calibration is directly retrieved, thereby improving calibration efficiency in the production process.

[0037] S202: When fixing the electronic device on the robotic arm, the relative pose between the first sensor and the second camera inside the electronic device is calibrated to obtain a second pose relationship.

[0038] In one embodiment, the first sensor is a motion sensor, such as an inertial measurement unit (IMU). The electronic device is fixed to a robotic arm, which is then controlled to perform multi-degree-of-freedom movements (e.g., translation, pitch, yaw, roll, or figure-eight trajectory). An image sequence, the first image sequence, is obtained by a second camera capturing images of a calibration chart during the robotic arm's movement. Simultaneously, motion data of the electronic device, including its angular velocity and acceleration, is acquired by the motion sensor during the robotic arm's movement. Based on the motion data of the electronic device and the first image sequence, a second pose relationship is determined, namely, the coordinate transformation matrix between the coordinate system of the motion sensor and the coordinate system of the second camera.

[0039] In one embodiment, after obtaining the motion data of the electronic device and the first image sequence, the Kalibr engine is used to determine the second pose relationship. Specifically, the motion data of the electronic device, the first image sequence, the intrinsic parameters of the second camera, the configuration file of the motion sensor, and the size information of the calibration board are input into the Kalibr engine. The Kalibr engine uses an optimization algorithm to output the second pose relationship. Specifically, the Kalibr engine uses continuous B-splines to model the motion trajectory of the electronic device and constructs a nonlinear least-squares optimization problem. The motion trajectory, the second pose relationship, and the time offset between the motion sensor and the second camera are used as optimization variables. The visual reprojection error of the feature points in the calibration chart in the image captured by the second camera and the measurement error of the motion sensor are used as cost functions to solve the optimization problem, outputting the second pose relationship. This improves the reliability of the obtained second pose relationship.

[0040] In one embodiment, the first sensor is a Simultaneous Localization and Mapping (SLAM) camera. An electronic device is fixed to a robotic arm, and the robotic arm's movement is controlled to acquire a first image sequence obtained by the second camera capturing images of the calibration map during the robotic arm's movement. Simultaneously, a second image sequence obtained by the SLAM camera capturing images of the calibration map during the robotic arm's movement is acquired. Based on the second image sequence and the first image sequence, a second pose relationship is determined.

[0041] Specifically, for any frame in the first image sequence, the pose transformation matrix of the second camera relative to the coordinate system of the calibration chart is obtained based on the coordinates of multiple feature points in that frame and the physical coordinates of the corresponding feature points on the calibration chart in the coordinate system of the calibration chart. This is the extrinsic parameter of the second camera. Similarly, for any frame in the second image sequence, the pose transformation matrix of the SLAM camera relative to the coordinate system of the calibration chart is obtained based on the coordinates of multiple feature points in that frame and the physical coordinates of the corresponding feature points on the calibration chart in the coordinate system of the calibration chart. This is the extrinsic parameter of the SLAM camera. For any robotic arm posture (or position), the images in the first and second image sequences corresponding to that posture are determined. The extrinsic parameters of the second camera are determined based on the images in the first and second image sequences corresponding to that posture. The extrinsic parameters of the SLAM camera are determined based on the images in the second and second image sequences corresponding to that posture. The relative pose of the SLAM camera and the second camera in that posture can be obtained based on the extrinsic parameters of the second camera and the SLAM camera. The relative poses of the SLAM camera and the second camera are calculated under multiple robotic arm poses. The optimal relative poses of the SLAM camera and the second camera are calculated using a nonlinear optimization algorithm, which is the second pose relationship.

[0042] S203: Determine the relative pose between the first sensor and the first camera based on the first pose relationship and the second pose relationship.

[0043] Specifically, both the first pose relation and the second pose relation are coordinate transformation matrices. Based on the spatial coordinate system transfer relation, the relative pose between the first sensor and the first camera is obtained through matrix multiplication. For example, the relative pose between the first sensor and the first camera is determined according to the formula T_eye_sensor = T_eye_aux × T_aux_sensor, where T_eye_sensor represents the relative pose between the first sensor and the first camera, T_eye_aux represents the inverse matrix of the first pose relation, and T_aux_sensor represents the second pose transformation matrix.

[0044] In the above embodiments, when the electronic device is not fixed on the robotic arm, the first and second cameras do not need to photograph the calibration chart through the display module of the electronic device, eliminating the influence of optical distortion and obtaining clear, distortion-free images. This allows for the acquisition of a high-precision first pose relationship between the first and second cameras. Subsequently, the relative pose between the first sensor and the first camera is determined using the second pose relationship between the first sensor and the second camera within the electronic device, as well as the first pose relationship. This avoids feature point extraction errors caused by the first camera photographing through the display module, thus improving calibration accuracy.

[0045] In one embodiment, the electronic device further includes a second sensor, where one of the first and second sensors is a motion sensor and the other is a SLAM camera. When fixing the electronic device to the robotic arm, in addition to calibrating the second pose relationship between the first sensor and the second camera, a third pose relationship between the second sensor and the second camera is also calibrated. Then, based on the first and third pose relationships, the relative pose between the second sensor and the first camera is determined. That is, during the movement of the robotic arm, the calibration device acquires a first image sequence captured by the second camera, motion data of the electronic device collected by the motion sensor, and a second image sequence captured by the SLAM camera. Based on the motion data of the electronic device and the first image sequence, the relative pose between the motion sensor and the second camera is determined; simultaneously, based on the second image sequence and the first image sequence, the relative pose between the SLAM camera and the second camera is determined. Therefore, the calibration of the motion sensor and the SLAM camera in the electronic device can be completed simultaneously during one movement of the robotic arm, improving calibration efficiency.

[0046] In one embodiment, after obtaining the relative poses between the motion sensor and the first camera, and between the SLAM camera and the first camera, the relative pose between the motion sensor and the SLAM camera can also be obtained through matrix multiplication. Writing the relative poses between the motion sensor and the first camera, the relative poses between the SLAM camera and the first camera, and the relative poses between the motion sensor and the SLAM camera into the electronic device can be used for calibration during the use of the electronic device.

[0047] In one embodiment, after obtaining the relative poses between the motion sensor and the first camera, and the relative poses between the SLAM camera and the first camera, a verification image obtained by the SLAM camera from a calibration chart is acquired. Based on the relative poses between the SLAM camera and the first camera, preset calibration points (e.g., corner points in the calibration chart) in the verification image are projected onto a virtual image captured by the first camera. Based on the structure and size information of the calibration chart and the intrinsic parameters of the first camera, the theoretical positions of the calibration points in the virtual image obtained by the first camera from the calibration chart are calculated. The pixel error between the projected position of the calibration point in the virtual image and its theoretical position is determined. If the pixel error is within a set threshold range (e.g., 0.5 pixels), the relative poses between the first sensor and the first camera, and the relative poses between the second sensor and the first camera, are verified; that is, the relative poses between the motion sensor and the first camera, and the relative poses between the SLAM camera and the first camera, are verified. If the pixel error is not within the set threshold range, a recalibration prompt is output.

[0048] In the above embodiments, the calibration points in the calibration chart are projected onto the virtual image of the first camera by the relative pose between the SLAM camera and the first camera. At the same time, the theoretical position of the calibration points in the virtual image is calculated. The obtained relative pose is verified by the pixel error between the projected position of the calibration points in the virtual image and the theoretical position, so as to improve the reliability of the obtained relative pose.

[0049] In one embodiment, the sampling times of the electronic device, the first camera, and the second camera are based on the same time reference. Before determining the first pose relationship, the electronic device, the first camera, and the second camera can be synchronized to the same time reference based on a trigger signal sent by a clock synchronization controller. The electronic device can also send trigger signals to the first camera and the second camera, causing the first camera and the second camera to be synchronized to the same time reference with the electronic device based on the trigger signals, thereby ensuring that heterogeneous sensors are aligned under the same time reference and improving the accuracy of subsequent calibration.

[0050] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0051] Figure 3 This is a schematic diagram of the calibration device provided in the embodiments of this application.

[0052] like Figure 3 As shown, the calibration device 10 of this embodiment includes: a processor 11, a memory 12, and a computer program 13 stored in the memory 12 and executable on the processor 11. When the processor 11 executes the computer program 13, it implements the steps in the calibration method embodiment of the above-described electronic device, for example... Figure 2 Steps S201 to S203 are shown.

[0053] For example, the computer program 13 may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 11 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 13 in the calibration device.

[0054] Those skilled in the art will understand that Figure 3 This is merely an example of a calibration device and does not constitute a limitation on the calibration device. It may include more or fewer components than shown, or combine certain components, or different components. For example, the calibration device may also include input / output devices, network access devices, buses, etc.

[0055] The processor 11 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0056] The memory 12 can be an internal storage unit of the calibration device, such as the hard drive or memory of the calibration device. The memory 12 can also be an external storage device of the calibration device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the calibration device. Furthermore, the memory 12 can include both internal and external storage units of the calibration device. The memory 12 is used to store the computer program and other programs and data required by the calibration device. The memory 12 can also be used to temporarily store data that has been output or will be output.

[0057] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0058] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0059] In the embodiments provided in this application, it should be understood that the disclosed apparatus / calibration devices and methods can be implemented in other ways. For example, the apparatus / calibration device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0060] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0061] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0062] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0063] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0064] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A calibration method for an electronic device, characterized in that, include: Obtain the first pose relationship, which is the relative pose between the first camera and the second camera when no electronic device is placed on the robotic arm, and the first camera and the second camera are fixed on the robotic arm. When fixing the electronic device on the robotic arm, the relative pose between the first sensor and the second camera inside the electronic device is calibrated to obtain a second pose relationship. The relative pose between the first sensor and the first camera is determined based on the first pose relationship and the second pose relationship.

2. The calibration method for electronic devices according to claim 1, characterized in that, The process of obtaining the first pose relation includes: When the electronic device is not fixed to the robotic arm, the robotic arm is controlled to move relative to the calibration map card so that the calibration map card is located within the target depth of field range of the first camera and the second camera. The first camera and the second camera are controlled to simultaneously capture images of the calibration chart to obtain a first image and a second image. The first pose relationship is determined based on the first image and the second image.

3. The calibration method for electronic devices according to claim 1, characterized in that, The first sensor is a motion sensor, and the step of calibrating the relative pose between the first sensor and the second camera within the electronic device to obtain a second pose relationship includes: Control the movement of the robotic arm; Acquire a first image sequence and motion data of the electronic device. The first image sequence is an image sequence obtained by the second camera capturing a calibration chart during the movement of the robotic arm. The motion data is data collected by the motion sensor during the movement of the robotic arm. The second pose relationship is determined based on the motion data and the first image sequence.

4. The calibration method for electronic devices according to claim 1, characterized in that, The first sensor is a real-time localization and mapping (RTD) camera. The step of calibrating the relative pose between the first sensor and the second camera within the electronic device to obtain a second pose relationship includes: Control the movement of the robotic arm; Acquire a second image sequence and a first image sequence, wherein the second image sequence is an image sequence obtained by the real-time localization and mapping camera capturing images of the calibration map during the movement of the robotic arm, and the first image sequence is an image sequence obtained by the second camera capturing images of the calibration map during the movement of the robotic arm; The second pose relationship is determined based on the second image sequence and the first image sequence.

5. The calibration method for electronic devices according to claim 1, characterized in that, The electronic device further includes a second sensor, wherein one of the first and second sensors is a motion sensor and the other is a real-time localization and mapping camera, and the calibration method further includes: When the electronic device is fixed on the robotic arm, the relative pose between the second sensor and the second camera is calibrated to obtain a third pose relationship. The relative pose between the second sensor and the first camera is determined based on the first pose relationship and the third pose relationship.

6. The calibration method for an electronic device according to any one of claims 1 to 5, characterized in that, The sampling times of the electronic device, the first camera, and the second camera are based on the same time reference.

7. The calibration method for an electronic device according to claim 5, characterized in that, After determining the relative pose between the second sensor and the first camera based on the first pose relationship and the third pose relationship, the method further includes: Obtain the verification image of the calibration map card captured by the real-time positioning and mapping camera; Based on the real-time positioning and mapping, the relative pose between the camera and the first camera is constructed, and the preset calibration points in the verification image are projected onto the virtual image captured by the first camera. Determine the pixel error between the projected position of the calibration point in the virtual image and the theoretical position of the calibration point in the virtual image; If the pixel error is within a set threshold range, the relative pose between the first sensor and the first camera and the relative pose between the second sensor and the first camera are verified. If the pixel error is not within the set threshold range, a recalibration prompt message will be output.

8. A calibration device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the calibration method of the electronic device as described in any one of claims 1 to 7.

9. A calibration system for an electronic device, characterized in that, include: Robotic arms are used to provide multi-degree-of-freedom motion; A clamp, mounted on the robotic arm, is used to secure the electronic device, the first camera, and the second camera. The calibration chart is placed within a preset field of view in front of the robotic arm; And the calibration device as described in claim 8.

10. The calibration system for an electronic device according to claim 9, characterized in that, The calibration system also includes a clock synchronization controller, which is connected to the electronic device, the first camera, and the second camera respectively. The clock synchronization controller is used to output a hardware trigger signal to the electronic device, the first camera and the second camera, and the hardware trigger signal is used to trigger the sampling time of the electronic device, the first camera and the second camera to be unified to the same time base.

11. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the calibration method of the electronic device as described in any one of claims 1 to 7.

Images