Combined calibration method, combined calibration control device, electronic equipment and medium

Abstract

The present application discloses a joint calibration method, a joint calibration control device, an electronic device and a medium. The joint calibration method includes: controlling a turntable to drive smart glasses and a multi-degree-of-freedom device to synchronously perform posture transformation motions in each degree of freedom, and collecting the first inertial measurement unit data of the smart glasses and the second inertial measurement unit data of the multi-degree-of-freedom device through a data acquisition component when performing posture transformation motions in each degree of freedom; obtaining the first coordinate system conversion coefficient between the world coordinate system where the smart glasses are located and the local coordinate system where the multi-degree-of-freedom device is located in each degree of freedom according to the first inertial measurement unit data and the second inertial measurement unit data; fusing and calibrating multiple first coordinate system conversion coefficients corresponding to all degrees of freedom through a fusion algorithm model to obtain joint calibration data. According to the joint calibration method, the smart glasses and the multi-degree-of-freedom device can be jointly calibrated in coordinate system.

Description

Technical Field

[0001] The present application relates to the field of smart glasses calibration, and in particular to a joint calibration method, a joint calibration control device, an electronic device and a medium. Background Art

[0002] With the development of science and technology, smart glasses, as a kind of intelligent wearable device, have been applied in various usage scenarios in people's lives. Smart glasses are devices that can be worn on the user's head, have a similar appearance to glasses, and combine virtual information display technologies such as augmented reality (AR) or virtual reality (VR).

[0003] In some scenarios where smart glasses are used, some operations on the smart glasses need to be triggered by the position changes of other auxiliary devices. However, other auxiliary devices and smart glasses are separately set up and each has its own coordinate system of position data. It is still impossible to achieve accurate operation of smart glasses through auxiliary devices just by connecting the two. Summary of the invention

[0004] The present application provides a joint calibration method, a joint calibration control device, an electronic device and a medium, which can jointly calibrate the coordinate systems of smart glasses and multi-degree-of-freedom devices. After calibration, it is convenient to trigger precise operation control of the smart glasses through the multi-degree-of-freedom device.

[0005] In a first aspect, an embodiment of the present application provides a joint calibration method, which performs joint calibration on smart glasses and a multi-degree-of-freedom device through a joint calibration device, wherein the joint calibration device includes a turntable and a data acquisition component, the smart glasses and the multi-degree-of-freedom device are installed on the turntable, and the multi-degree-of-freedom device is communicatively connected with the smart glasses, and the joint calibration method includes: controlling the turntable to drive the smart glasses and the multi-degree-of-freedom device to synchronously perform posture transformation movements in each degree of freedom, and collecting first inertial measurement unit data (Inertial Measurement Unit, IMU) of the smart glasses and second inertial measurement unit data of the multi-degree-of-freedom device through the data acquisition component when performing posture transformation movements in each degree of freedom; obtaining a first coordinate system conversion coefficient between a world coordinate system where the smart glasses are located and a local coordinate system where the multi-degree-of-freedom device is located in each degree of freedom according to the first inertial measurement unit data and the second inertial measurement unit data; and fusing and calibrating multiple first coordinate system conversion coefficients corresponding to all degrees of freedom through a fusion algorithm model to obtain joint calibration data, and the joint calibration data is the transformation relationship data between the world coordinate system and the local coordinate system.

[0006] According to the aforementioned implementation of the first aspect of the present application, the first coordinate system conversion coefficient between the world coordinate system where the smart glasses are located in each degree of freedom and the local coordinate system where the multi-degree-of-freedom device is located is obtained according to the first inertial measurement unit data and the second inertial measurement unit data, including: obtaining a first posture angle, a first acceleration, and a first angular velocity corresponding to the preset posture transformation movement of the smart glasses in a preset degree of freedom based on the first inertial measurement unit data; obtaining a first coordinate system according to the first posture angle, the first acceleration, and the first angular velocity, the first coordinate system being the world coordinate system of the smart glasses in the degree of freedom corresponding to the preset posture transformation movement; obtaining a second posture angle, a second acceleration, and a second angular velocity corresponding to the multi-degree-of-freedom device in the preset posture transformation movement based on the second inertial measurement unit data; obtaining a second coordinate system according to the second posture angle, the second acceleration, and the second angular velocity, the second coordinate system being the local coordinate system of the multi-degree-of-freedom device based on the world coordinate system in the degree of freedom corresponding to the preset posture transformation movement; performing a matrix rotation transformation based on the first coordinate system and the second coordinate system to obtain the first coordinate system conversion coefficient under the corresponding degree of freedom.

[0007] According to any of the aforementioned implementations of the first aspect of the present application, obtaining a first coordinate system conversion coefficient between a world coordinate system where the smart glasses are located and a local coordinate system where the multi-degree-of-freedom device is located in each degree of freedom according to the first inertial measurement unit data and the second inertial measurement unit data includes: performing a first preprocessing on the first inertial measurement unit data; obtaining a first posture angle, a first acceleration, and a first angular velocity corresponding to the smart glasses in a preset posture transformation motion in a preset degree of freedom based on the first inertial measurement unit data that has completed the first preprocessing; obtaining a first coordinate system according to the first posture angle, the first acceleration, and the first angular velocity, the first coordinate system being the world coordinate system of the smart glasses in the degree of freedom corresponding to the preset posture transformation motion; obtaining a second posture angle, a second acceleration, and a second angular velocity corresponding to the multi-degree-of-freedom device in the preset posture transformation motion based on the second inertial measurement unit data; obtaining a second coordinate system according to the second posture angle, the second acceleration, and the second angular velocity, the second coordinate system being the local coordinate system of the multi-degree-of-freedom device based on the world coordinate system in the degree of freedom corresponding to the preset posture transformation motion; performing a matrix rotation transformation based on the first coordinate system and the second coordinate system to obtain a first coordinate system conversion coefficient in the corresponding degree of freedom.

[0008] According to any of the aforementioned implementations of the first aspect of the present application, the first preprocessing of the first inertial measurement unit data includes: performing a first denoising process, a first dynamic compensation process and a first correction process on the first inertial measurement unit data.

[0009] According to any of the aforementioned embodiments of the first aspect of the present application, obtaining the first posture angle, first acceleration, and first angular velocity corresponding to the preset posture transformation movement of the smart glasses under a preset degree of freedom based on the first inertial measurement unit data that has completed the first preprocessing includes: obtaining the first posture angle through a first posture algorithm model based on the first inertial measurement unit data that has completed the first preprocessing; obtaining the first acceleration and first angular velocity through a first motion parameter algorithm model based on the first inertial measurement unit data that has completed the first preprocessing.

[0010] According to any of the aforementioned implementations of the first aspect of the present application, the joint calibration method also includes: obtaining the size of the virtual image and the spatial position coordinate system of each area of ​​the virtual image relative to the world coordinate system based on the eye frame size and the combined image distance of the smart glasses; and controlling the smart glasses to display the virtual image based on the size of the virtual image and the spatial position coordinate system of each area of ​​the virtual image relative to the world coordinate system.

[0011] According to any of the aforementioned embodiments of the first aspect of the present application, controlling the turntable to drive the smart glasses and the multi-degree-of-freedom device to synchronously perform posture change movements in each degree of freedom includes: controlling the turntable to drive the smart glasses and the multi-degree-of-freedom device to synchronously perform rotational movement along a first axis, synchronously perform rotational movement along a second axis, and synchronously perform rotational movement along a third axis, and the first axis, the second axis, and the third axis are orthogonal to each other.

[0012] In a second aspect, an embodiment of the present application provides a joint calibration control device for controlling a joint calibration device to perform joint calibration on smart glasses and a multi-degree-of-freedom device, wherein the joint calibration device comprises a turntable and a data acquisition component, the smart glasses and the multi-degree-of-freedom device are mounted on the turntable, and the multi-degree-of-freedom device is communicatively connected with the smart glasses, and the joint calibration control device comprises: a turntable control module for controlling the turntable to drive the smart glasses and the multi-degree-of-freedom device to synchronously perform posture transformation motion in each degree of freedom; a data acquisition control module for acquiring data through the data acquisition component when performing posture transformation motion in each degree of freedom. The invention relates to a method for obtaining a first inertial measurement unit data of the smart glasses and a second inertial measurement unit data of the multi-degree-of-freedom device; a conversion coefficient acquisition module, used for obtaining a first coordinate system conversion coefficient between a world coordinate system where the smart glasses are located and a local coordinate system where the multi-degree-of-freedom device is located under each degree of freedom according to the first inertial measurement unit data and the second inertial measurement unit data; and a fusion calibration module, used for fusing and calibrating a plurality of the first coordinate system conversion coefficients corresponding to all degrees of freedom through a fusion algorithm model to obtain joint calibration data, wherein the joint calibration data is transformation relationship data between the world coordinate system and the local coordinate system.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, comprising a controller, the controller comprising a memory and at least one processor, the memory storing instructions, the at least one processor calling the instructions in the memory so that the controller executes a joint calibration method according to any of the aforementioned implementations of the first aspect of the present application.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having instructions stored thereon, wherein when the instructions are executed by a processor, a joint calibration method according to any of the aforementioned embodiments of the first aspect of the present application is implemented.

[0015] According to the joint calibration method of the embodiment of the present application, the smart glasses and the multi-degree-of-freedom device are jointly calibrated by the joint calibration device. When the joint calibration starts, the multi-degree-of-freedom device is connected to the smart glasses for communication. The control turntable drives the smart glasses and the multi-degree-of-freedom device to synchronously perform posture transformation motions in each degree of freedom, and collects the first inertial measurement unit data of the smart glasses and the second inertial measurement unit data of the multi-degree-of-freedom device through the data acquisition component when performing posture transformation motions in each degree of freedom. Then, the first coordinate system conversion coefficient between the world coordinate system where the smart glasses are located and the local coordinate system where the multi-degree-of-freedom device is located under each degree of freedom is obtained according to the first inertial measurement unit data and the second inertial measurement unit data, so that multiple first coordinate system conversion coefficients corresponding to multiple degrees of freedom can be obtained in sequence. Afterwards, the multiple first coordinate system conversion coefficients corresponding to all degrees of freedom are fused and calibrated by the fusion algorithm model to obtain joint calibration data, which is the transformation relationship data between the world coordinate system and the local coordinate system, and completes the joint calibration of the smart glasses and the multi-degree-of-freedom device. After completing the joint calibration, the position data of the multi-degree-of-freedom device can be accurately converted into coordinate data in the world coordinate system of the smart glasses based on the joint calibration data, thereby facilitating the precise operation control of the smart glasses triggered by the multi-degree-of-freedom device. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0017] Figure 1 This is a flow chart of an embodiment of the joint calibration method of the present application;

[0018] Figure 2 A flowchart of obtaining a first coordinate system conversion coefficient between a world coordinate system where the smart glasses are located and a local coordinate system where the multi-degree-of-freedom device is located in each degree of freedom according to first inertial measurement unit data and second inertial measurement unit data in an embodiment of the joint calibration method of the present application;

[0019] Figure 3 This is a structural schematic diagram of an embodiment of a joint calibration control device of the present application;

[0020] Figure 4 This is a schematic diagram of the hardware structure of a controller of an embodiment of the electronic device of the present application.

[0021] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0023] It should be noted that all directional indications in the embodiments of the present application (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0024] In addition, the descriptions of "first", "second", etc. in this application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0025] An embodiment of the present application provides a joint calibration method. The joint calibration method jointly calibrates smart glasses and multi-degree-of-freedom devices through a joint calibration device. In this embodiment, the smart glasses are, for example, augmented reality (AR) glasses. In some other embodiments, the smart glasses may also be virtual reality (VR) glasses and other head-mounted display devices with a glasses-like appearance. In this embodiment, the multi-degree-of-freedom device is, for example, a stylus. In some other embodiments, the multi-degree-of-freedom device may also be other devices that are separate from the smart glasses and can generate inertial measurement unit data (IMU), such as smart bracelets, handles, etc.

[0026] The joint calibration device includes a turntable and a data acquisition component. When the smart glasses and the multi-degree-of-freedom device are jointly calibrated by the joint calibration device, the smart glasses and the multi-degree-of-freedom device are installed on the turntable, and the multi-degree-of-freedom device is connected to the smart glasses for communication.

[0027] In some embodiments, the joint calibration device further includes a base and a carrier. The base is, for example, a marble carrier. The turntable is mounted on the base. The smart glasses and the multi-degree-of-freedom device can be mounted on the carrier, and thus mounted on the turntable through the carrier. In some embodiments, when the smart glasses and the multi-degree-of-freedom device are jointly calibrated by the joint calibration device, the smart glasses and the multi-degree-of-freedom device are horizontally mounted on the turntable through the carrier.

[0028] In some embodiments, the turntable includes a movable bearing part and a driving component, the movable bearing part is used to bear or connect the smart glasses and the multi-degree-of-freedom device, and the driving component is used to drive the movable bearing part to move along multiple degrees of freedom. In one example, the driving component includes at least two servo motors to enable the turntable to drive the smart glasses and the multi-degree-of-freedom device to move along six degrees of freedom.

[0029] The data acquisition component includes an inertial measurement unit data (IMU) acquisition component. When the smart glasses and the multi-degree-of-freedom device are jointly calibrated by a joint calibration device, the data acquisition component is connected to the smart glasses and the multi-degree-of-freedom device in communication, such as a wired communication connection.

[0030] Figure 1 This is a flow chart of an embodiment of a joint calibration method of the present application. The joint calibration method includes steps S110 to S130.

[0031] In step S110, the control turntable drives the smart glasses and the multi-degree-of-freedom device to synchronously perform posture change movements in each degree of freedom, and collects the first inertial measurement unit data of the smart glasses and the second inertial measurement unit data of the multi-degree-of-freedom device through the data acquisition component when performing posture change movements in each degree of freedom.

[0032] In some embodiments, the above-mentioned control turntable drives the smart glasses and the multi-degree-of-freedom device to synchronously perform posture change movements in each degree of freedom, including: the control turntable drives the smart glasses and the multi-degree-of-freedom device to synchronously perform rotational movements along the first axis, synchronously perform rotational movements along the second axis, and synchronously perform rotational movements along the third axis, and the first axis, the second axis, and the third axis are orthogonal to each other.

[0033] The first axis, the second axis, and the third axis are, for example, the X axis, the Y axis, and the Z axis, respectively. Therefore, in one example, step S110 drives the smart glasses and the multi-degree-of-freedom device to synchronously perform rotational motion along the first axis (X axis) by controlling the turntable, so as to obtain a set of first inertial measurement unit data and second inertial measurement unit data under the degree of freedom of rotational motion along the first axis (X axis); drives the smart glasses and the multi-degree-of-freedom device to synchronously perform rotational motion along the second axis (Y axis) by controlling the turntable, so as to obtain a set of first inertial measurement unit data and second inertial measurement unit data under the degree of freedom of rotational motion along the second axis (Y axis); drives the smart glasses and the multi-degree-of-freedom device to synchronously perform rotational motion along the third axis (Z axis) by controlling the turntable, so as to obtain a set of first inertial measurement unit data and second inertial measurement unit data under the degree of freedom of rotational motion along the third axis (Z axis).

[0034] In step S120, a first coordinate system conversion coefficient between a world coordinate system where the smart glasses are located and a local coordinate system where the multi-degree-of-freedom device is located in each degree of freedom is obtained according to the first inertial measurement unit data and the second inertial measurement unit data.

[0035] As described above, after obtaining multiple groups of first inertial measurement unit data and second inertial measurement unit data corresponding to multiple degrees of freedom of rotational motion along the first axis (X axis), rotational motion along the second axis (Y axis), and rotational motion about the third axis (Z axis) in sequence in step S110, in step S120, multiple first coordinate system conversion coefficients corresponding to multiple degrees of freedom of rotational motion along the first axis (X axis), rotational motion along the second axis (Y axis), and rotational motion about the third axis (Z axis) in sequence can be obtained.

[0036] Figure 2 The present invention is a flowchart of obtaining a first coordinate system conversion coefficient between a world coordinate system where the smart glasses are located and a local coordinate system where the multi-degree-of-freedom device is located in each degree of freedom according to the first inertial measurement unit data and the second inertial measurement unit data in an embodiment of the joint calibration method of the present application. In some embodiments, step S120 of obtaining a first coordinate system conversion coefficient between a world coordinate system where the smart glasses are located and a local coordinate system where the multi-degree-of-freedom device is located in each degree of freedom according to the first inertial measurement unit data and the second inertial measurement unit data may include steps S121 to S127.

[0037] In step S121 , first preprocessing is performed on the first inertial measurement unit data.

[0038] In some embodiments, the step S121 of performing a first preprocessing on the first inertial measurement unit data includes: performing a first denoising process, a first dynamic compensation process, and a first correction process on the first inertial measurement unit data to improve the data accuracy and data stability of the joint calibration process. In some embodiments, the first preprocessing on the first inertial measurement unit data may be preprocessing the first inertial measurement unit data through a first Kalman filter algorithm.

[0039] In step S122, based on the first inertial measurement unit data that has completed the first preprocessing, a first posture angle, a first acceleration, and a first angular velocity corresponding to the preset posture transformation movement of the smart glasses under a preset degree of freedom are obtained.

[0040] In some embodiments, based on the first inertial measurement unit data that has completed the first preprocessing, the step S122 of obtaining the first posture angle, the first acceleration, and the first angular velocity corresponding to the preset posture transformation movement of the smart glasses under a preset degree of freedom includes: based on the first inertial measurement unit data that has completed the first preprocessing, obtaining the first posture angle through the first posture algorithm model; based on the first inertial measurement unit data that has completed the first preprocessing, obtaining the first acceleration and the first angular velocity through the first motion parameter algorithm model. In some embodiments, the first posture angle is obtained by integrating or analyzing the first posture algorithm model.

[0041] In step S123, a first coordinate system is obtained according to the first posture angle, the first acceleration, and the first angular velocity. The first coordinate system is a world coordinate system of the smart glasses under the degree of freedom corresponding to the preset posture transformation movement.

[0042] In step S124 , second pre-processing is performed on the second inertial measurement unit data.

[0043] In some embodiments, the step S124 of performing a second preprocessing on the second inertial measurement unit data includes: performing a second denoising process, a second dynamic compensation process, and a second correction process on the second inertial measurement unit data to improve the data accuracy and data stability of the joint calibration process. In some embodiments, the second preprocessing on the second inertial measurement unit data may be preprocessing the second inertial measurement unit data through a second Kalman filter algorithm.

[0044] In step S125, based on the second inertial measurement unit data that has completed the second preprocessing, a second posture angle, a second acceleration, and a second angular velocity corresponding to the multi-degree-of-freedom device in the preset posture transformation movement are obtained.

[0045] In some embodiments, based on the second inertial measurement unit data that has completed the second preprocessing, the step S125 of obtaining the second posture angle, second acceleration, and second angular velocity corresponding to the multi-degree-of-freedom device in the preset posture transformation movement includes: based on the second inertial measurement unit data that has completed the second preprocessing, obtaining the second posture angle through the second posture algorithm model; based on the second inertial measurement unit data that has completed the second preprocessing, obtaining the second acceleration and second angular velocity through the second motion parameter algorithm model. In some embodiments, the second posture angle is obtained by integrating or analyzing the second posture algorithm model.

[0046] In step S126, a second coordinate system is obtained according to the second posture angle, the second acceleration, and the second angular velocity. The second coordinate system is a local coordinate system based on the world coordinate system for the multi-degree-of-freedom device in the degree of freedom corresponding to the preset posture transformation movement.

[0047] In step S127, a matrix rotation transformation is performed based on the first coordinate system and the second coordinate system to obtain a first coordinate system conversion coefficient under corresponding degrees of freedom.

[0048] In some embodiments, specifically, according to a matrix rotation transformation method from the first coordinate system to the second coordinate system, a matrix rotation calculation is performed to obtain the first coordinate system conversion coefficient.

[0049] In the above embodiment, the first preprocessing is performed on the first inertial measurement unit data, and the second preprocessing is performed on the second inertial measurement unit data. In some other embodiments, the first preprocessing and the second preprocessing steps may not be included. That is, in some embodiments, step S120 of obtaining the first coordinate system conversion coefficient between the world coordinate system where the smart glasses are located in each degree of freedom and the local coordinate system where the multi-degree-of-freedom device is located according to the first inertial measurement unit data and the second inertial measurement unit data may include the following steps: based on the first inertial measurement unit data, obtaining the first posture angle, the first acceleration, and the first angular velocity corresponding to the preset posture transformation movement of the smart glasses in a preset degree of freedom; obtaining the first coordinate system according to the first posture angle, the first acceleration, and the first angular velocity, the first coordinate system being the world coordinate system of the smart glasses in the degree of freedom corresponding to the preset posture transformation movement; based on the second inertial measurement unit data, obtaining the second posture angle, the second acceleration, and the second angular velocity corresponding to the multi-degree-of-freedom device in the preset posture transformation movement; obtaining the second coordinate system according to the second posture angle, the second acceleration, and the second angular velocity, the second coordinate system being the local coordinate system based on the world coordinate system of the multi-degree-of-freedom device in the degree of freedom corresponding to the preset posture transformation movement; performing matrix rotation transformation based on the first coordinate system and the second coordinate system to obtain the first coordinate system conversion coefficient under the corresponding degree of freedom.

[0050] In step S130, a plurality of first coordinate system conversion coefficients corresponding to all degrees of freedom are fused and calibrated through a fusion algorithm model to obtain joint calibration data, where the joint calibration data is transformation relationship data between the world coordinate system and the local coordinate system.

[0051] In some embodiments, the fusion algorithm model is a Kalman filter fusion algorithm model or an extended Kalman filter fusion algorithm model.

[0052] In some embodiments, the joint calibration data is expressed as the following functional relationship: DOF_xyz=R*AR_xyz+T, wherein AR_xyz is the world coordinate system; DOF_xyz is the local coordinate system; R is the second coordinate system conversion coefficient, and the second coordinate system conversion coefficient R is calculated by a fusion algorithm model through a rotation transformation from the world coordinate system AR_xyz to the local coordinate system DOF_xyz; T is the coordinate of the origin of the world coordinate system AR_xyz in the local coordinate system DOF_xyz.

[0053] In some embodiments, the joint calibration method further includes: obtaining the size of the virtual image screen and the spatial position coordinate system of each area of ​​the virtual image screen relative to the world coordinate system according to the eyebox size and the combined image distance of the smart glasses; based on the size of the virtual image screen and the spatial position coordinate system of each area of ​​the virtual image screen relative to the world coordinate system, controlling the smart glasses to display the virtual image screen. Preferably, the spatial position coordinate system of each area of ​​the virtual image screen relative to the world coordinate system is unified with the world coordinate system AR_xyz.

[0054] In some embodiments, the joint calibration method further includes: when the multi-degree-of-freedom device generates specific position change data, triggering a preset operation on the smart glasses according to the joint calibration data. When the multi-degree-of-freedom device is a stylus, the above step is: when the stylus generates specific position change data, triggering a preset operation on the smart glasses according to the joint calibration data. Therefore, according to the joint calibration data, the correspondence between the world coordinate system AR_xyz of the smart glasses and the local coordinate system DOF_xyz of the multi-degree-of-freedom device can be accurately expressed. By changing the position change data of the multi-degree-of-freedom device and presetting the corresponding judgment logic, the application function of the smart glasses can be accurately triggered by the multi-degree-of-freedom device, and the operation control of the smart glasses by the multi-degree-of-freedom device can be accurately realized.

[0055] According to the joint calibration method of the embodiment of the present application, the smart glasses and the multi-degree-of-freedom device are jointly calibrated by the joint calibration device. When the joint calibration starts, the multi-degree-of-freedom device is connected to the smart glasses for communication. The control turntable drives the smart glasses and the multi-degree-of-freedom device to synchronously perform posture transformation motions in each degree of freedom, and collects the first inertial measurement unit data of the smart glasses and the second inertial measurement unit data of the multi-degree-of-freedom device through the data acquisition component when performing posture transformation motions in each degree of freedom. Then, the first coordinate system conversion coefficient between the world coordinate system where the smart glasses are located and the local coordinate system where the multi-degree-of-freedom device is located under each degree of freedom is obtained according to the first inertial measurement unit data and the second inertial measurement unit data, so that multiple first coordinate system conversion coefficients corresponding to multiple degrees of freedom can be obtained in sequence. Afterwards, the multiple first coordinate system conversion coefficients corresponding to all degrees of freedom are fused and calibrated by the fusion algorithm model to obtain joint calibration data, which is the transformation relationship data between the world coordinate system and the local coordinate system, and completes the joint calibration of the smart glasses and the multi-degree-of-freedom device. After completing the joint calibration, the position data of the multi-degree-of-freedom device can be accurately converted into coordinate data in the world coordinate system of the smart glasses based on the joint calibration data, thereby facilitating the precise operation control of the smart glasses triggered by the multi-degree-of-freedom device.

[0056] The embodiment of the present application provides a joint calibration control device, which is used to control a joint calibration device to perform joint calibration on smart glasses and a multi-degree-of-freedom device. The joint calibration device includes a turntable and a data acquisition component, the smart glasses and the multi-degree-of-freedom device are installed on the turntable, and the multi-degree-of-freedom device is communicatively connected with the smart glasses.

[0057] Figure 3 The schematic diagram of the structure of an embodiment of the combined calibration control device of the present application is as follows. The combined calibration control device comprises a turntable control module 110 , a data acquisition control module 120 , a conversion coefficient acquisition module 130 , and a fusion calibration module 140 .

[0058] The turntable control module 110 is used to control the turntable to drive the smart glasses and the multi-degree-of-freedom device to synchronously perform posture change movements in each degree of freedom.

[0059] The data acquisition control module 120 is used to collect the first inertial measurement unit data of the smart glasses and the second inertial measurement unit data of the multi-degree-of-freedom device through the data acquisition component when performing posture transformation movement in each degree of freedom.

[0060] The conversion coefficient acquisition module 130 is used to obtain the first coordinate system conversion coefficient between the world coordinate system where the smart glasses are located and the local coordinate system where the multi-degree-of-freedom device is located in each degree of freedom according to the first inertial measurement unit data and the second inertial measurement unit data.

[0061] The fusion calibration module 140 is used to perform fusion calibration on multiple first coordinate system conversion coefficients corresponding to all degrees of freedom through a fusion algorithm model to obtain joint calibration data, which is the transformation relationship data between the world coordinate system and the local coordinate system.

[0062] According to the joint calibration control device of the embodiment of the present application, the joint calibration device can be controlled to perform joint calibration on the smart glasses and the multi-degree-of-freedom device. When the joint calibration starts, the multi-degree-of-freedom device is connected to the smart glasses for communication. The turntable control module 110 controls the turntable to drive the smart glasses and the multi-degree-of-freedom device to synchronously perform posture transformation movements in each degree of freedom. At the same time, the data acquisition control module 120 collects the first inertial measurement unit data of the smart glasses and the second inertial measurement unit data of the multi-degree-of-freedom device through the data acquisition component when performing posture transformation movements in each degree of freedom. The conversion coefficient acquisition module 130 obtains the first coordinate system conversion coefficient between the world coordinate system where the smart glasses are located and the local coordinate system where the multi-degree-of-freedom device is located in each degree of freedom according to the first inertial measurement unit data and the second inertial measurement unit data, so that multiple first coordinate system conversion coefficients corresponding to multiple degrees of freedom can be obtained in sequence. The fusion calibration module 140 performs fusion calibration on multiple first coordinate system conversion coefficients corresponding to all degrees of freedom through a fusion algorithm model to obtain joint calibration data, which is the transformation relationship data between the world coordinate system and the local coordinate system, and completes the joint calibration of the smart glasses and the multi-degree-of-freedom device. After the joint calibration is completed, the position data of the multi-degree-of-freedom device can be accurately converted into coordinate data in the world coordinate system of the smart glasses according to the joint calibration data, so as to facilitate the precise operation control of the smart glasses triggered by the multi-degree-of-freedom device.

[0063] An embodiment of the present application also provides an electronic device, which includes a controller. Figure 4 The hardware structure diagram of the controller of an embodiment of the electronic device of the present application is shown in FIG. The controller includes a memory 410 and at least one processor 420. The memory 410 stores instructions, and at least one processor 420 calls the instructions in the memory 410 so that the controller executes the joint calibration method according to any of the aforementioned embodiments of the present application.

[0064] The joint calibration method performs joint calibration on the smart glasses and the multi-degree-of-freedom device through a joint calibration device. The joint calibration device includes a turntable and a data acquisition component. The smart glasses and the multi-degree-of-freedom device are installed on the turntable, and the multi-degree-of-freedom device is communicatively connected with the smart glasses.

[0065] The joint calibration method includes: controlling a turntable to drive the smart glasses and the multi-degree-of-freedom device to synchronously perform posture transformation movements in each degree of freedom, and collecting first inertial measurement unit data of the smart glasses and second inertial measurement unit data of the multi-degree-of-freedom device through a data acquisition component when performing posture transformation movements in each degree of freedom; obtaining a first coordinate system conversion coefficient between a world coordinate system where the smart glasses are located and a local coordinate system where the multi-degree-of-freedom device is located in each degree of freedom according to the first inertial measurement unit data and the second inertial measurement unit data; and fusing and calibrating multiple first coordinate system conversion coefficients corresponding to all degrees of freedom through a fusion algorithm model to obtain joint calibration data, which is the transformation relationship data between the world coordinate system and the local coordinate system.

[0066] Specifically, the processor 420 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0067] The memory 410 may include a large capacity memory 410 for data or instructions. By way of example and not limitation, the memory 410 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 410 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 410 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 410 is a non-volatile solid-state memory. In a specific embodiment, the memory 410 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0068] In one example, the control device may further include a communication interface 430 and a bus 440. The processor 420, the memory 410, and the communication interface 430 are connected via the bus 440 and communicate with each other.

[0069] The communication interface 430 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0070] Bus 440 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-end bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory 410 bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 440 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0071] In addition, in combination with the joint calibration method in the above embodiments, the present application embodiment may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores instructions, and when the instructions are executed by a processor, any one of the joint calibration methods in the above embodiments is implemented.

[0072] The joint calibration method jointly calibrates the smart glasses and the multi-degree-of-freedom device through a joint calibration device. The joint calibration device includes a turntable and a data acquisition component. The smart glasses and the multi-degree-of-freedom device are installed on the turntable, and the multi-degree-of-freedom device is communicatively connected with the smart glasses.

[0073] The joint calibration method performs joint calibration on the smart glasses and the multi-degree-of-freedom device through a joint calibration device. The joint calibration device includes a turntable and a data acquisition component. The smart glasses and the multi-degree-of-freedom device are installed on the turntable, and the multi-degree-of-freedom device is communicatively connected with the smart glasses.

[0074] The joint calibration method includes: controlling a turntable to drive the smart glasses and the multi-degree-of-freedom device to synchronously perform posture transformation movements in each degree of freedom, and collecting first inertial measurement unit data of the smart glasses and second inertial measurement unit data of the multi-degree-of-freedom device through a data acquisition component when performing posture transformation movements in each degree of freedom; obtaining a first coordinate system conversion coefficient between a world coordinate system where the smart glasses are located and a local coordinate system where the multi-degree-of-freedom device is located in each degree of freedom according to the first inertial measurement unit data and the second inertial measurement unit data; and fusing and calibrating multiple first coordinate system conversion coefficients corresponding to all degrees of freedom through a fusion algorithm model to obtain joint calibration data, which is the transformation relationship data between the world coordinate system and the local coordinate system.

[0075] The present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0076] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0077] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0078] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A joint calibration method, characterized in that: The smart glasses and the multi-degree-of-freedom device are jointly calibrated by a joint calibration device, the joint calibration device includes a turntable and a data acquisition component, the smart glasses and the multi-degree-of-freedom device are installed on the turntable, the multi-degree-of-freedom device is communicatively connected with the smart glasses, and the joint calibration method includes: Controlling the turntable to drive the smart glasses and the multi-degree-of-freedom device to synchronously perform posture change motion in each degree of freedom, and collecting first inertial measurement unit data of the smart glasses and second inertial measurement unit data of the multi-degree-of-freedom device through the data acquisition component when performing posture change motion in each degree of freedom; Obtaining a first coordinate system conversion coefficient between a world coordinate system where the smart glasses are located and a local coordinate system where the multi-degree-of-freedom device is located in each degree of freedom according to the first inertial measurement unit data and the second inertial measurement unit data; The plurality of first coordinate system conversion coefficients corresponding to all degrees of freedom are fused and calibrated through a fusion algorithm model to obtain joint calibration data, where the joint calibration data is transformation relationship data between the world coordinate system and the local coordinate system.

2. The joint calibration method according to claim 1, characterized in that: The step of obtaining a first coordinate system conversion coefficient between a world coordinate system where the smart glasses are located and a local coordinate system where the multi-degree-of-freedom device is located in each degree of freedom according to the first inertial measurement unit data and the second inertial measurement unit data comprises: Obtaining, based on the first inertial measurement unit data, a first posture angle, a first acceleration, and a first angular velocity corresponding to a preset posture transformation movement of the smart glasses under a preset degree of freedom; Obtaining a first coordinate system according to the first posture angle, the first acceleration, and the first angular velocity, where the first coordinate system is a world coordinate system of the smart glasses in the degree of freedom corresponding to the preset posture transformation movement; Obtaining, based on the second inertial measurement unit data, a second posture angle, a second acceleration, and a second angular velocity corresponding to the multi-degree-of-freedom device in the preset posture transformation movement; Obtaining a second coordinate system according to the second posture angle, the second acceleration, and the second angular velocity, where the second coordinate system is a local coordinate system of the multi-degree-of-freedom device based on the world coordinate system in the degree of freedom corresponding to the preset posture transformation motion; A matrix rotation transformation is performed based on the first coordinate system and the second coordinate system to obtain a first coordinate system conversion coefficient under corresponding degrees of freedom.

3. The joint calibration method according to claim 1, characterized in that: The step of obtaining a first coordinate system conversion coefficient between a world coordinate system where the smart glasses are located and a local coordinate system where the multi-degree-of-freedom device is located in each degree of freedom according to the first inertial measurement unit data and the second inertial measurement unit data comprises: performing a first preprocessing on the first inertial measurement unit data; Based on the first inertial measurement unit data after the first preprocessing, obtaining a first posture angle, a first acceleration, and a first angular velocity corresponding to a preset posture transformation movement of the smart glasses under a preset degree of freedom; Obtaining a first coordinate system according to the first posture angle, the first acceleration, and the first angular velocity, where the first coordinate system is a world coordinate system of the smart glasses in the degree of freedom corresponding to the preset posture transformation movement; Obtaining, based on the second inertial measurement unit data, a second posture angle, a second acceleration, and a second angular velocity corresponding to the multi-degree-of-freedom device in the preset posture transformation movement; Obtaining a second coordinate system according to the second posture angle, the second acceleration, and the second angular velocity, where the second coordinate system is a local coordinate system of the multi-degree-of-freedom device based on the world coordinate system in the degree of freedom corresponding to the preset posture transformation motion; A matrix rotation transformation is performed based on the first coordinate system and the second coordinate system to obtain a first coordinate system conversion coefficient under corresponding degrees of freedom.

4. The joint calibration method according to claim 3, characterized in that: The first preprocessing of the first inertial measurement unit data includes: The first inertial measurement unit data is subjected to a first denoising process, a first motion compensation process, and a first correction process.

5. The joint calibration method according to claim 3 or 4, characterized in that: The obtaining, based on the first inertial measurement unit data having completed the first preprocessing, a first posture angle, a first acceleration, and a first angular velocity corresponding to the preset posture transformation movement of the smart glasses under a preset degree of freedom comprises: Based on the first inertial measurement unit data that has completed the first preprocessing, obtaining a first attitude angle through a first attitude algorithm model; Based on the first inertial measurement unit data that has completed the first preprocessing, a first acceleration and a first angular velocity are obtained through a first motion parameter algorithm model.

6. The joint calibration method according to claim 1, characterized in that: Also includes: According to the eye frame size and the combined image distance of the smart glasses, the size of the virtual image and the spatial position coordinate system of each area of ​​the virtual image relative to the world coordinate system are obtained; Based on the size of the virtual image and the spatial position coordinate system of each area of ​​the virtual image relative to the world coordinate system, the smart glasses are controlled to display the virtual image.

7. The joint calibration method according to claim 1, characterized in that: The controlling the turntable to drive the smart glasses and the multi-degree-of-freedom device to synchronously perform posture transformation motion in each degree of freedom includes: The turntable is controlled to drive the smart glasses and the multi-degree-of-freedom device to synchronously perform rotational motion along a first axis, synchronously perform rotational motion along a second axis, and synchronously perform rotational motion along a third axis, wherein the first axis, the second axis, and the third axis are orthogonal to each other.

8. A combined calibration control device, characterized in that: Used to control a joint calibration device to perform joint calibration on smart glasses and a multi-degree-of-freedom device, the joint calibration device includes a turntable and a data acquisition component, the smart glasses and the multi-degree-of-freedom device are mounted on the turntable, the multi-degree-of-freedom device is communicatively connected with the smart glasses, and the joint calibration control device includes: A turntable control module, used to control the turntable to drive the smart glasses and the multi-degree-of-freedom device to synchronously perform posture change movements in each degree of freedom; A data acquisition control module, configured to acquire first inertial measurement unit data of the smart glasses and second inertial measurement unit data of the multi-degree-of-freedom device through the data acquisition component when performing posture transformation motion in each degree of freedom; A conversion coefficient acquisition module, used for obtaining a first coordinate system conversion coefficient between a world coordinate system where the smart glasses are located and a local coordinate system where the multi-degree-of-freedom device is located in each degree of freedom according to the first inertial measurement unit data and the second inertial measurement unit data; The fusion calibration module is used to fuse and calibrate multiple first coordinate system conversion coefficients corresponding to all degrees of freedom through a fusion algorithm model to obtain joint calibration data, where the joint calibration data is the transformation relationship data between the world coordinate system and the local coordinate system.

9. An electronic device, characterized in that: The electronic device comprises a controller, the controller comprises a memory and at least one processor, the memory stores instructions, The at least one processor calls the instructions in the memory, so that the controller executes the joint calibration method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the joint calibration method according to any one of claims 1 to 7 is implemented.

Images