Intelligent glasses, calibration method and calibration device thereof and computer readable storage medium
By using sensors to calculate the relative rotation matrix in smart glasses, the problem of offline calibration in the prior art is solved, and convenient calibration and more efficient adaptation of the camera module and glasses body are achieved.
Patent Information
- Application Number
- CN202310605260.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-26
- Publication Date
- 2024-11-26
AI Technical Summary
After changing the camera module, existing smart glasses need to perform cumbersome offline calibration, resulting in complex and inconvenient adaptation process.
By setting the first sensor and the second sensor in the smart glasses, data of the camera module and the glasses body are obtained, and the relative rotation matrix is calculated to realize calibration, avoid offline calibration methods, and improve adaptability.
The calibration process between the camera module and the glasses body is simplified, the adaptability of different camera modules is improved, and convenient calibration and more efficient adaptation is achieved.
Smart Images

Figure CN119027508A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of extended reality, and in particular, relates to a pair of smart glasses and a calibration method, a calibration device and a computer-readable storage medium thereof. Background Art
[0002] There are usually two ways to install the camera modules on the augmented reality (AR) glasses on the market, namely integrated and detachable. For the integrated method, although it can improve the integration of the product and reduce the product volume, the camera module cannot be replaced when the camera is damaged or needs to adapt to different scenes; for the current detachable method, although it is easy to replace different camera modules, when a new camera module is replaced on the glasses body, the glasses body and the camera module are usually recalibrated using the traditional offline calibration method, which is a cumbersome calibration work. Summary of the invention
[0003] The technical problem solved by the present invention is: how to conveniently realize the calibration between the glasses body and the camera module in the smart glasses.
[0004] A calibration method for smart glasses, the calibration method:
[0005] Acquire first sensor data of the camera module through the first sensor, and acquire second sensor data of the glasses body through the second sensor;
[0006] A relative rotation matrix between the camera module and the glasses body is determined according to the first sensor data and the second sensor data, so as to calibrate the camera module and the glasses body based on the relative rotation matrix.
[0007] Optionally, the first sensor data is a first angular velocity data set of the camera module when it rotates, and the second sensor data is a second angular velocity data set of the glasses body when it rotates. The method for calculating the relative rotation matrix between the camera module and the glasses body according to the first sensor data and the second sensor data is:
[0008] Determining a time delay value between the camera module and the glasses body according to the first angular velocity data set and the second angular velocity data set;
[0009] Selecting first angular velocity data from the first angular velocity data set and selecting second angular velocity data from the second angular velocity data set according to the time delay value;
[0010] The relative rotation matrix between the camera module and the glasses body is calculated according to the first angular velocity data and the second angular velocity data.
[0011] Optionally, a method for obtaining a time delay value between the camera module and the glasses body according to the first angular velocity data set and the second angular velocity data set is:
[0012] Obtaining at least two different preset candidate delay values;
[0013] Selecting a plurality of first angular velocity data values at different times from the first angular velocity data set and a plurality of second angular velocity data values at different times from the second angular velocity data set according to each candidate delay value, wherein the number of the plurality of first angular velocity data values and the number of the plurality of second angular velocity data values are the same;
[0014] The trace correlation value corresponding to each candidate delay value is calculated in sequence according to the multiple first angular velocity data values and the multiple second angular velocity data values corresponding to each candidate delay value;
[0015] The candidate delay value corresponding to the largest trace correlation value among the trace correlation values is used as the time delay value.
[0016] Optionally, the first sensor data and the second sensor data are data at the same time, the first sensor data are the first acceleration value and the first magnetic value of the camera module, and the second sensor data are the second acceleration value and the second magnetic value of the glasses body. The method for calculating the relative rotation matrix between the camera module and the glasses body according to the first sensor data and the second sensor data is:
[0017] Calculate a first rotation matrix of the camera module in a geographic coordinate system according to the first accelerometer value and the first magnetometer value, and calculate a second rotation matrix of the glasses body in a geographic coordinate system according to the second accelerometer value and the second magnetometer value;
[0018] The relative rotation matrix between the camera module and the glasses body is calculated according to the first rotation matrix and the second rotation matrix.
[0019] Optionally, determining a relative rotation matrix between the camera module and the glasses body according to the first sensor data and the second sensor data includes:
[0020] If the sampling frequencies corresponding to collecting the first sensor data and collecting the second sensor data are different, resampling the first sensor data and / or the second sensor data to obtain processed first sensor data and processed second sensor data, wherein the sampling frequencies corresponding to the processed first sensor data and the processed second sensor data are the same;
[0021] A relative rotation matrix between the camera module and the glasses body is determined according to the processed first sensor data and the processed second sensor data.
[0022] The present application also discloses a calibration device for smart glasses, the calibration device comprising:
[0023] A data acquisition module, the data acquisition module is used to acquire first sensor data of the camera module through the first sensor, and acquire second sensor data of the glasses body through the second sensor;
[0024] A calibration module is used to determine a relative rotation matrix between the camera module and the glasses body according to the first sensor data and the second sensor data, so as to perform calibration between the camera module and the glasses body based on the relative rotation matrix.
[0025] The present application also discloses a pair of smart glasses, which include a glasses body, a camera module, a first sensor, and a second sensor. The camera module is installed on the glasses body, the first sensor is connected to the camera module, and the second sensor is installed on the glasses body. The first sensor is used to obtain first sensor data of the camera module, and the second sensor is used to obtain second sensor data of the glasses body. The first sensor data and the second sensor data are used to calculate a relative rotation matrix between the camera module and the glasses body, and the relative rotation matrix is used to calibrate the camera module and the glasses body.
[0026] Optionally, both the first sensor and the second sensor are inertial sensors.
[0027] Optionally, the camera module is detachably mounted on the glasses body, and the first sensor is fixedly connected to the camera module.
[0028] The present application also discloses a computer-readable storage medium, which stores a calibration program for smart glasses. When the calibration program for smart glasses is executed by a processor, the calibration method for smart glasses is implemented.
[0029] The present invention discloses a smart glasses and a calibration method, a calibration device and a computer-readable storage medium, which have the following technical effects:
[0030] By acquiring the first sensor data of the camera module and the second sensor data of the glasses body, the relative rotation matrix between the camera module and the glasses body can be calculated, which is conducive to further calibration between the camera module and the glasses body based on the relative rotation matrix. There is no need to adopt an offline calibration method, thereby improving the adaptability of the glasses body to different camera modules. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 This is a principle block diagram of the smart glasses according to the first embodiment of the present invention;
[0032] Figure 2 This is a flow chart of a calibration method for smart glasses according to Embodiment 2 of the present invention;
[0033] Figure 3 This is a schematic diagram of calibration of the eyeglass body and the camera module according to the second embodiment of the present invention;
[0034] Figure 4 This is a flow chart of a method for calculating a relative rotation matrix in the first implementation manner of the second embodiment of the present invention;
[0035] Figure 5 This is a flow chart of a method for calculating a time delay value in a first implementation manner in Embodiment 2 of the present invention;
[0036] Figure 6 This is a flow chart of a method for calculating a relative rotation matrix in a second implementation manner in Embodiment 2 of the present invention;
[0037] Figure 7 This is a principle block diagram of a calibration device for smart glasses according to a fourth embodiment of the present invention;
[0038] Figure 8 Schematic diagram of a computer device according to Embodiment 5 of the present invention. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0040] Before describing the various embodiments of the present application in detail, the technical concept of the present application is first briefly described: in the detachable smart glasses of the prior art, although the camera module can be replaced, after the camera is replaced, an offline calibration method is required to calibrate the camera module and the glasses body, and the working process is relatively cumbersome. To this end, the smart glasses provided by the present application set a first sensor on the camera module to obtain the first sensor data, and use the second sensor carried by the glasses body itself to obtain the second sensor data, and calculate the relative rotation matrix between the camera module and the glasses body based on the first sensor data and the second sensor data, so as to further calibrate the camera module and the glasses body based on the relative rotation matrix, without the need to adopt cumbersome offline calibration methods, thereby improving the adaptability of the glasses body to different camera modules.
[0041] Specifically, Figure 1 As shown, the smart glasses of the first embodiment include a glasses body 10, a camera module 20, a first sensor 40 and a second sensor (not shown), the camera module 20 is mounted on the glasses body 10, the first sensor 40 is connected to the camera module 20, and the second sensor is mounted on the glasses body 10. The first sensor 40 is used to obtain the first sensor data of the camera module 20, the second sensor is used to obtain the second sensor data of the glasses body 10, the first sensor data and the second sensor data are used to calculate the relative rotation matrix between the camera module 20 and the glasses body 10, and the relative rotation matrix is used to calibrate the camera module 20 and the glasses body 10. Among them, the smart glasses can be AR glasses, VR glasses, MR glasses and other extended reality glasses.
[0042] The camera module 20 can be detachably mounted on the eyeglass body 10, or the camera module 20 can be fixedly mounted on the eyeglass body 10. The first sensor 40 and the camera module 20 can be fixedly connected or detachably connected. The detachable installation can be installed by a bayonet connection, a threaded connection, a magnetic connection, etc. The camera module 20 can be a low-resolution camera, a high-resolution camera, a grayscale camera, a color camera, a near-infrared camera, a far-infrared camera, etc.
[0043] Exemplarily, the first sensor 40 and the second sensor carried by the eyeglass body 10 are both inertial sensors. The inertial sensor can be any one or more of a gyroscope, an accelerometer and a magnetometer.
[0044] Furthermore, the smart glasses further include a communication module 30, which is used to realize communication between the camera module 20 and the glasses body 10. The communication module 30 may be a Bluetooth communication module, a near field communication module (NFC), etc. Exemplarily, the communication module 30 is used to transmit the image acquired by the camera module 20 to the glasses body 10. In addition, the communication module 30 may also be used to transmit the image information obtained after the image acquired by the camera module 20 is processed to the glasses body 10, so that the camera module 20 and the glasses body 10 can interact with each other.
[0045] Furthermore, the smart glasses further include a processing chip 50, which is used to process the image acquired by the camera module 20 to form image information, and the communication module 30 is also used to send the image information to the glasses body 10. The processing chip 50 can be a chip related to image processing such as a CPU, a GPU, or an NPU. The image information includes target recognition classification information, extracted text information, and the like.
[0046] Exemplarily, the camera module 20 , the communication module 30 , the first sensor 40 and the processing chip 50 are integrally arranged and can be detachably mounted on the eyeglass body 10 .
[0047] Furthermore, the smart glasses further include a power module 60, which supplies power to the camera module 20, the communication module 30, the first sensor 40 and the processing chip 50. The power module 60 is detachably connected to the glasses body 10 or fixedly connected thereto.
[0048] Specifically, if Figure 2 As shown, the calibration method of the smart glasses of the second embodiment includes the following steps:
[0049] Step S10: acquiring first sensor data of the camera module 20 through the first sensor 40, and acquiring second sensor data of the glasses body;
[0050] Step S20: determining a relative rotation matrix between the camera module and the glasses body according to the first sensor data and the second sensor data, so as to calibrate the camera module 20 and the glasses body 10 based on the relative rotation matrix.
[0051] The calibration in the second embodiment refers to determining the mapping relationship between the screen coordinate system of the glasses body 10 of the smart glasses and the camera coordinate system of the camera module 20. After the mapping relationship is determined, the fusion and superposition between the virtual information and the real object can be better achieved. Figure 3 As shown in the figure, cylinder A represents a physical object, and the dotted box B on the right side of the cylinder represents virtual information. More specifically, suppose any point P on cylinder A has a coordinate of P in the world coordinate system. w , the coordinates in the camera coordinate system of the camera module 20 are PC , the coordinates in the screen coordinate system of the eyeglass body 10 are P S , each coordinate satisfies the following relationship:
[0052] P C =[R WC |T WC ; 0 0 0 0]P W
[0053] P S =K[R CV |T CV ]P C =GP C
[0054] Among them, R WC , R CV Represents the 3*3 relative rotation matrix, T WC , T CV Represents a 3*1 relative translation matrix, K represents an intrinsic parameter matrix, and G represents a mapping relationship between the screen coordinate system and the camera coordinate system. In other words, the solution of the mapping relationship G can be converted into the solution of the intrinsic parameter matrix, the relative translation matrix, and the relative rotation matrix. Among them, the intrinsic parameter matrix is determined according to the lens parameters of the camera module itself, and the relative translation matrix is determined according to the relative position between the camera module 20 and the eyeglass body 10. When the camera module 20 is installed on the eyeglass body 10, the relative position between the camera module 20 and the eyeglass body 10 can be determined, so that the relative translation matrix can be further determined. The determination process of the intrinsic parameter matrix and the relative translation matrix is a technical content that is familiar to those skilled in the art. The determination process of the relative rotation matrix is more difficult. The main content and main improvement of the calibration method of the second embodiment is how to determine the relative rotation matrix. Once the relative rotation matrix is determined, it can be further combined with the relative translation matrix and the intrinsic parameter matrix to further map the relationship G, thereby realizing the calibration between the camera module 20 and the eyeglass body 10. Two implementation methods of the relative rotation matrix R solution process are described below.
[0055] In the first embodiment, the first sensor 40 and the second sensor are both gyroscopes, and the first sensor data is a first angular velocity data set of the camera module 20 when rotating. The second sensor data is a second angular velocity data set when the eyeglass body 10 is rotating. in, Respectively represent the first angular velocity data at the 1st moment, the 2nd moment, the 3rd moment, ... the nth moment, They respectively represent the second angular velocity data at the 1st moment, the 2nd moment, the 3rd moment ... the nth moment, where "moment" refers to the moment when the angular velocity is collected by the gyroscope.
[0056] If the sampling frequencies corresponding to the first sensor data and the second sensor data are different, the first sensor data and / or the second sensor data are resampled to obtain processed first sensor data and processed second sensor data, and the processed first sensor data and the processed second sensor data have the same sampling frequencies; the relative rotation matrix between the camera module and the eyeglass body is determined based on the processed first sensor data and the processed second sensor data. "Resampling" here means reprocessing the already collected data set to obtain a new data set.
[0057] Among them, the resampling method includes two methods, downsampling and upsampling. The frequency of a data set with a high sampling frequency can be reduced by downsampling, and the frequency of a data set with a low sampling frequency can be increased by upsampling. For example, when the sampling frequency of the first angular velocity data set is twice the sampling frequency of the second angular velocity data set, the first angular velocity data set can be downsampled, or the second angular velocity data set can be upsampled, so that the sampling frequency of the first angular velocity data set is consistent with that of the second angular velocity data set. Specifically, data collection is performed over a period of time. Assuming that the first angular velocity data set The data of each represent the first angular velocity data of the 1st second, the 2nd second, the 3rd second ... the 50th second, the second angular velocity data set The data of represent the second angular velocity data of the 1st second, the 3rd second, the 5th second ... the 49th second respectively. At this time, the sampling frequencies of the two groups are different. The second angular velocity data set can be upsampled. The upsampling can be performed by interpolation to obtain the second angular velocity data of the 2nd second, the 4th second, the 6th second ... the 50th second, so that the processed second angular velocity data set With the first angular velocity data set The sampling frequency is the same.
[0058] Furthermore, if Figure 4 As shown, step S20 includes the following steps:
[0059] Step S201: Obtain the time delay value between the camera module and the glasses body according to the first angular velocity data set and the second angular velocity data set. Figure 5 As shown, step S201 includes the following steps:
[0060] Step S2011: Obtain at least two preset different candidate delay values δt. Exemplarily, multiple candidate delay values from large to small may be set.
[0061] Step S2012: Selecting a plurality of first angular velocity data values at different times from the first angular velocity data set and selecting a plurality of second angular velocity data values at different times from the second angular velocity data set according to each candidate delay value δt The selected multiple first angular velocity data values are the same in number as the multiple second angular velocity data values, and the multiple first angular velocity data values are respectively represented in chronological order as follows: The multiple second angular velocity data values are respectively expressed in chronological order as The first angular velocity data value The time and the second angular velocity data value The time interval between the moments is the candidate delay value δt, and the first angular velocity data value The time and the second angular velocity data value The time interval between the moments is the candidate delay value δt, and so on. The time and the second angular velocity data value The time interval between the moments is the candidate delay value δt. For example, when δt=0.2 seconds, the multiple first angular velocity data values are the angular velocity data of the 1st second, 2nd second, 3rd second, ... 50th second selected from the first angular velocity data set, and the multiple second angular velocity data values are the angular velocity data of the 1st second, 2.2 seconds, 3.2 seconds, ... 50.2 seconds selected from the second angular velocity data set. It should be noted that the first angular velocity data value and the second angular velocity data value are both represented in matrix form.
[0062] Step S2013: Calculate the trace correlation value corresponding to each candidate delay value according to the multiple first angular velocity data values and the multiple second angular velocity data values corresponding to each candidate delay value. Exemplarily, the calculation method of the trace correlation value TC corresponding to each candidate delay value δt is as follows:
[0063]
[0064] Among them, Tr() in the formula represents the trace of the calculation matrix. Each angular velocity data is expressed in matrix form, so the trace of the matrix can be calculated. represents the first angular velocity data value at the i-th moment, Represents the second angular velocity data value at the ith moment, the number of first angular velocity data values and second angular velocity data values contained in each cumulative symbol in the formula is the number of first angular velocity data values and second angular velocity data values selected at multiple different moments in the previous step S2013, and the number of first angular velocity data values and second angular velocity data values selected can be determined according to actual conditions.
[0065] Step S2014: taking the candidate delay value corresponding to the largest trace correlation value among the trace correlation values as the time delay value.
[0066] Since the true time delay value is unknown, a series of candidate delay values are assumed, and the first angular velocity data value and the second angular velocity data value are screened out accordingly, and the correlation between the two angular velocity data values is determined by calculating the trace correlation value. The larger the trace correlation value, the greater the correlation between the two angular velocity data values. The higher the correlation, the closer the moments of the two angular velocity data values are to the same moment, that is, the closer the corresponding candidate delay value is to the true time delay value. Therefore, after traversing each candidate delay value δt, the candidate delay value δt corresponding to the largest trace correlation value is used as the final time delay value dt.
[0067] Of course, in other implementations, the average of the candidate delay values corresponding to several larger trace correlation values ranked in front of the others can be selected as the final time delay value dt. For example, the candidate delay value δt1 corresponding to the largest trace correlation value and the candidate delay value δt2 corresponding to the second largest trace correlation value are selected, and their average value is used as the time delay value dt, that is, dt = (δt1+δt2) / 2.
[0068] Step S202: selecting first angular velocity data from the first angular velocity data set and selecting second angular velocity data from the second angular velocity data set according to the time delay value.
[0069] Step S203: Calculate the relative rotation matrix between the camera module and the glasses body according to the first angular velocity data and the second angular velocity data.
[0070] Specifically, when the camera module 20 and the eyeglass body 10 are relatively fixed, during the movement, the angular velocities of the two satisfy the following relationship:
[0071] ω sd (t) = Rω g (t+dt)
[0072] Among them, t represents the time t when the angular velocity is collected by the gyroscope, dt represents the time delay value, ω sd (t) is the first angular velocity data at the tth moment selected from the first angular velocity data set, ω g (t+dt) is the second angular velocity data at the time t+dt selected from the second angular velocity data set. The relative rotation matrix R between the camera module and the eyeglass body can be obtained by linearly solving the above equation.
[0073] It should be noted that the first sensor data and the second sensor data used in the first implementation manner may also be acceleration information or magnetometer data, and the calculation process remains unchanged.
[0074] In a second implementation, the first sensor data and the second sensor data are data at the same time, the first sensor data are the first accelerometer value and the first magnetometer value of the camera module, and the second sensor data are the second accelerometer value and the second magnetometer value of the glasses body.
[0075] like Figure 6 As shown, step S20 includes the following steps:
[0076] Step S204, calculate the first rotation matrix of the camera module in the geographic coordinate system according to the first accelerometer value and the first magnetometer value, and calculate the second rotation matrix of the glasses body in the geographic coordinate system according to the second accelerometer value and the second magnetometer value. Wherein, the geographic coordinate system is the Northeast Celestial Coordinate System (ENU) or the North-East Earth Coordinate System (NED). The Northeast Celestial Coordinate System (ENU) belongs to the station-centered coordinate system, which is a coordinate system with the observation point as the origin. It is mainly used to understand the motion laws of other objects centered on the observation point. The three coordinate axes of the Northeast Celestial Coordinate System point to the east, north and sky directions that are perpendicular to each other, respectively. The North-East Earth Coordinate System (NED) belongs to the navigation coordinate system, and the three coordinate axes of the North-East Earth Coordinate System point to the north, east and ground directions that are perpendicular to each other, respectively. Exemplarily, the Northeast Celestial Coordinate System (ENU) is used as an example for description.
[0077] At a certain moment, the first accelerometer value a obtained is a=[a1, a2, a3] T and the first magnetometer value m = [m1, m2, m3] T Here, the first accelerometer value a and the first magnetometer value m are both expressed in matrix form, and the symbol T represents the transpose of the matrix. The first accelerometer value is normalized to obtain the vector pointing to the sky direction. ||a|| represents the modulus of the first accelerometer value a. Normalize the first magnetometer value ||m|| represents the modulus of the first magnetometer value m, and further calculation of the vector cross product gives the east vector The north vector N = U × E is obtained by cross-producting the celestial vector and the east vector, and the first rotation matrix R of the camera module 20 in the northeast celestial coordinate system can be further obtained by combining sd =[E,N,U].
[0078] Similarly, the second rotation matrix R of the eyeglass body 10 in the northeast celestial coordinate system can be obtained in the above manner: g .
[0079] Step S205: According to the first rotation matrix Rsd With the second rotation matrix R g The relative rotation matrix R between the camera module and the glasses body is calculated. The specific calculation method is as follows:
[0080]
[0081] Therefore, through the above two implementations, the relative rotation matrix between the camera module 20 and the glasses body 10 can be realized, so that calibration can be further realized without adopting an offline calibration method. Calibration can be achieved only through the sensor data of both the camera module 20 and the glasses body 10. The process is convenient, so that the glasses body 10 can quickly match different camera modules.
[0082] like Figure 7 As shown, this third embodiment also discloses a calibration device for smart glasses, the calibration device includes a data acquisition module 100 and a calibration module 200, the data acquisition module 100 is used to acquire first sensor data of the camera module through a first sensor, and acquire second sensor data of the glasses body through a second sensor; the calibration module 200 is used to determine the relative rotation matrix between the camera module and the glasses body according to the first sensor data and the second sensor data, so as to perform calibration between the camera module and the glasses body based on the relative rotation matrix.
[0083] Among them, when the first sensor data is a first angular velocity data set of the camera module when it is rotating, and the second sensor data is a second angular velocity data set of the glasses body when it is rotating, the calibration module 200 is also used to: determine the time delay value between the camera module and the glasses body according to the first angular velocity data set and the second angular velocity data set; select the first angular velocity data from the first angular velocity data set and select the second angular velocity data from the second angular velocity data set according to the time delay value; calculate the relative rotation matrix between the camera module and the glasses body according to the first angular velocity data and the second angular velocity data.
[0084] Furthermore, the calibration module 200 is also used to: obtain at least two different preset candidate delay values; select a plurality of first angular velocity data values at different moments from the first angular velocity data set and a plurality of second angular velocity data values at different moments from the second angular velocity data set according to each candidate delay value, and the number of the plurality of first angular velocity data values and the plurality of second angular velocity data values are the same; calculate the trace correlation value corresponding to each candidate delay value according to the plurality of first angular velocity data values and the plurality of second angular velocity data values corresponding to each candidate delay value, and take the candidate delay value corresponding to the largest trace correlation value among the trace correlation values as the time delay value.
[0085] In another embodiment, when the first sensor data is the first acceleration value and the first magnetic value of the camera module and the second sensor data is the second acceleration value and the second magnetic value of the glasses body, the calibration module 200 is also used to: calculate the first rotation matrix of the camera module in the geographic coordinate system according to the first accelerometer value and the first magnetometer value, calculate the second rotation matrix of the glasses body in the geographic coordinate system according to the second accelerometer value and the second magnetometer value, and calculate the relative rotation matrix between the camera module and the glasses body according to the first rotation matrix and the second rotation matrix.
[0086] Furthermore, if the sampling frequencies corresponding to collecting the first sensor data and collecting the second sensor data are different, the calibration module 200 is also used to: resample the first sensor data and / or the second sensor data to obtain processed first sensor data and processed second sensor data, wherein the sampling frequencies corresponding to the processed first sensor data and the processed second sensor data are the same, and determine the relative rotation matrix between the camera module and the glasses body based on the processed first sensor data and the processed second sensor data.
[0087] For a more detailed data processing process of the data acquisition module 100 and the calibration module 200 , reference may be made to the description of the second embodiment.
[0088] The fourth embodiment further discloses a computer-readable storage medium, which stores a calibration program for smart glasses. When the calibration program for smart glasses is executed by a processor, the calibration method for smart glasses in the second embodiment is implemented.
[0089] This fourth embodiment also discloses a computer device, at the hardware level, such as Figure 8 As shown, the computer device includes a processor 12, an internal bus 13, a network interface 14, and a computer-readable storage medium 11. The processor 12 reads the corresponding computer program from the computer-readable storage medium and then runs it, forming a request processing device at the logical level. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logical unit, but can also be hardware or logic devices. The computer-readable storage medium 11 stores a calibration program for smart glasses, and when the calibration program for smart glasses is executed by the processor, the above-mentioned calibration method for smart glasses is implemented.
[0090] Computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, modules of programs or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.
[0091] The specific implementation methods of the present invention are described in detail above. Although some embodiments have been shown and described, those skilled in the art should understand that these embodiments can be modified and improved without departing from the principles and spirit of the present invention, the scope of which is defined by the claims and their equivalents. These modifications and improvements should also be within the scope of protection of the present invention.
Claims
1. A calibration method for smart glasses, characterized in that: The calibration method: Acquire first sensor data of the camera module through the first sensor, and acquire second sensor data of the glasses body through the second sensor; A relative rotation matrix between the camera module and the glasses body is determined according to the first sensor data and the second sensor data, so as to calibrate the camera module and the glasses body based on the relative rotation matrix.
2. The calibration method according to claim 1, characterized in that: The first sensor data is a first angular velocity data set when the camera module rotates, and the second sensor data is a second angular velocity data set when the glasses body rotates. The method for calculating the relative rotation matrix between the camera module and the glasses body according to the first sensor data and the second sensor data is: determining a time delay value between the camera module and the glasses body according to the first angular velocity data set and the second angular velocity data set; Selecting first angular velocity data from the first angular velocity data set and selecting second angular velocity data from the second angular velocity data set according to the time delay value; The relative rotation matrix between the camera module and the glasses body is calculated according to the first angular velocity data and the second angular velocity data.
3. The calibration method according to claim 2, characterized in that: The method for obtaining the time delay value between the camera module and the glasses body according to the first angular velocity data set and the second angular velocity data set is: Obtaining at least two different preset candidate delay values; Selecting a plurality of first angular velocity data values at different times from the first angular velocity data set and a plurality of second angular velocity data values at different times from the second angular velocity data set according to each candidate delay value, wherein the number of the plurality of first angular velocity data values and the number of the plurality of second angular velocity data values are the same; The trace correlation value corresponding to each candidate delay value is calculated in sequence according to the multiple first angular velocity data values and the multiple second angular velocity data values corresponding to each candidate delay value; The candidate delay value corresponding to the largest trace correlation value among the trace correlation values is used as the time delay value.
4. The calibration method according to claim 1, characterized in that: The first sensor data and the second sensor data are data at the same time, the first sensor data are the first acceleration value and the first magnetic value of the camera module, and the second sensor data are the second acceleration value and the second magnetic value of the glasses body. The method for calculating the relative rotation matrix between the camera module and the glasses body according to the first sensor data and the second sensor data is: Calculate a first rotation matrix of the camera module in a geographic coordinate system according to the first accelerometer value and the first magnetometer value, and calculate a second rotation matrix of the glasses body in a geographic coordinate system according to the second accelerometer value and the second magnetometer value; The relative rotation matrix between the camera module and the glasses body is calculated according to the first rotation matrix and the second rotation matrix.
5. The calibration method according to claim 1, characterized in that: The determining of the relative rotation matrix between the camera module and the glasses body according to the first sensor data and the second sensor data comprises: If the sampling frequencies corresponding to collecting the first sensor data and collecting the second sensor data are different, resampling the first sensor data and / or the second sensor data to obtain processed first sensor data and processed second sensor data, wherein the sampling frequencies corresponding to the processed first sensor data and the processed second sensor data are the same; The relative rotation matrix between the camera module and the glasses body is determined according to the processed first sensor data and the processed second sensor data.
6. A calibration device for smart glasses, characterized in that: The calibration device comprises: A data acquisition module, the data acquisition module is used to acquire first sensor data of the camera module through the first sensor, and acquire second sensor data of the glasses body through the second sensor; A calibration module is used to determine a relative rotation matrix between the camera module and the glasses body according to the first sensor data and the second sensor data, so as to perform calibration between the camera module and the glasses body based on the relative rotation matrix.
7. A smart glasses, characterized in that: The smart glasses include a glasses body, a camera module, a first sensor and a second sensor, the camera module is installed on the glasses body, the first sensor is connected to the camera module, the second sensor is installed on the glasses body, the first sensor is used to obtain first sensor data of the camera module, the second sensor is used to obtain second sensor data of the glasses body, the first sensor data and the second sensor data are used to calculate the relative rotation matrix between the camera module and the glasses body, and the relative rotation matrix is used to calibrate the camera module and the glasses body.
8. The smart glasses according to claim 7, characterized in that: The first sensor and the second sensor are both inertial sensors.
9. The smart glasses according to claim 7, characterized in that: The camera module is detachably mounted on the glasses body, and the first sensor is fixedly connected to the camera module.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a calibration program for the smart glasses, and when the calibration program for the smart glasses is executed by the processor, the calibration method for the smart glasses according to any one of claims 1 to 5 is implemented.
