Equipment opening and closing detection method and electronic equipment

Abstract

The embodiment of the invention discloses an equipment opening and closing detection method and electronic equipment. According to the method, the electronic equipment can obtain motion sensing data of a plurality of rigid members carrying IMUs through the IMUs, and the motion sensing data comprises one or more of the following items: angular velocity and gravitational acceleration. The electronic device may determine a rotation transformation relationship between the plurality of IMU coordinate systems based on the motion sensing data. Afterwards, the electronic equipment can determine the opening and closing state of the electronic equipment based on the rotation transformation relation among the IMU coordinate systems, and the opening and closing state comprises an unfolding state and a closing state. In some embodiments, the open-close state also includes an intermediate state. According to the method for detecting the opening and closing of the equipment based on the multiple IMUs, the load, the cost and the power consumption of the electronic equipment can be reduced while the opening and closing state of the electronic equipment can be accurately judged.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminal, and in particular, to a device opening and closing detection method and an electronic device. BACKGROUND

[0002] The opening and closing detection technology is a technology that relies on various sensors to detect the opening and closing state of an electronic device. The opening and closing detection technology can be applied to foldable electronic devices, which can adjust the working state by judging the opening and closing state of the electronic device, thereby providing a more intelligent use experience for the user.

[0003] A common opening and closing detection scheme is to judge the opening and closing state of an electronic device through a proximity light sensor, but the precision of this scheme is limited, and there is a certain probability of false touch when other objects are close to the electronic device. Another common opening and closing detection scheme is to judge the opening and closing state of an electronic device through electromagnetic induction. However, the electromagnet device necessary in this scheme not only increases the weight and power consumption of the electronic device, but also causes interference to other electromagnetic-related devices (such as a loudspeaker) in the electronic device. Moreover, through the above electromagnetic induction scheme, the electronic device can only detect one opening and closing state, such as only detecting the closed state and being unable to detect the unfolded state, and cannot fully detect the opening and closing state of the electronic device. SUMMARY

[0004] The present application provides a device opening and closing detection method and an electronic device. In the method, the electronic device can obtain motion sensing data of a plurality of rigid members carrying IMUs through a plurality of IMUs, and the motion sensing data includes one or more of angular velocity and gravity acceleration. The electronic device can determine the rotation transformation relationship between the plurality of IMU coordinate systems based on the motion sensing data. Subsequently, the electronic device can determine the opening and closing state of the electronic device based on the rotation transformation relationship between the plurality of IMU coordinate systems, and the opening and closing state includes an unfolded state and a closed state. In some embodiments, the opening and closing state also includes an intermediate state. The above method of detecting the opening and closing state of the device based on multiple IMUs can accurately determine the opening and closing state of the electronic device while reducing the weight, cost and power consumption of the electronic device.

[0005] In a first aspect, a device opening and closing detection method is provided. The method is applied to a first electronic device, which is a foldable device composed of a plurality of rigid members. The first electronic device includes a plurality of inertial mass units (IMUs), and the plurality of IMUs are respectively located on different rigid members. The method includes: obtaining motion sensing data of the plurality of rigid members carrying the IMUs by the plurality of IMUs, the motion sensing data including one or more of angular velocity and gravitational acceleration; determining a rotation transformation relationship between the plurality of IMU coordinate systems based on the motion sensing data; and determining an opening and closing state of the first electronic device based on the rotation transformation relationship between the plurality of IMU coordinate systems, the opening and closing state including an unfolded state and a closed state.

[0006] By implementing the method provided in the first aspect, the first electronic device (i.e., a foldable electronic device) can calculate the rotation transformation relationship between the IMUs based on the motion sensing data detected by the IMUs distributed on different rigid members of the first electronic device, and then determine the opening and closing state of the first electronic device based on the rotation transformation relationship.

[0007] Compared with other sensors such as proximity light sensors and Hall sensors, the IMU itself has lower weight and cost. Therefore, the above-mentioned method of detecting the opening and closing state of the device based on multiple IMUs can accurately determine the opening and closing state of the electronic device while reducing the weight, cost, and power consumption of the electronic device. In particular, for some electronic devices that already carry IMUs, such as smart earphones, smart glasses, and smart head-mounted displays with spatial audio functions, the above-mentioned electronic devices can reuse the data of the IMUs for the above-mentioned device opening and closing detection without the need to additionally increase other sensors.

[0008] In combination with the first aspect, in some embodiments, the obtaining motion sensing data of the plurality of rigid members carrying the IMUs by the plurality of IMUs specifically includes: obtaining angular velocity of the plurality of rigid members carrying the IMUs by the plurality of IMUs; the determining a rotation transformation relationship between the plurality of IMU coordinate systems based on the motion sensing data specifically includes: determining a first rotation transformation relationship between the plurality of IMU coordinate systems based on the angular velocity of the plurality of rigid members carrying the IMUs; and the determining an opening and closing state of the first electronic device based on the rotation transformation relationship between the plurality of IMU coordinate systems specifically includes: determining the opening and closing state of the first electronic device based on the first rotation transformation relationship between the plurality of IMU coordinate systems.

[0009] By implementing the method provided in the above-mentioned embodiments, the electronic device can calculate the rotation transformation relationship between the IMUs based on the angular velocity data measured by the IMUs, and then determine the opening and closing state of the electronic device.

[0010] In some embodiments, the plurality of IMUs comprises a first IMU and a second IMU, the first IMU is located on a first rigid member, and the second IMU is located on a second rigid member, the first rigid member and the second rigid member are included in the plurality of rigid members, the angular velocities of the plurality of rigid members carrying IMUs are obtained by the plurality of IMUs, specifically comprising: the angular velocity of the first rigid member is obtained by the first IMU, and the angular velocity of the second rigid member is obtained by the second IMU; the first rotation transformation relationship between the plurality of IMU coordinate systems is determined based on the angular velocities of the plurality of rigid members carrying IMUs, specifically comprising: the first rotation transformation relationship between the first coordinate system and the second coordinate system is determined based on the angular velocity of the first rigid member and the angular velocity of the second rigid member, the first coordinate system is the coordinate system of the first IMU, and the second coordinate system is the coordinate system of the second IMU; and the open-close state of the first electronic device is determined based on the first rotation transformation relationship between the first coordinate system and the second coordinate system.

[0011] The electronic device can include IMU1 (i.e., the first IMU) and IMU2 (i.e., the second IMU) when the method provided by the above-mentioned embodiments is implemented. At this time, the electronic device specifically calculates the rotation transformation relationship (i.e., the first rotation transformation relationship) between the coordinate systems of IMU1 and IMU2 based on the angular velocities measured by IMU1 and IMU2 to implement the above-mentioned device open-close detection method.

[0012] In some embodiments, the first rotation transformation relationship between the first coordinate system and the second coordinate system is compared with a standard rotation transformation relationship to determine the open-close state of the first electronic device, the standard rotation transformation relationship comprises a first standard rotation transformation relationship and a second standard rotation transformation relationship, the first standard rotation transformation relationship is the rotation transformation relationship between the first coordinate system and the second coordinate system in a standard closed state, and the second standard rotation transformation relationship is the rotation transformation relationship between the first coordinate system and the second coordinate system in a standard unfolded state; if the deviation between the first rotation transformation relationship and the first standard rotation transformation relationship is less than a first threshold value, it is determined that the open-close state of the first electronic device is the closed state; and if the deviation between the rotation transformation relationship and the second standard rotation transformation relationship is less than a second threshold value, it is determined that the open-close state of the first electronic device is the unfolded state.

[0013] By implementing the method provided in the above embodiment, the electronic device can determine whether the current rotation transformation relationship is consistent with the standard unfolded state or the standard closed state by comparing the rotation transformation relationship between the IMUs calculated in real time (i.e., the first rotation transformation relationship) with the standard rotation transformation relationship, and then identify the open / close state of the device. The standard rotation transformation relationship includes a first standard rotation transformation relationship (which can be represented as R-fold-std) and a second standard rotation transformation relationship (which can be represented as R-unfold-std).

[0014] In combination with the first aspect, in some embodiments, the open / close state of the first electronic device further includes one or more intermediate states, and the standard rotation transformation relationship further includes a third standard rotation transformation relationship, which is a rotation transformation relationship between the first coordinate system and the second coordinate system in a standard intermediate state. The determining of the open / close state of the first electronic device based on the first rotation transformation relationship between the first coordinate system and the second coordinate system further includes: if the deviation between the first rotation transformation relationship and the third standard rotation transformation relationship is less than a third threshold, determining that the open / close state of the first electronic device is the intermediate state.

[0015] In combination with the first aspect, in some embodiments, the open / close state of the first electronic device further includes one or more intermediate states, and the standard rotation transformation relationship further includes a third standard rotation transformation relationship, which is a rotation transformation relationship between the first coordinate system and the second coordinate system in a standard intermediate state. The determining of the open / close state of the first electronic device based on the first rotation transformation relationship between the first coordinate system and the second coordinate system further includes: if the deviation between the first rotation transformation relationship and the third standard rotation transformation relationship is less than a third threshold, determining that the open / close state of the first electronic device is the intermediate state.

[0016] In combination with the first aspect, in some embodiments, the first rotation transformation relationship between the first coordinate system and the second coordinate system is represented by a first rotation axis and a first rotation angle.

[0017] In combination with the first aspect, in some embodiments, the first rotation axis and the first rotation angle are obtained by minimizing the cumulative deviation between the actual angular velocity of the second IMU and the estimated angular velocity of the second IMU, which is calculated based on the actual angular velocity of the first IMU.

[0018] The method provided by the above embodiment can be implemented in an electronic device, which can be configured with an optimization function for reflecting the cumulative deviation between the actual angular velocity of the IMU2 and the estimated angular velocity of the IMU2. By minimizing the optimization function, the most suitable rotation axis (i.e., the first rotation axis) and rotation angle (i.e., the first rotation angle) can be found to make the estimated IMU2 coordinate system and the actual IMU2 coordinate system closest after rotation, that is, the most accurate rotation transformation relationship can be found by the optimization function.

[0019] In some embodiments in combination with the first aspect, the estimated angular velocity of the second IMU is calculated based on the actual angular velocity of the first IMU, specifically comprising: the estimated angular velocity of the second IMU is calculated based on the following formula: Gyro (IMU2) ′=Q·Gyro (IMU1) ·Q -1 wherein Gyro (IMU2) ′ represents the estimated angular velocity of the second IMU, Gyro (IMU1) represents the actual angular velocity of the first IMU, and quaternion θ1 represents the first rotation angle, and the first rotation axis is i, j, and k are imaginary units in a quaternion calculation formula.

[0020] In some embodiments in combination with the first aspect, the cumulative deviation is calculated based on the following formula:

[0021]

[0022] wherein δ represents the cumulative deviation, Gyro (IMU2) ′ represents the estimated angular velocity of the second IMU, Gyro (IMU1) represents the actual angular velocity of the first IMU, and quaternion θ1 represents the first rotation angle, and the first rotation axis is i, j, and k are imaginary units in a quaternion calculation formula.

[0023] In some embodiments in combination with the first aspect, before the first rotation transformation relationship between the multiple IMU coordinate systems is determined based on the angular velocities of the multiple rigid members carrying the IMUs, the method further comprises: detecting that the multiple rigid members carrying the IMUs are all in a motion state.

[0024] The method provided in the above embodiment can be implemented, before the angular velocity data is used to calculate the rotation transformation relationship between the IMU coordinate systems, the electronic device can first determine that it is in a moving rigid state at this time, that is, all the rigid members carrying the IMU are in a moving state, for example, the user unfolds the electronic device and is wearing the electronic device. At this time, it is meaningful for the electronic device to determine in which state the electronic device is, so that the electronic device does not have to perform the calculation of the rotation transformation relationship at all times, thereby reducing the power consumption when the above device opening and closing detection method is performed.

[0025] In combination with the first aspect, in some embodiments, the motion sensing data of the rigid members carrying the IMU is obtained by the plurality of IMUs, specifically comprising: the gravitational acceleration of the rigid members carrying the IMU is obtained by the plurality of IMUs; the rotation transformation relationship between the plurality of IMU coordinate systems is determined based on the motion sensing data, specifically comprising: the second rotation transformation relationship between the plurality of IMU coordinate systems is determined based on the gravitational acceleration of the rigid members carrying the IMU; the opening and closing state of the first electronic device is determined based on the rotation transformation relationship between the plurality of IMU coordinate systems, specifically comprising: the opening and closing state of the first electronic device is determined based on the second rotation transformation relationship between the plurality of IMU coordinate systems.

[0026] The method provided in the above embodiment can be implemented, and the electronic device can also calculate the rotation transformation relationship between the IMUs by using the acceleration data measured by the IMU, and then determine the opening and closing state of the electronic device. It can be understood that, since the IMU of the electronic device cannot actually measure the angular velocity data in a completely stationary state, the use scenario of the above device opening and closing detection method can be improved by using the acceleration data to calculate the rotation transformation relationship between the IMUs, so that the electronic device can also recognize the opening and closing state of the electronic device by using the motion sensing data measured by the IMU in a completely stationary state.

[0027] In some embodiments, the plurality of IMUs includes a first IMU and a second IMU, the first IMU is located on a first rigid member, and the second IMU is located on a second rigid member, the first rigid member and the second rigid member are included in the plurality of rigid members. The method for obtaining the gravity acceleration of the plurality of rigid members carrying the IMUs by the plurality of IMUs specifically includes: obtaining the gravity acceleration of the first rigid member by the first IMU, and obtaining the gravity acceleration of the second rigid member by the second IMU. The method for determining the second rotation transformation relationship between the plurality of IMU coordinate systems based on the gravity acceleration of the plurality of rigid members carrying the IMUs specifically includes: determining the second rotation transformation relationship between the first coordinate system and the second coordinate system based on the gravity acceleration of the first rigid member and the gravity acceleration of the second rigid member, the first coordinate system being the coordinate system of the first IMU, and the second coordinate system being the coordinate system of the second IMU. The method for determining the open-close state of the first electronic device based on the second rotation transformation relationship between the plurality of IMU coordinate systems specifically includes: determining the open-close state of the first electronic device based on the second rotation transformation relationship between the first coordinate system and the second coordinate system.

[0028] The electronic device can include IMU1 (i.e., the first IMU) and IMU2 (i.e., the second IMU) when the method provided by the above embodiments is implemented. At this time, the electronic device specifically performs the above device open-close detection method by calculating the rotation transformation relationship (i.e., the second rotation transformation relationship) between the IMU1 and the IMU2 coordinate systems based on the acceleration measured by the IMU1 and the IMU2.

[0029] In some embodiments, the method for determining the open-close state of the first electronic device based on the second rotation transformation relationship between the first coordinate system and the second coordinate system specifically includes: comparing the second rotation transformation relationship with a standard rotation transformation relationship, the standard rotation transformation relationship including a first standard rotation transformation relationship and a second standard rotation transformation relationship, the first standard rotation transformation relationship being the rotation transformation relationship between the first coordinate system and the second coordinate system in a standard closed state, and the second standard rotation transformation relationship being the rotation transformation relationship between the first coordinate system and the second coordinate system in a standard unfolded state. If the deviation between the second rotation transformation relationship and the first standard rotation transformation relationship is less than a first threshold value, it is determined that the open-close state of the first electronic device is a closed state. If the deviation between the second rotation transformation relationship and the second standard rotation transformation relationship is less than a second threshold value, it is determined that the open-close state of the first electronic device is an unfolded state.

[0030] By implementing the method provided in the above embodiments, the electronic device can determine whether the current rotation transformation relationship is consistent with the standard unfolded state or the standard folded state by comparing the rotation transformation relationship between the IMUs calculated in real time (i.e., the second rotation transformation relationship) with the standard rotation transformation relationship, and then identify the open / close state of the device. The standard rotation transformation relationship includes a first standard rotation transformation relationship (which can be represented as R-fold-std) and a second standard rotation transformation relationship (which can be represented as R-unfold-std).

[0031] In combination with the first aspect, in some embodiments, the open / close state of the first electronic device further includes one or more intermediate states, and the standard rotation transformation relationship further includes a third standard rotation transformation relationship, which is a rotation transformation relationship between the first coordinate system and the second coordinate system in a standard intermediate state. The determining of the open / close state of the first electronic device based on the second rotation transformation relationship between the first coordinate system and the second coordinate system further includes: if a deviation between the second rotation transformation relationship and the third standard rotation transformation relationship is less than a third threshold, determining that the open / close state of the first electronic device is the intermediate state.

[0032] In combination with the first aspect, in some embodiments, the open / close state of the first electronic device further includes one or more intermediate states, and the standard rotation transformation relationship further includes a third standard rotation transformation relationship, which is a rotation transformation relationship between the first coordinate system and the second coordinate system in a standard intermediate state. The determining of the open / close state of the first electronic device based on the second rotation transformation relationship between the first coordinate system and the second coordinate system further includes: if a deviation between the second rotation transformation relationship and the third standard rotation transformation relationship is less than a third threshold, determining that the open / close state of the first electronic device is the intermediate state.

[0033] In combination with the first aspect, in some embodiments, the second rotation transformation relationship between the first coordinate system and the second coordinate system is represented by a second rotation axis and a second rotation angle.

[0034] In combination with the first aspect, in some embodiments, the second rotation axis and the second rotation angle are calculated based on the gravitational acceleration of the first IMU and the gravitational acceleration of the second IMU.

[0035] The method provided by the above embodiment can effectively reduce the dependence on other sensors, and make full use of the low cost and low power consumption of the IMU. In addition, for some electronic devices that have already been equipped with an IMU, the motion sensing data detected by the IMU can be reused, and the IMU does not need to be newly added or only needs to be slightly added.

[0036] In some embodiments, the second rotation axis is calculated based on the following formula:

[0037] u2=Acc IMU1 ×Acc IMU2

[0038] wherein u2 represents the second rotation axis, Acc IMU1 represents the gravity acceleration of the first IMU, Acc IMU2 represents the gravity acceleration of the second IMU.

[0039] The second rotation angle is calculated based on the following formula:

[0040]

[0041] wherein θ2 represents the second rotation angle, Acc IMU1 represents the gravity acceleration of the first IMU, Acc IMU2 represents the gravity acceleration of the second IMU.

[0042] In some embodiments, before determining the second rotation transformation relationship between the plurality of IMU coordinate systems based on the gravity accelerations of the plurality of rigid members carrying the IMU, the method further comprises: detecting that the plurality of rigid members carrying the IMU are all in a stationary state.

[0043] Before the rotation transformation relationship between the IMU coordinate systems is calculated by using the acceleration data, the electronic device can first determine that it is in a stationary state at this time, i.e., the plurality of rigid members carrying the IMU are all in a stationary state. At this time, it is meaningful for the electronic device to determine in which state it is, so that the electronic device does not need to perform the calculation of the rotation transformation relationship at all times, thereby reducing the power consumption when the above device opening and closing detection method is performed.

[0044] In some embodiments, the method further comprises: if the first electronic device is in the unfolded state, automatically turning on or exiting the standby state; and if the first electronic device is in the closed state, automatically turning off or entering the standby state.

[0045] By implementing the method provided in the above embodiments, the electronic device can adjust the working state by judging the open or closed state of itself, thereby providing a more intelligent use experience for the user.

[0046] In a second aspect, the present application provides an electronic device, comprising a memory, a processor and a computer program stored in the memory; the processor executes the computer program to implement the method described in the first aspect and any possible implementation manner of the first aspect.

[0047] In a third aspect, the present application provides a computer readable storage medium, having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect and any possible implementation manner of the first aspect.

[0048] In a fourth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in the first aspect and any possible implementation manner of the first aspect.

[0049] It can be understood that the electronic device provided in the above second aspect, the computer storage medium provided in the third aspect, and the computer program product provided in the fourth aspect are all used to execute the method provided in the present application. Therefore, the beneficial effects that can be achieved thereby can refer to the beneficial effects in the corresponding method, which will not be described herein again. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is a layout schematic diagram of a dual-IMU in smart glasses provided by an embodiment of the present application;

[0051] Figure 2 is a schematic diagram of an IMU coordinate system provided by an embodiment of the present application;

[0052] Figure 3 is a schematic diagram of an IMU coordinate system in an unfolded state of smart glasses provided by an embodiment of the present application;

[0053] Figure 4 is a schematic diagram of an IMU coordinate system in a closed state of smart glasses provided by an embodiment of the present application;

[0054] Figure 5 is a schematic diagram of rotation of an IMU1 coordinate system to an IMU2 coordinate system in a closed state of smart glasses provided by an embodiment of the present application;

[0055] Figure 6 is a flowchart of an open or closed detection method executed by smart glasses provided by an embodiment of the present application;

[0056] Figure 7 is a schematic diagram of an open or closed detection system provided by an embodiment of the present application;

[0057] Figure 8 is a flowchart of calculating the rotation transformation relationship between IMU coordinate systems by an IMU relationship calculation unit provided in an embodiment of the present application;

[0058] Figure 9 is a diagram showing alignment of IMU time by interpolation provided in an embodiment of the present application;

[0059] Figure 10 is a diagram showing a production line process provided in an embodiment of the present application;

[0060] Figure 11 is a diagram showing the IMU gravity direction of an intelligent eyeglass in an unfolded state provided in an embodiment of the present application;

[0061] Figure 12 is a diagram showing the comparison of the IMU gravity directions of two IMUs in each IMU coordinate system in the unfolded state of an intelligent eyeglass provided in an embodiment of the present application;

[0062] Figure 13 is a diagram showing the IMU gravity direction of an intelligent eyeglass in a folded state provided in an embodiment of the present application;

[0063] Figure 14 is a diagram showing the comparison of the IMU gravity directions of two IMUs in each IMU coordinate system in the folded state of an intelligent eyeglass provided in an embodiment of the present application;

[0064] Figure 15 is a diagram showing the layout of two IMUs on a head-mounted earphone provided in an embodiment of the present application;

[0065] Figure 16 is a diagram showing the unfolded state of a head-mounted earphone with three IMUs provided in an embodiment of the present application;

[0066] Figure 17 is a diagram showing the folded state of a head-mounted earphone with three IMUs provided in an embodiment of the present application;

[0067] Figure 18 is a diagram showing the folded state of a three-fold device provided in an embodiment of the present application;

[0068] Figure 19 is a diagram showing the unfolded state of a three-fold device provided in an embodiment of the present application;

[0069] Figure 20 is a diagram showing the intermediate state of a three-fold device provided in an embodiment of the present application;

[0070] Figure 21 is a diagram showing the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] The terms used in the embodiments of the present application are for the purpose of describing particular embodiments only and are not intended to be limiting of the present application.

[0072] The opening and closing detection technology is a technology that relies on various sensors to detect the opening and closing state of an electronic device. For example, a common opening and closing detection technology can determine the opening and closing state of a notebook computer by detecting the electromagnetic induction signal generated when the notebook computer is closed. The above opening and closing detection technology can be applied to foldable electronic devices (such as smart glasses and foldable smart earphones). For foldable electronic devices, the opening and closing state of the device refers to the state of the electronic device being unfolded or closed. When a user uses the above foldable electronic device, the electronic device is in the above unfolded state, and the electronic device can automatically boot or be woken up. When the user finishes using the above foldable electronic device or the user wants to carry the electronic device for convenience, based on the user's usage habits, the electronic device is usually in the above closed state (also known as the fully folded state), and the electronic device can automatically shut down or enter standby mode. That is, the above electronic device can adjust the working state by judging the opening and closing state of itself, thereby providing a more intelligent use experience for the user.

[0073] A common opening and closing detection scheme is to judge the opening and closing state of an electronic device through a proximity light sensor. The above proximity light sensor is a sensor that works based on the principle of photoelectric effect, which determines the existence of an object and the distance between the object and the device by detecting the reflection of a specific wavelength of light by the object. Specifically, the above proximity light sensor can include a light source and a photosensitive detector. Taking the electronic device as a head-mounted device as an example, in the opening and closing detection of the electronic device, when the user's head approaches the above proximity light sensor, the light emitted by the light source will be reflected by the user's head and thus captured by the photosensitive detector. Subsequently, according to the intensity of the reflected light, the proximity light sensor can determine whether the user has worn the device. However, the above opening and closing detection scheme through the proximity light sensor is prone to misjudgment, that is, through the proximity light sensor, the electronic device can only detect whether there is a close object, and cannot directly determine the opening and closing state of the device. In other words, when the electronic device in the closed state (such as smart glasses, etc.) is placed in a pocket, the sensor may also detect that there is a close object (other items in the pocket or bag), but at this time, in order to facilitate carrying, the electronic device should be in the closed state, that is, the electronic device is not in the true unfolded state. It can be understood that in the above solution, since there is an object approaching the device in both cases of the user wearing the electronic device and the electronic device being in the closed state, the accuracy of the above scheme for judging the opening and closing state of the electronic device through the proximity light sensor is limited, and there is a certain probability of false touch.

[0074] Another common solution for open-close detection is to determine the open-close state of the electronic device through electromagnetic induction. Specifically, the electronic device can generate different electromagnetic induction signals in different open-close states, so as to control the circuit switch to realize the automatic control of the electronic device. Taking the above electronic device as an example of smart glasses, the two legs of the smart glasses can be respectively configured with electromagnets, so that when the electronic device is in a closed state, the two legs are closed and close to each other, and the electronic device can detect electromagnetic induction based on a Hall sensor, a magneto-inductive sensor, etc. Sensor generates a closed signal, so that the smart glasses can determine that it is currently in a closed state. As can be seen, this solution relies on the electromagnet device in the electronic device. However, this solution has the following disadvantages: on the one hand, the above-mentioned electromagnet device increases the weight and power consumption of the electronic device, on the other hand, the electromagnet used to detect the open-close state itself is easy to cause interference to other electromagnetic-related devices (such as a loudspeaker) in the electronic device. And through the above electromagnetic induction scheme, the electronic device can only detect one open-close state, for example, in the above example, the smart glasses can only generate a closed signal through the electromagnet device, that is, the smart glasses can only detect the closed state and cannot detect the unfolded state, and cannot fully reflect the open-close state of the electronic device.

[0075] In view of this, the embodiments of the present application provide a device open-close detection method. In this method, the electronic device can obtain motion sensing data of a plurality of rigid members carrying IMUs through a plurality of IMUs. The motion sensing data includes one or more of the following: angular velocity, gravitational acceleration. The electronic device can determine the rotation transformation relationship between the IMU coordinate systems based on the motion sensing data. Subsequently, the electronic device can determine the open-close state of the electronic device based on the rotation transformation relationship between the IMU coordinate systems. The open-close state includes an unfolded state and a closed state. Wherein, the electronic device is a foldable device composed of a plurality of rigid members, and the electronic device includes a plurality of IMUs, and the plurality of IMUs are respectively located on different rigid members. It can be understood that different rigid members can be connected to each other through hinges, shafts or other connecting devices, so as to allow the electronic device to be more compact when not in use, thereby facilitating carrying. The rigid member refers to the part of the main body of the foldable electronic device that is hard and not easy to bend. The electronic device is also referred to as a first electronic device.

[0076] Compared with other sensors such as proximity light sensors, Hall sensors, etc., the IMU itself has lower weight and cost. Therefore, the above method of detecting the opening and closing of the device based on multiple IMUs can accurately determine the opening and closing state of the electronic device while reducing the weight, cost and power consumption of the electronic device. In particular, for some electronic devices that already carry IMUs, such as smart earphones, smart glasses, smart head-mounted displays, etc. with spatial audio function, the above electronic device can reuse the IMU data for the above device opening and closing detection, without the need to additionally increase other sensors.

[0077] The following takes the smart glasses as an example to illustrate the device opening and closing detection method provided by the embodiments of the present application.

[0078] Figure 1 is a layout schematic diagram of a dual-IMU in smart glasses provided by the embodiments of the present application.

[0079] As shown in Figure 1 , the smart glasses can be composed of a glasses front frame and two temples, wherein each of the two temples can carry an IMU. The above-mentioned IMU can include an accelerometer and a gyroscope, therefore, the IMU on the temple can be used to measure the acceleration and angular velocity of the temple. It can be understood that when the temple is completely stationary, the angular velocity detected by the IMU is zero, and the acceleration detected is only the gravitational acceleration.

[0080] It can be understood that the above device opening and closing detection method is actually to judge the opening and closing state of the device by detecting the rotation transformation relationship of the two IMUs in space, and therefore the layout of the IMUs in the smart glasses has certain limitations. That is, if the above device opening and closing detection method needs to be executed by the data detected by the IMU, the IMU should be arranged on the key component of the electronic device. The key component refers to the component of the plurality of rigid components that directly participates in the folding action of the electronic device, that is, the part of the rigid component that the user operates when folding the device. For example, in the example of smart glasses, the two legs are the key components, and the front frame part of the glasses does not directly participate in the folding action. This part is actually used to provide stable support and comfortable wearing experience, and therefore only when the IMU is distributed on the two legs, the opening and closing state of the smart glasses can be judged by the acceleration and angular velocity detected by the IMU. In other words, when the two IMUs are distributed on the left leg and the front frame part of the glasses respectively, the smart glasses can only judge whether the left leg is folded by the angular velocity and acceleration detected by the two IMUs, and cannot judge the current opening and closing state of the right leg, and further cannot judge the opening and closing state of the entire device. Therefore, for smart glasses, when the device opening and closing detection method provided by the embodiments of the present application is used to detect the opening and closing of the device, IMUs should be arranged on the two legs of the smart glasses. The key component and the hardware design and folding mode of the foldable electronic device are strongly related and can be determined by the developer in advance.

[0081] Figure 2 is a schematic diagram of an IMU coordinate system provided by an embodiment of the present application.

[0082] The IMU coordinate system is a reference coordinate system for describing the measurement data of the IMU sensor, that is, the above-mentioned acceleration and angular velocity. Figure 2 As shown in the figure, taking the case that the smart glasses are in a fully unfolded state and placed horizontally on the desktop as an example, the IMU can construct an IMU coordinate system with itself as the origin, the direction of the leg as the x-axis, the direction horizontal to the desktop and perpendicular to the leg as the y-axis, and the direction perpendicular to the ground as the z-axis. The positive half of the x-axis points to the connection between the leg and the front frame of the glasses, the positive half of the y-axis points horizontally to the left of the smart glasses, and the positive half of the z-axis points to the ground. The two IMUs on the two legs of the smart glasses both construct their own coordinate systems in the above-mentioned manner. It can be understood that the coordinate system of the IMU is preset according to the installation position and direction of the IMU in the electronic device, and therefore the axis direction of the IMU coordinate system is relative to the device. In other words, with the movement and rotation of the smart glasses, the IMU coordinate system will also change correspondingly, that is, the IMU coordinate system can dynamically reflect the change of the position of the electronic device in space. Therefore, Figure 2It is only used as a reference example, and does not mean that the axis direction of the IMU coordinate system is fixed, nor does it mean that the IMU coordinate system is only valid in the above case.

[0083] When the smart glasses are in an unfolded state as shown in Figure 1 or Figure 2 , the coordinate systems of the IMUs on the left and right temples are as shown in Figure 3 , the axis directions of IMU1 and IMU2 are both parallel, that is, the rotation angle of the coordinate system of IMU1 relative to the coordinate system of IMU2 is 0 degrees. Among them, the above-mentioned IMU1 is the IMU on the left temple of the smart glasses, also known as the first IMU; the above-mentioned IMU2 is the IMU on the right temple of the smart glasses, also known as the second IMU. In the above structure, the left temple where the first IMU is located is also called the first rigid member, and the right temple where the second IMU is located is also called the second rigid member. The coordinate system of the above-mentioned IMU1 is also called the first coordinate system, and the coordinate system of the above-mentioned IMU2 is also called the second coordinate system. When the smart glasses change from the unfolded state as shown in Figure 1 or Figure 2 to the closed state as shown in Figure 4 , the coordinate systems of the above two IMUs are as shown in Figure 5 . For the convenience of understanding, it can be assumed here that when the smart glasses change from the unfolded state to the closed state, the temples are rotated horizontally around the connecting piece (that is, the z-axis direction of IMU1 or IMU2 in the unfolded state), and the temples after rotation are still in a horizontal state relative to the ground. It can be understood that, since Figure 4 is a top view of the smart glasses, the z-axes of IMU1 and IMU2 in Figure 4 and Figure 5 point to the ground (that is, point to the inside of the paper in the top view), so the z-axis directions are not shown in Figure 4 and Figure 5 , but based on the general left-hand rule for establishing a coordinate system, the z-axis is also determined when the x-axis and y-axis are determined. Figure 5 The solid lines in Figure 5 are the x-axis and y-axis of IMU1, and the dashed lines are the x-axis and y-axis of IMU2. As shown in Figure 2 , when the z-axes of the two IMUs are parallel, the coordinate system of IMU1 can be rotated 120° clockwise around the z-axis to obtain the coordinate system of IMU2. Not limited to the way of establishing the coordinate system as shown in , that is, using the left-hand rule to establish the coordinate system, the IMU can also be established according to the right-hand rule, and the present application does not make special limitations on this. The establishment of the coordinate system in this paper is taken as an example to illustrate the scheme with the left-hand rule (that is, the figure).

[0084] It can be seen that, under the premise of the hardware design and the folding mode being determined, the rotation transformation relationship between the different rigid members in the unfolded state and the closed state of the smart glasses is determined, that is, the rotation transformation relationship between the two IMU coordinate systems distributed on the different rigid members is fixed. Therefore, the smart glasses can determine whether the smart glasses is currently in the unfolded state or the closed state by calculating the rotation transformation relationship between the two IMU coordinate systems in real time.

[0085] It can be understood that the rotation angle of 120° is only used to illustrate that the rotation transformation relationship (or also referred to as the spatial relationship) between the IMU coordinate systems of the smart glasses in the determined open and closed state is determined. The specific rotation angle and the determination of the rotation axis are determined by the hardware design of the smart glasses, which is not specially limited here. That is, when the two legs are in the inclined state in the closed state of the smart glasses, the rotation scheme from the coordinate system of IMU1 to the coordinate system of IMU2 changes in the rotation axis and the rotation angle.

[0086] The electronic device determines the open and closed state of the electronic device based on the rotation transformation relationship between the IMU1 coordinate system and the IMU2 coordinate system. Specifically, the rotation transformation relationship between IMU1 and IMU2 can be calculated in real time and compared with the standard rotation transformation relationship to achieve the open and closed state of the electronic device. The standard rotation transformation relationship includes a first standard rotation transformation relationship and a second standard rotation transformation relationship. The first standard rotation transformation relationship is the rotation transformation relationship between the IMU1 coordinate system and the IMU2 coordinate system in the standard closed state, and the second standard rotation transformation relationship is the rotation transformation relationship between the IMU1 coordinate system and the IMU2 coordinate system in the standard unfolded state. If the deviation between the rotation transformation relationship and the first standard rotation transformation relationship is less than a first threshold, it can be determined that the open and closed state of the electronic device is the closed state. If the deviation between the rotation transformation relationship and the second standard rotation transformation relationship is less than a second threshold, it can be determined that the open and closed state of the electronic device is the unfolded state. The rotation relationship between the IMU1 coordinate system and the IMU2 coordinate system can be represented by a rotation axis and a rotation angle.

[0087] Figure 6 is a flowchart of a method for performing open and closed detection of a smart glasses provided by an embodiment of the present application. Optionally, as shown in Figure 6As shown, the smart glasses can use a rotation matrix to represent the rotation axis and rotation angle described above, and then the smart glasses can calculate the Euclidean distance between the rotation matrix between the two IMU coordinate systems and the rotation matrix between the IMU1 coordinate system and the IMU2 coordinate system in the standard closed state. When the Euclidean distance is less than a first threshold, the smart glasses can identify that it is in a closed state and perform standby screen-off and other operations. The smart glasses can also calculate the Euclidean distance between the rotation matrix between the two IMU coordinate systems and the rotation matrix between the IMU1 coordinate system and the IMU2 coordinate system in the standard open state. When the Euclidean distance is less than a second threshold, the smart glasses can identify that it is in an open state and perform active screen-on and other operations. Not limited to the scheme as Figure 6 shown, other methods that can calculate the similarity between the rotation transformation relationship between the two IMU coordinate systems in the current smart glasses and the rotation transformation relationship between the two IMU coordinate systems in the standard open state and the standard closed state can be used here, and the embodiments of the present application do not make special limitations on this.

[0088] Specifically, the smart glasses execute the open-close state detection method as Figure 7 shown using the open-close detection system as Figure 6 shown.

[0089] As shown in Figure 7 , the open-close detection system can include a plurality of IMUs (such as IMU1, IMU2, and other IMUs, not shown), an integrated input unit, an IMU relationship calculation unit, an open-close state matching unit, an open-close state maintenance unit, and other modules. Figure 7

[0090] The integrated input unit is used to integrate the motion sensing data (such as acceleration data, angular velocity data) detected by the IMU from the firmware, that is, the integrated input unit can collect the motion sensing data of the IMU on the smart glasses and input the collected motion sensing data to the IMU relationship calculation unit.

[0091] The IMU relationship calculation unit is used to calculate the rotation transformation relationship between the plurality of IMU coordinate systems and output the calculation result to the open-close state matching unit.

[0092] ​The opening and closing state matching unit is configured to compare the rotation transformation relationship between the plurality of IMU coordinate systems calculated by the IMU relationship calculation unit with a preset rotation transformation relationship between IMU coordinate systems, i.e., the opening and closing state matching unit is configured to calculate the Euclidean distance between rotation matrices, and obtain the recognition result of the opening and closing state of the smart glasses based on the calculation result. Subsequently, based on the recognition result, the opening and closing state matching unit can output a corresponding unfolded state signal or a closed state signal to the opening and closing state maintenance unit. For example, in the smart glasses, a state signal of 1 represents that the state signal is an unfolded state signal, and a state signal of 0 represents that the state signal is a closed state signal. However, the present application is not limited to this, and the unfolded state signal and the closed state signal are not specially limited in the present application in terms of the presentation form in the electronic device.

[0093] The opening and closing state maintenance unit can perform the operation of turning on the screen or turning off the screen and entering the standby mode according to the received state signal.

[0094] It can be understood that the integrated input unit, the IMU relationship calculation unit, the opening and closing state matching unit, and the opening and closing state maintenance unit are all functional units in the smart glasses, and therefore, the functional units can communicate with each other through inter process communication (IPC). For example, the opening and closing state matching unit outputs a corresponding state signal to the opening and closing state maintenance unit, which actually writes the latest state signal into a shared memory. Subsequently, the opening and closing state maintenance unit can read the latest state signal from the shared memory. Therefore, the opening and closing state maintenance unit can perform the operation of turning on the screen or turning off the screen and entering the standby mode according to the read latest state signal. For example, when the state signal is 1, the opening and closing state maintenance unit performs the operation of turning on the screen; when the state signal is 0, the opening and closing state maintenance unit performs the operation of turning off the screen and entering the standby mode. The communication of other functional modules can also be based on this example, and will not be described here.

[0095] Figure 8 A flowchart of the process in which the IMU relationship calculation unit calculates the rotation transformation relationship between IMU coordinate systems is provided.

[0096] As Figure 8As shown, after the IMU relationship calculation unit obtains the motion sensing data of the plurality of IMUs from the integrated input unit, the IMU relationship calculation unit can determine whether all the IMUs are in the same motion state. It can be understood that when determining the opening and closing state of itself, the smart glasses should be in a relatively rigid body state, i.e., at this time the smart glasses will not temporarily undergo a dramatic deformation, and the opening and closing state is relatively stable. At this time, the opening and closing state of the device is determined and the working state of the device is changed to improve the user experience. At the same time, only when the plurality of IMUs located on different rigid components of the smart glasses are in a motion state or the plurality of IMUs are in a stationary state, the smart glasses can be in a relatively rigid body state. Among them, the smart glasses detect that the plurality of IMUs are in a stationary state, i.e., the smart glasses are in a stationary state. This branch is not shown in Figure 8 the description, and how the IMU relationship calculation unit of the smart glasses determines the rotation transformation relationship between the IMU coordinate systems will be described in detail later. At this time, the smart glasses detect that the plurality of IMUs are in a motion state, i.e., the smart glasses detect that the plurality of rigid components on which the IMUs are arranged are in a motion state, which can include the case that the user is wearing the smart glasses or successfully wearing the smart glasses. Therefore, when all the IMUs are in a motion state, the IMU relationship calculation unit of the smart glasses can calculate the rotation transformation relationship between the IMU coordinate systems based on the gyroscope data (i.e., angular velocity data).

[0097] Optionally, the IMU relationship calculation unit can determine whether a single IMU is in a motion state by whether the average modulus of the angular velocity is greater than a modulus threshold. If the average modulus of the angular velocity is greater than or equal to the modulus threshold, the IMU relationship calculation unit can determine that the IMU is in a motion state; if the average modulus of the angular velocity is less than the modulus threshold, the IMU relationship calculation unit can determine that the IMU is in a stationary state. Exemplarily, the IMU relationship calculation unit can take the angular velocity of the last 1000 frames of each IMU, and calculate the average modulus of the angular velocity measured by the IMU by the following formula:

[0098]

[0099] wherein Gyro(m) represents the angular velocity data of the last m frame of the IMU. Assuming that the modulus threshold is 0.3 rad / s, when the above , the IMU relationship calculation unit can determine that the IMU is in a motion state, otherwise, the IMU is in a stationary state.

[0100] It can be understood that if the IMU relationship calculation unit determines that all IMUs are in a motion state, the electronic device can perform the following method: obtaining the angular velocity of the plurality of rigid members carrying the IMUs through the plurality of IMUs, then determining the rotation transformation relationship between the plurality of IMU coordinate systems based on the angular velocity of the plurality of rigid members carrying the IMUs, and further determining the open-close state of the electronic device based on the rotation transformation relationship between the plurality of IMU coordinate systems. In the example of the smart glasses as shown in Figure 1 the actual method executed is as follows: obtaining the angular velocity of the left temple through IMU1, obtaining the angular velocity of the right temple through IMU2; determining the rotation transformation relationship between the IMU1 coordinate system and the IMU2 coordinate system based on the angular velocity of the left temple and the angular velocity of the right temple, and further determining the open-close state of the smart glasses. Therefore, if the IMU relationship calculation unit determines that all IMUs are in a motion state, that is, the smart glasses detect that the plurality of rigid members carrying the IMUs are in a motion state, the IMU relationship calculation unit can further calculate the rotation transformation relationship between the IMU coordinate systems. The rotation transformation relationship obtained at this time is also called the first rotation transformation relationship, the rotation angle in the first rotation relationship is also called the first rotation angle, and the rotation axis in the first rotation relationship is also called the first rotation axis, that is, the first rotation relationship can be represented by the first rotation axis and the first rotation angle.

[0101] Specifically, the rotation axis and the rotation angle in the above rotation transformation relationship are obtained by minimizing the cumulative deviation of the actual angular velocity of IMU2 and the estimated angular velocity of IMU2, wherein the estimated angular velocity of IMU2 is calculated based on the actual angular velocity of IMU1.

[0102] Optionally, the rotation transformation relationship between the two IMU coordinate systems can be represented by a quaternion Q, that is:

[0103] Gyro (IMU2) ′=Q·Gyro (IMU1) ·Q -1 (1)

[0104]

[0105] The above formula (1) is used to illustrate the process of converting the gyro data of IMU1 from its own coordinate system to the coordinate system of IMU2, that is, the process of obtaining the coordinate system of IMU2 through rotation transformation of the coordinate system of IMU1. Wherein, Gyro (IMU1) represents the angular velocity vector measured by IMU1 in its own coordinate system (i.e. the actual angular velocity of IMU1), and Gyro (IMU2) ′ represents the estimated angular velocity of IMU2 calculated based on the actual angular velocity of IMU1. Q -1denotes the inverse of Q. Wherein, the expansion of Q is the above formula (2), and the expansion of Q -1 is the above formula (3). In the above formula (2), formula (3), θ1 is the rotation angle from the IMU1 coordinate system to the IMU2 coordinate system, and the rotation axis is u = (u x , u y , u z ), and the above i, j, k are imaginary units in the quaternion calculation formula.

[0106] It can be understood that Q changes with the change of the rotation angle θ1 and the rotation axis u1. The essence of the above IMU relationship calculation unit further calculating the relative rotation between the IMU coordinate systems is to find a suitable rotation angle θ1 and rotation axis u1, and then obtain a suitable quaternion Q so that the coordinate system of IMU1 can be changed by the above formula (1) to maximize the proximity to the coordinate system of IMU2, that is, the found Q can make the above Gyro (IMU2) ' maximally close to Gyro (IMU2) , Gyro (IMU2) denotes the angular velocity vector measured by IMU2 in its own coordinate system (i.e. the actual angular velocity of IMU2). Optionally, the above IMU relationship calculation unit can construct the following optimization function, and then solve the above quaternion Q:

[0107]

[0108] It can be understood that the above formula δ is used to represent the cumulative effect of the difference between the IMU2 gyroscope data rotated by the quaternion Q and the IMU1 gyroscope data in time and space three axes, also known as cumulative bias. Optimizing this function, that is, minimizing this difference, can help us find the best quaternion Q, so as to accurately describe the rotation transformation relationship between the two IMU coordinate systems. In some embodiments, the above IMU relationship calculation unit can use gradient descent to determine the rotation angle θ1 and the rotation axis u1 when the above formula δ takes the minimum value.

[0109] Optionally, after determining the rotation angle θ1 and the rotation axis u1, the IMU relationship calculation unit can construct the rotation matrix R according to the Rodrigues formula:

[0110] R = I + sin(θ)K + (1-cos(θ))K 2 (4)

[0111]

[0112] wherein, in the above formula (4), I is a 3x3 unit matrix, K is an anti-symmetric matrix, the expansion of K is formula (5), and θ is the rotation angle θ1 calculated above. In formula (5), K is constructed based on the base rotation axis u=(u x , u y , u z ), and when formula (5) is used, u=u1 can be used for calculation. The rotation matrix R constructed using the above formula (4) and formula (5) can also be used to perform the coordinate conversion of IMU2 to IMU1 described above. Subsequently, this real-time calculated rotation matrix can be output by the IMU relationship calculation unit to the open-close state matching unit described above.

[0113] The open-close state matching unit described above can use the Euclidean distance d1 between the standard rotation matrix R-unfold-std between the IMU1 coordinate system and the IMU2 coordinate system in the standard unfolded state and the real-time calculated rotation matrix R, and the Euclidean distance d2 between the standard rotation matrix R-fold-std between the IMU1 coordinate system and the IMU2 coordinate system in the standard closed state and the real-time calculated rotation matrix R, respectively. The Euclidean distance d (such as d1 or d2) can be calculated based on the following formula:

[0114]

[0115] wherein, a ij is the element in the i-th row and j-th column of the standard rotation matrix between the IMU1 coordinate system and the IMU2 coordinate system in the standard unfolded state or the standard closed state, and b ij is the element in the i-th row and j-th column of the real-time calculated rotation matrix R. When d1 is less than or equal to a first threshold value, the open-close state matching unit can identify that the current smart glasses are in a closed state, and then output a closed state signal. When d2 is less than or equal to a second threshold value, the open-close state matching unit can identify that the current smart glasses are in an unfolded state, and then output an unfolded state signal. In some embodiments, the open-close state matching unit described above requires that d1 be less than or equal to the first threshold value for a first time in order to identify that the current smart glasses are in a closed state, and requires that d2 be less than or equal to the second threshold value for a second time in order to identify that the current smart glasses are in an unfolded state. The first time and the second time are pre-set by the developer. Optionally, the first time can be equal to the second time.

[0116] Since the timestamps at which the motion sensing data is generated by different IMUs on the smart glasses can not be the same, as Figure 8As shown, in some embodiments, before determining whether all IMUs are in the same motion state, the IMU relationship calculation unit can also align the time when different IMUs generate motion sensing data with one of the IMUs through interpolation.

[0117] Figure 9 is a schematic diagram of aligning IMU time through interpolation provided by an embodiment of the present application.

[0118] When interpolating, the IMU relationship calculation unit can take the IMU that first generates motion sensing data in a period as the reference time. For example, Figure 9 As shown, the IMU relationship calculation unit takes the time when IMU1 generates motion sensing data as the reference time when interpolating, and interpolates IMU2 according to the time when IMU1 generates the motion sensing data. The following describes the scheme of aligning IMU time through interpolation by taking the first interpolation of IMU2 as an example. The time when IMU2 first generates motion sensing data IMU2 t1 is t1, the time when IMU2 secondly generates motion sensing data IMU2 t2 is t2, the interpolation time, i.e., the time when IMU1 secondly generates motion sensing data is t3, then the IMU2 data IMU2 t3 at t3 generated through linear fitting according to the front and rear frame data is:

[0119]

[0120] wherein, IMU2 t = {Acc IMU2 (t), Gyro IMU2 (t)}, i.e., the motion sensing data of IMU2 at t generated by the sensor and generated through fitting all include the acceleration data Acc IMU2 (t) and the angular velocity data Gyro IMU2 (t) of IMU2 at t.

[0121] It can be understood that, since the frequency of the IMU generating motion sensing data is high, and the interval of each data is small, the motion sensing data generated through linear interpolation is approximately equal to the motion sensing data that the IMU can actually generate at the time. The present application does not specially limit how the smart glasses align different IMU time.

[0122] In the above determination process, the R-unfold-std and R-fold-std of the smart glasses can be calculated through, for example, Figure 10The shown production line process is determined in advance. The production line process refers to the process that the smart glasses need to perform in advance in the production process. Specifically, the smart glasses can collect and store the rotation matrix between the two IMU coordinate systems in the closed state in the production line process as R-fold-std. Correspondingly, the smart glasses can also collect and store the rotation matrix between the two IMU coordinate systems in the unfolded state in the production line process as R-unfold-std. It can be understood that the above production line process actually repeatedly performs the above calculation process until the corresponding rotation matrix is obtained in the case of determining the opening and closing state of the smart glasses. That is, compared with real-time opening and closing state detection, the production line process can only not perform the subsequent Euclidean distance solving step. Therefore, the specific process of the production line process can refer to the related description in the foregoing, which will not be described here.

[0123] It can be understood that the opening and closing detection of the device by the angular velocity data of the IMU has certain limitations. The above limitations are reflected in that the above scheme judges the opening and closing state of the device on the premise that the smart glasses still have slight shaking when they are in the unfolded state or the closed state. These slight shaking may be caused by the user's wearing behavior (for example, the user's head has a slight turning action when wearing the smart glasses), so that the gyroscope in the IMU of the smart glasses can detect the angular velocity data, so that the rotation matrix can be calculated in real time, and then the calculated rotation matrix and the standard rotation matrix in the unfolded state or the closed state are compared to judge the state of the smart glasses. This means that when the smart glasses are completely stationary, the angular velocity data cannot be obtained by the above scheme, and the rotation matrix cannot be calculated to judge the opening and closing state of the smart glasses. Since the smart glasses are in a completely stationary state, at this time, the only acceleration that the IMU in the smart glasses can detect is the gravitational acceleration. Therefore, in some embodiments, the device opening and closing detection method provided by the embodiments of the present application can also use the gravity direction of the multi-IMU to determine the opening and closing state of the device. The gravity direction of the IMU refers to the direction of gravity in the IMU coordinate system, which is embodied by the gravitational acceleration.

[0124] Figure 11Fig. 1 is a schematic diagram of the IMU gravity direction of an intelligent glasses in an unfolded state according to an embodiment of the present application. It can be understood that, if the ground is taken as the reference system, the gravity direction of any IMU is always directed to the center of the earth, so in the ground reference system, the gravity direction is constant. However, the position of the IMU will change with the change of the opening and closing state of the device, so the IMU coordinate system will also change with the change of the opening and closing state of the device, such as rotation, tilt, etc. Therefore, if the ground is still taken as the reference system at this time, the gravity direction does not change, the IMU coordinate system tilts or rotates, and when the scene is converted to the IMU coordinate system as the reference system, the gravity direction will change with the movement of the IMU, that is, the gravity direction in the IMU coordinate system can not be a vector along the z-axis vertically downward. Although for a single IMU, when the placement posture of the intelligent glasses is different, the direction of the gravity in the above-mentioned IMU coordinate system is different, for example, the intelligent glasses are placed horizontally on the horizontal table top and the intelligent glasses are placed upside down on the horizontal table top, the direction of the gravity in the IMU coordinate system of the single IMU is opposite. However, for the intelligent glasses in the rigid state, no matter whether it is placed upright or upside down, the spatial relationship between the gravity directions in the multiple IMU coordinate systems on the intelligent glasses at the same time is constant, that is, the relative direction relationship between the multiple IMUs is constant, and the above-mentioned constant relative direction relationship can be determined by analyzing the direction of the gravity in each IMU coordinate system. And because the relative direction relationship between the multiple IMUs in the intelligent glasses is different in different opening and closing states, the intelligent glasses can determine the opening and closing state of the intelligent glasses through the gravity directions of the multiple IMUs thereon.

[0125] Therefore, in the above-mentioned case, the electronic device actually adopts the following method to detect the opening and closing state of the device: acquiring the gravity acceleration of the multiple rigid members carrying the IMUs through the multiple IMUs, determining the rotation transformation relationship between the multiple IMU coordinate systems based on the gravity acceleration of the multiple rigid members carrying the IMUs, and then determining the opening and closing state of the electronic device based on the rotation transformation relationship between the multiple IMU coordinate systems. Figure 1 In the example of the intelligent glasses shown in Fig. 1, the above-mentioned method can be specifically the following scheme: acquiring the gravity acceleration of the left leg through the IMU1, acquiring the gravity acceleration of the right leg through the IMU2, and then the intelligent glasses can determine the rotation transformation relationship between the IMU1 coordinate system and the IMU2 coordinate system based on the gravity acceleration of the left leg and the gravity acceleration of the right leg, and determine the opening and closing state of the intelligent glasses based on the rotation transformation relationship. The rotation transformation relationship determined by the above-mentioned method is also called the second rotation transformation relationship. The rotation angle in the second rotation relationship is also called the second rotation angle, and the rotation axis in the second rotation relationship is also called the second rotation axis, that is, the second rotation relationship can be represented by the second rotation axis and the second rotation angle.

[0126] Specifically, when the intelligent glasses are inFigure 11 in the unfolded state, Figure 12 (a) of FIG. 1A represents the direction of gravity in the IMU1 coordinate system, Figure 12 (b) of FIG. 1A represents the direction of gravity in the IMU2 coordinate system, as Figure 12 (a) and (b) of FIG. 1A, in the unfolded state, the directions of gravity of the IMU1 and the IMU2 of the smart glasses are both along the z-axis downward. When the smart glasses are in the closed state as shown in Figure 13 (a) of FIG. 1A, the direction of gravity of the IMU1 of the smart glasses is as shown in Figure 14 (b) of FIG. 1A, the direction of gravity of the IMU2 of the smart glasses is as shown in Figure 14 It can be understood that, due to the thickness of the temple, when the smart glasses are in the closed state as shown in Figure 13 , one temple of the smart glasses will partially overlap the other temple, and this design will cause a slight difference in the tilt angles of the two temples relative to the central axis of the glasses. Therefore, the smart glasses in the closed state are not strictly left-right symmetrical in structure, that is, the included angle 1 between the left temple and the central axis and the included angle 2 between the right temple and the central axis are not equal.

[0127] For example, the included angle 1 between the left temple and the central axis is 60°, and based on the above design of the smart glasses in the closed state, the tilt angle of the right temple can be larger, that is, the included angle 2 in Figure 13 is less than 60°, which is 55°. At this time, if the direction of gravity of the IMU is represented by a unit vector, then Figure 14 the direction of gravity Acc IMU1 in (a) of FIG. 1A can be represented as Figure 14 the direction of gravity Acc IMU2 in (b) of FIG. 1A can be represented as (-sin 35°, 0, -cos 35°), and therefore, the smart glasses can use the unit vectors of the two directions of gravity and the following formula to calculate the rotation angle θ2 between the two IMU coordinate systems:

[0128]

[0129] Since Acc IMU1 and Acc IMU2are unit vectors, thus, the angle between the coordinate systems of the two IMUs can be calculated based on the above formula. It can be understood that, since θ2 is determined according to the cosine value, and for the same cosine value, there are two θ2 values within 360°, thus, the above θ should be limited within 0-180°, i.e., for the electronic device represented by the smart glasses, the folding angle is limited within 180°. However, only determining the rotation angle is insufficient to describe the rotation transformation relationship between the two IMUs, thus, the smart glasses can further calculate the rotation axis u2 between the coordinate systems of the two IMUs through the above two unit vectors of the gravity direction and the following formula:

[0130] u2 = Acc IMU1 x Acc IMU2

[0131] It can be understood that, in the above formula, the vector cross product means finding a vector u, which is perpendicular to the gravity direction in IMU1 and the gravity direction in IMU2, i.e., the vector u2 can be the rotation axis between the coordinate systems of the two IMUs. Subsequently, the smart glasses can calculate the rotation matrix R between the coordinate systems of the two IMUs based on the Rodrigues formula (i.e., the above formula (4)) and the above formula (5). At this time, θ in formula (4) = θ2, and u = u2. The above process of calculating R can be performed by the IMU relationship calculation unit in Figure 7 . It can be understood that, in the above formula, the subscripts of θ1 and θ2 only distinguish the method of calculating the rotation angle in which scenario, and do not mean that there are multiple values of the rotation angle in the same opening and closing state, and the same applies to u1 and u2.

[0132] Subsequently, the smart glasses can also calculate the Euclidean distance between R and R-unfold-std, and the Euclidean distance between R and R-fold-std, and further determine the opening and closing state of the smart glasses. Here, the process and method of the smart glasses calculating the Euclidean distance and determining the opening and closing state of the smart glasses based on the calculation result are similar to the related content in the foregoing, i.e., the method of the smart glasses determining the opening and closing state of the smart glasses based on the above second rotation transformation relationship is similar to the method of the smart glasses determining the opening and closing state based on the above first rotation relationship, which will not be described herein again.

[0133] It can be understood that, in the above calculation process, there is a special case, i.e., for the unfolded state as shown in Figure 11 , since the two vectors Acc IMU1 and Acc IMU2 are parallel, thus, in the above vector cross product calculation, the calculated rotation axis u is actually a zero vector, and thus the above rotation matrix R cannot be calculated.

[0134] In some embodiments, the smart glasses can confirm that there is only the case as shown in Figure 11 Acc IMU1 and Acc IMU2 are parallel, the smart glasses can skip the subsequent calculation steps and directly determine that the smart glasses are in the unfolded state when v is calculated as a zero vector.

[0135] In other embodiments, if there is not only the case as shown in Figure 11 Acc IMU1 and Acc IMU2 are parallel, but also other cases that make Acc IMU1 and Acc IMU2 are parallel, the smart glasses can give up detecting the current opening and closing state of the smart glasses after calculating u2 as a zero vector. It can be understood that the case that Acc IMU1 and Acc IMU2 are parallel has a low probability of occurring during the entire use of the smart glasses, and therefore, appropriately giving up detecting the opening and closing state of the smart glasses in this case does not affect the user's experience in actual application.

[0136] Not limited to smart glasses, the electronic device in the device opening and closing detection method described above can also include foldable headsets, smart head-mounted displays, and the like.

[0137] Not limited to the smart glasses described above and Figure 15 the headset shown in includes two IMUs, in some embodiments, the electronic device in the device opening and closing detection method described above can also use more than two IMUs to determine the opening and closing state of the device. For example, the headset can include three IMUs, Figure 16 a schematic diagram of the unfolded state of the headset with three IMUs is provided, Figure 17 and a schematic diagram of the closed state of the headset.

[0138] It can be understood that in the smart glasses described above, the left and right legs are key components, and in the headset described above, the left and right legs are key components. The IMU mounted on the key components can achieve the device opening and closing detection method provided in the embodiments of the present application. However, for these electronic devices, increasing IMUs on other rigid components (such as the headset bridge of the headset and the front frame of the smart glasses) in addition to the key components collects more motion sensing data for calculation and comparison, and the scheme of using multiple IMUs to detect the opening and closing of the device is more accurate than the scheme of using double IMUs to detect the opening and closing of the device.

[0139] Not limited to electronic devices with symmetrical structure such as smart glasses and headsets, the above-mentioned device opening and closing detection method can also be used for some devices with more complex folding relationship, such as electronic devices with diversified structure, which can realize opening and closing detection of the device through multiple IMUs. The electronic device with diversified structure refers to an electronic device with multiple forms in design, such as foldable smartphones, foldable Bluetooth keyboards, etc. These devices usually have innovative hinge mechanisms that allow them to be folded when not in use to save space, or unfolded when needed to provide a larger display or input area.

[0140] For the above-mentioned electronic device with diversified structure, the opening and closing detection of the device can not be limited to detecting the above-mentioned unfolded state and the above-mentioned closed state, but also can detect the intermediate state. For example, as shown in the three-fold device, Figure 18 consisting of three rigid members, each of which carries an IMU, and the three rigid members are connected through a connecting piece (i.e. Figure 18 the circular part shown in the middle, such as a hinge). Figure 18 shows that the three-fold device is in a closed state, Figure 19 shows that the three-fold device is in an unfolded state. It can be understood that the above-mentioned unfolded state refers to the state that the device is completely unfolded, that is, the device cannot be further unfolded, at this time the device usually reaches its maximum size and use area. Similarly, the above-mentioned closed state refers to the state that the device is completely closed, that is, the device cannot be further folded. In addition to this, the three-fold device can also exist in an intermediate state as shown in Figure 20 At this time, the rigid members at both ends form a 90° perpendicular angle with the rigid member in the middle. As shown in the intermediate state, Figure 20 the intermediate state can be transformed into a closed state as shown in Figure 18 by further folding, or can be transformed into an unfolded state as shown in Figure 19 by further unfolding. Figure 18 to Figure 20 Only the Y-axis direction of the IMU coordinate system is shown, so it can be seen that the relative position and direction of each IMU coordinate system on the above-mentioned three-fold device changes under different opening and closing states. The three-fold device can use the above-mentioned changes to identify the spatial pose of the device. That is, by analyzing the rotation transformation relationship between these IMUs, the above-mentioned three-fold device can accurately determine the current opening and closing state of the device. Specifically, for the three-fold device as shown in Figure 18 to Figure 20 the y-axis direction in the IMU coordinate system is as shown, the z-axis direction is perpendicular to the paper, and the x-axis direction of the coordinate system of each IMU can be determined based on the left-hand rule. The three-fold device in the unfolded state, Figure 19In the open / closed state shown, IMU1, IMU2, and IMU3 are sequentially supported on the rigid components from left to right. If the three-folding device detects that the rotation angle between IMU1 and IMU2 is 180°, the rotation angle between IMU2 and IMU3 is 180°, and the rotation angle between IMU1 and IMU3 is 0°, and the rotation axis is the axis where the connecting parts are located (i.e., the z-axis perpendicular to the paper), then the three-folding device can determine that it is in the open / closed state. Figure 18 The three-folding device is in the closed state shown. If the rotation angle between IMU1 and IMU2 is 0°, the rotation angle between IMU2 and IMU3 is 0°, and the rotation angle between IMU1 and IMU3 is 0°, and the rotation axes of the above rotations are all the axes where the connecting parts are located (i.e., the z-axis perpendicular to the paper), then the three-folding device can determine that the device is in the closed state. Figure 19 The unfolded state is shown. If the three-folding device detects that the rotation angle between IMU1 and IMU2 is 90° and the rotation angle between IMU2 and IMU3 is 90°, while the rotation angle between IMU1 and IMU3 is 0°, and the rotation axes of the above rotations are all the axes where the connecting parts are located (i.e., the z-axis perpendicular to the paper), then the three-folding device can determine that the device is in the unfolded state. Figure 20 The intermediate states shown are as follows. It is understandable that for determining the open / closed state of an electronic device carrying multiple IMUs, the corresponding open / closed state can only be identified when the rotational transformation relationships between all pairs of these IMUs are satisfied. It is also understandable that, similar to determining the unfolded and closed states, for intermediate states, a corresponding standard rotational transformation relationship, called the third standard rotational transformation relationship, is determined during the production line process. When the deviation between the real-time calculated rotational transformation relationship (including the first and second rotational transformation relationships) and the aforementioned third standard rotational transformation relationship is less than a third threshold, the electronic device (such as...) can be identified as open / closed. Figure 18 The open / closed state of the three-fold device shown is the aforementioned intermediate state. In some embodiments, the opening / closed state matching unit identifies that the current electronic device is in the intermediate state only if the aforementioned deviation is less than or equal to a third threshold and can be maintained for a third time. The aforementioned third time is preset by the developers. Optionally, the third time can be equal to the first time or the second time. The specific calculation process can be referred to the relevant description above, and will not be repeated here.

[0141] Understandably, the premise for detecting the aforementioned intermediate states is that electronic devices with the aforementioned diverse structures can stably maintain these intermediate states. To ensure that electronic devices with diverse structures can stably maintain these intermediate states, they can possess one or more locking mechanisms. These locking mechanisms can include physical locking devices, such as hinges, latches, or magnetic designs, which can lock the device at specific usage angles or configurations, providing stable support at multiple angles to ensure that the electronic device does not change its state due to its own weight or slight touches when in the intermediate state. Furthermore, some electronic devices can also employ electronically controlled locking mechanisms, using software control to fix the form. The existence of these locking mechanisms not only improves the ease of use and stability of the device but also enhances its adaptability in different usage scenarios. For example, some foldable screen devices are designed with in mind that users may need to use the device at multiple angles, therefore providing stable locking points at multiple angles so that users can adjust the form of the device as needed, thus providing the best user experience in different usage environments, such as watching videos, reading documents, or making video calls.

[0142] Therefore, it is not limited to such Figure 20 The aforementioned intermediate state, as shown, refers to a situation where the multi-structured electronic device can exist in several intermediate states (such as a first intermediate state, a second intermediate state, etc.) during the opening and closing process. The electronic device can determine which intermediate state it is in by analyzing the spatial relationships between these IMUs. The number and characteristics of these intermediate states depend on the mechanical structure of the electronic device. It is understandable that the electronic device can enrich its interaction methods through these intermediate states during opening and closing; for example, when the device is opened or closed to one of these intermediate states, the electronic device can activate a split-screen mode, etc.

[0143] Figure 21 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device is used to perform the above-described device opening and closing detection method.

[0144] like Figure 21 As shown, the electronic device may include components such as a processor 311, a memory 312, a wireless communication processing module 313, a power switch 314, and an IMU 315. The components within the device are connected via a bus and communicate with each other based on the bus.

[0145] The processor 311 can include one or more processing units, for example: the processor 311 can include an application processor, a modem processor, a graphics processor, an image signal processor, a controller, a video codec, a digital signal processor, a baseband processor and / or a neural network processor, etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors. The controller can generate operation control signals according to instruction operation codes and timing signals, complete the control of fetching instructions and executing instructions.

[0146] The memory 312 is coupled with the processor 311, for storing various software programs and / or groups of instructions. The memory 312 can be used to store computer executable program codes, which include instructions. The processor 311 executes various functional applications and data processing of the device by running the instructions stored in the memory 312. The processor 311 can also be provided with a memory for storing instructions and data.

[0147] Among them, the memory 312 can include one or more random access memories and one or more non-volatile memories. The random access memory can be directly read and written by the processor 311. The random access memory can be used to store executable programs (such as machine instructions) of operating systems or other running programs, and can also be used to store user and application data, etc. The non-volatile memory can also store executable programs and store user and application data, etc. The executable programs and user data stored in the non-volatile memory can be loaded in advance into the random access memory for the processor 311 to read and write directly.

[0148] The executable program codes and user data for implementing the device opening and closing detection method provided by the embodiments of the present application can be stored in the non-volatile memory. In the process of implementing the above-mentioned device opening and closing detection method, the remote controller can load the non-volatile memory executable program codes and user data into the random access memory to provide the user with an efficient and convenient device control method.

[0149] The wireless communication processing module 313 can provide wireless communication solutions including WLAN, such as Wi-Fi, Bluetooth communication, ZigBee communication, NFC communication, infrared communication, UWB communication, etc.

[0150] The power switch 314 can be used to control the power supply of the device, and then power the processor 311, the memory 312, the wireless communication processing module 313, the IMU 315, etc.

[0151] The IMU 315 includes an accelerometer and a gyroscope for measuring acceleration and angular velocity of the electronic device in three-dimensional space, respectively. In some embodiments, the IMU 315 further includes a magnetometer for measuring the magnetic field around the object. In the embodiments of the present application, there are multiple IMUs 315, which are used to collect motion sensing data of different rigid structures of the electronic device, including but not limited to the tilt angle and acceleration of the remote controller, and the like. The motion sensing data is used by the processor 311 to calculate the spatial relationship between the multiple IMUs, and then determine the opening and closing state of the electronic device. It can be understood that for electronic devices that do not use the device opening and closing detection method provided by the embodiments of the present application, there is usually only one IMU in the electronic device for detecting the acceleration and angular velocity of the entire electronic device. Therefore, when there are multiple IMUs 315 in the electronic device and there are no corresponding ambient light sensors, Hall sensors, magnetic induction sensors or other sensors for opening and closing detection, if the electronic device can still achieve the above opening and closing detection effect, it can be considered that the device opening and closing detection method provided by the embodiments of the present application is executed.

[0152] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the device. In other embodiments of the present application, the device can include more or fewer components than illustrated, or combine certain components, or split certain components, or different component arrangements. The components can be implemented in hardware, software, or a combination of software and hardware.

[0153] Those skilled in the art should be aware that in one or more of the examples described above, the functions described in the embodiments of the present application can be implemented in hardware, software, firmware or any combination thereof. When implemented in software, the functions can be stored in a computer readable medium or transmitted as one or more instructions or code on a computer readable medium. The computer readable medium includes computer storage media and communication media, and the communication media includes any medium that facilitates the transfer of computer programs from one place to another. The storage medium can be any available medium accessible by a general purpose or special purpose computer. The embodiments of the present application also provide a computer program product, which includes a computer program that can implement the steps in the various method embodiments when the computer program is run on a processor.

[0154] The above detailed description of the specific implementation of the present application further describes the purpose, technical solutions and beneficial effects of the embodiments of the present application. It should be understood that the above is only a specific implementation of the embodiments of the present application and does not limit the protection scope of the embodiments of the present application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the embodiments of the present application shall be included in the protection scope of the embodiments of the present application.

Claims

1. A method for detecting the opening and closing of equipment, characterized in that, The method is applied to a first electronic device, which is a foldable device composed of multiple rigid components. The first electronic device includes multiple inertial mass detection units (IMUs), each located on a different rigid component. Motion sensing data of the multiple rigid components carrying the IMUs are acquired through the multiple IMUs, and the motion sensing data includes one or more of the following: angular velocity, gravitational acceleration; Based on the motion sensing data, determine the rotational transformation relationship between the multiple IMU coordinate systems; Based on the rotational transformation relationship between the multiple IMU coordinate systems, the opening and closing state of the first electronic device is determined, including an unfolded state and a closed state.

2. The method according to claim 1, characterized in that, The acquisition of motion sensing data of the multiple rigid components carrying the IMUs through the multiple IMUs specifically includes: The angular velocities of the multiple rigid components carrying the IMUs are obtained through the multiple IMUs; Determining the rotation transformation relationship between the multiple IMU coordinate systems based on the motion sensing data specifically includes: The first rotational transformation relationship between the multiple IMU coordinate systems is determined based on the angular velocities of the multiple rigid components carrying the IMUs; Based on the rotational transformation relationship between the multiple IMU coordinate systems, the opening and closing state of the first electronic device is determined, specifically including: The opening / closing state of the first electronic device is determined based on the first rotational transformation relationship between the multiple IMU coordinate systems.

3. The method according to claim 2, characterized in that, The plurality of IMUs includes a first IMU and a second IMU, the first IMU being located on a first rigid member, and the second IMU being located on a second rigid member. The first rigid member and the second rigid member are contained within the plurality of rigid members. The step of obtaining the angular velocity of the plurality of rigid members carrying the IMUs through the plurality of IMUs specifically includes: The angular velocity of the first rigid component is obtained through the first IMU, and the angular velocity of the second rigid component is obtained through the second IMU; The determination of the first rotational transformation relationship between the multiple IMU coordinate systems based on the angular velocities of the multiple rigid components carrying the IMUs specifically includes: The first rotational transformation relationship between the first coordinate system and the second coordinate system is determined based on the angular velocity of the first rigid member and the angular velocity of the second rigid member, wherein the first coordinate system is the coordinate system of the first IMU and the second coordinate system is the coordinate system of the second IMU; Determining the opening / closing state of the first electronic device based on the first rotational transformation relationship between the plurality of IMU coordinate systems specifically includes: The opening and closing state of the first electronic device is determined based on the first rotational transformation relationship between the first coordinate system and the second coordinate system.

4. The method according to claim 3, characterized in that, Determining the opening / closing state of the first electronic device based on the first rotational transformation relationship between the first coordinate system and the second coordinate system specifically includes: The first rotation transformation relationship is compared with the standard rotation transformation relationship, which includes a first standard rotation transformation relationship and a second standard rotation transformation relationship. The first standard rotation transformation relationship is the rotation transformation relationship between the first coordinate system and the second coordinate system in the standard closed state, and the second standard rotation transformation relationship is the rotation transformation relationship between the first coordinate system and the second coordinate system in the standard unfolded state. If the deviation between the first rotational transformation relationship and the first standard rotational transformation relationship is less than the first threshold, then the opening / closing state of the first electronic device is determined to be the closed state. If the deviation between the rotational transformation relationship and the second standard rotational transformation relationship is less than the second threshold, then the opening / closing state of the first electronic device is determined to be the unfolded state.

5. The method according to claim 4, characterized in that, The opening / closing state of the first electronic device further includes one or more intermediate states. The standard rotational transformation relationship further includes a third standard rotational transformation relationship, which is the rotational transformation relationship between the first coordinate system and the second coordinate system in the standard intermediate state. Determining the opening / closing state of the first electronic device based on the first rotational transformation relationship between the first coordinate system and the second coordinate system further includes: If the deviation between the first rotational transformation relationship and the third standard rotational transformation relationship is less than the third threshold, then the opening / closing state of the first electronic device is determined to be the intermediate state.

6. The method according to any one of claims 3-5, characterized in that, The first rotational transformation relationship between the first coordinate system and the second coordinate system is represented by the first rotation axis and the first rotation angle.

7. The method according to claim 6, characterized in that, The first rotation axis and the first rotation angle are obtained by minimizing the cumulative deviation between the actual angular velocity of the second IMU and the estimated angular velocity of the second IMU, which is calculated based on the actual angular velocity of the first IMU.

8. The method according to claim 7, characterized in that, The estimated angular velocity of the second IMU is calculated based on the actual angular velocity of the first IMU, specifically including: The estimated angular velocity of the second IMU is calculated based on the following formula: Gyro (IMU2) ′=Q·Gyro (IMU1) ·Q -1 Among them, Gyro (IMU2) ′ represents the estimated angular velocity of the second IMU, Gyro (IMU1) Represents the actual angular velocity of the first IMU, a quaternion. θ1 represents the first rotation angle, and the first rotation axis is... i, j, and k are the imaginary units in the quaternion calculation formula.

9. The method according to claim 7 or 8, characterized in that, The cumulative deviation is calculated based on the following formula: Where δ represents the cumulative deviation, Gyro (IMU2) ′ represents the estimated angular velocity of the second IMU, Gyro (IMU1) Represents the actual angular velocity of the first IMU, a quaternion. θ1 represents the first rotation angle, and the first rotation axis is... i, j, and k are the imaginary units in the quaternion calculation formula.

10. The method according to any one of claims 2-9, characterized in that, Before determining the first rotational transformation relationship between the multiple IMU coordinate systems based on the angular velocities of the multiple rigid components carrying the IMUs, the method further includes: It was detected that all of the rigid components carrying the IMU were in motion.

11. The method according to any one of claims 1-10, characterized in that, The acquisition of motion sensing data of the multiple rigid components carrying the IMUs through the multiple IMUs specifically includes: The gravitational acceleration of the multiple rigid components carrying the IMUs is obtained through the multiple IMUs; Determining the rotation transformation relationship between the multiple IMU coordinate systems based on the motion sensing data specifically includes: The second rotational transformation relationship between the multiple IMU coordinate systems is determined based on the gravitational acceleration of the multiple rigid components carrying the IMU; Based on the rotational transformation relationship between the multiple IMU coordinate systems, the opening and closing state of the first electronic device is determined, specifically including: The opening / closing state of the first electronic device is determined based on the second rotational transformation relationship between the multiple IMU coordinate systems.

12. The method according to claim 11, characterized in that, The plurality of IMUs includes a first IMU and a second IMU, the first IMU being located on a first rigid member, and the second IMU being located on a second rigid member. The first rigid member and the second rigid member are contained within the plurality of rigid members. The step of obtaining the gravitational acceleration of the plurality of rigid members carrying the IMUs through the plurality of IMUs specifically includes: The gravitational acceleration of the first rigid component is obtained through the first IMU, and the gravitational acceleration of the second rigid component is obtained through the second IMU. The determination of the second rotational transformation relationship between the multiple IMU coordinate systems based on the gravitational acceleration of the multiple rigid components carrying the IMUs specifically includes: The second rotational transformation relationship between the first coordinate system and the second coordinate system is determined based on the gravitational acceleration of the first rigid component and the gravitational acceleration of the second rigid component. The first coordinate system is the coordinate system of the first IMU, and the second coordinate system is the coordinate system of the second IMU. Determining the opening / closing state of the first electronic device based on the second rotational transformation relationship between the plurality of IMU coordinate systems specifically includes: The opening and closing state of the first electronic device is determined based on the second rotational transformation relationship between the first coordinate system and the second coordinate system.

13. The method according to claim 12, characterized in that, Determining the opening / closing state of the first electronic device based on the second rotational transformation relationship between the first and second coordinate systems specifically includes: The second rotation transformation relationship is compared with the standard rotation transformation relationship, which includes a first standard rotation transformation relationship and a second standard rotation transformation relationship. The first standard rotation transformation relationship is the rotation transformation relationship between the first coordinate system and the second coordinate system in the standard closed state, and the second standard rotation transformation relationship is the rotation transformation relationship between the first coordinate system and the second coordinate system in the standard unfolded state. If the deviation between the second rotational transformation relationship and the first standard rotational transformation relationship is less than the first threshold, then the opening and closing state of the first electronic device is determined to be the closed state. If the deviation between the second rotational transformation relationship and the second standard rotational transformation relationship is less than the second threshold, then the opening / closing state of the first electronic device is determined to be the unfolded state.

14. The method according to claim 13, characterized in that, The opening / closing state of the first electronic device further includes one or more intermediate states. The standard rotational transformation relationship further includes a third standard rotational transformation relationship, which is the rotational transformation relationship between the first coordinate system and the second coordinate system in the standard intermediate state. Determining the opening / closing state of the first electronic device based on the second rotational transformation relationship between the first coordinate system and the second coordinate system further includes: If the deviation between the second rotational transformation relationship and the third standard rotational transformation relationship is less than the third threshold, then the opening / closing state of the first electronic device is determined to be the intermediate state.

15. The method according to any one of claims 12-14, characterized in that, The second rotational transformation relationship between the first coordinate system and the second coordinate system is represented by the second rotation axis and the second rotation angle.

16. The method according to claim 15, characterized in that, The second rotation axis and the second rotation angle are calculated based on the gravitational acceleration of the first IMU and the gravitational acceleration of the second IMU.

17. The method according to claim 15, characterized in that, The second rotation axis is calculated based on the following formula: u2=Acc IMU1 ×Acc IMU2 Where u2 represents the second rotation axis, Acc IMU1 Acc represents the gravitational acceleration of the first IMU. IMU2 This represents the gravitational acceleration of the second IMU.

18. The method according to claim 16 or 17, characterized in that, The second rotation angle is calculated based on the following formula: Where θ2 represents the second rotation angle, Acc IMU1 Acc represents the gravitational acceleration of the first IMU. IMU2 This represents the gravitational acceleration of the second IMU.

19. The method according to any one of claims 11-18, characterized in that, Before determining the second rotational transformation relationship between the multiple IMU coordinate systems based on the gravitational acceleration of the multiple rigid components carrying the IMU, the method further includes: It was detected that all of the rigid components carrying the IMU were in a stationary state.

20. The method according to any one of claims 1-19, characterized in that, The method further includes: If the first electronic device is in the unfolded state, it will automatically power on or de-standby. If the first electronic device is in the closed state, it will automatically shut down or enter standby mode.

21. An electronic device, characterized in that, The method includes one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when executed by the one or more processors, cause the method of any one of claims 1-20 to be performed.

22. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run on an electronic device, it causes the method described in any one of claims 1-20 to be performed.

23. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-20.

Images