Method, device and equipment for expanding screen and storage medium

Abstract

The application provides a control method of an extended screen, a terminal device, a wearable device and a computer readable storage medium, and relates to the technical field of intelligent terminals. The method comprises the following steps: acquiring control information corresponding to the extended screen according to the motion type of the terminal device; wherein the motion type of the terminal device is determined according to motion data collected by the terminal device; and controlling the extended screen according to the control information. The embodiments of the application aim to realize the control of the extended screen of the terminal device and improve the convenience and applicability of screen control.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent terminals, in particular to a control method of an extended screen, a control device of an extended screen, a terminal device, a wearable device and a computer readable storage medium. BACKGROUND

[0002] With the continuous development of terminal technology, people's daily life is brought more and more convenience, and the user's requirements for the screen of the terminal device are also gradually improved. Among them, the terminal device not only includes the existing display screen, but also the extended screen of the terminal device has become the development trend of the terminal device.

[0003] However, in the related art, for the display screen of the terminal device, the user needs to control the display of the screen by the touch mode. And for the extended screen of the terminal device, since the extended screen is a virtual screen, the user cannot control the display of the screen by the touch mode. Therefore, for the extended screen of the terminal device, the related art does not propose a corresponding method to control the extended screen. SUMMARY

[0004] The present application provides a control method of an extended screen, a control device of an extended screen, a terminal device, a wearable device and a computer readable storage medium, which aims to be able to control the extended screen of the terminal device and improve the convenience and applicability of screen control.

[0005] To achieve the above purpose, the present application provides a control method of an extended screen applied to a wearable device, which comprises:

[0006] According to the motion type of the terminal device, the control information corresponding to the extended screen is obtained; wherein the motion type of the terminal device is determined according to the motion data collected by the terminal device;

[0007] According to the control information, the extended screen is controlled.

[0008] In addition, to achieve the above purpose, the present application also provides a control method of an extended screen applied to a terminal device, which comprises:

[0009] The motion data collected by the motion collection device in the terminal device is obtained;

[0010] According to the motion data, the motion type corresponding to the terminal device is determined;

[0011] The motion type is sent to the wearable device, so that the wearable device determines the control information corresponding to the extended screen according to the motion type, and controls the extended screen according to the control information.

[0012] Further, to achieve the above object, the present application provides an extended screen control device applied to a wearable device, the extended screen control device comprising:

[0013] a motion type determination module configured to acquire control information corresponding to the extended screen according to a motion type of the terminal device, wherein the motion type of the terminal device is determined according to motion data collected by the terminal device;

[0014] an extended screen control module configured to control the extended screen according to the control information.

[0015] Further, to achieve the above object, the present application provides a terminal device, an extended screen of which is provided by a wearable device, the terminal device comprising a memory and a processor, wherein the memory is connected to the processor and is configured to store programs, and the processor is configured to realize any one of the control methods of the extended screen of the terminal device provided by the embodiments of the present application by running the programs stored in the memory.

[0016] Further, to achieve the above object, the present application provides a wearable device, which is configured to provide an extended screen for a terminal device, the wearable device comprising a memory and a processor, wherein the memory is connected to the processor and is configured to store programs, and the processor is configured to realize the control method of the extended screen of the wearable device provided by the embodiments of the present application by running the programs stored in the memory.

[0017] Further, to achieve the above object, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is configured to enable a processor to realize any one of the control methods of the extended screen of the terminal device or the control method of the extended screen of the wearable device provided by the embodiments of the present application when the processor executes the computer program.

[0018] The embodiments of the present application disclose a control method of an extended screen, a terminal device, a wearable device and a computer readable storage medium. The motion type of the terminal device is determined according to motion data collected by a motion collection device in the terminal device, the control information corresponding to the extended screen is acquired according to the motion type of the terminal device, and the extended screen is controlled according to the control information. Therefore, the control of the extended screen of the terminal device can be realized, an intuitive and simple interaction mode is provided for AR extended display of the terminal device, and the convenience of using the AR extended display of the terminal device is improved. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings.

[0020] Figure 1 is a flowchart of a control method of an extended screen applied to a terminal device provided by an embodiment of the present application;

[0021] Figure 2 is a structural diagram of a terminal device provided by an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of an application scenario of a control method of an extended screen provided by an embodiment of the present application;

[0023] Figure 4 is a flowchart of a control method of an extended screen applied to a wearable device provided by an embodiment of the present application;

[0024] Figure 5 is a flowchart of another control method of an extended screen applied to a wearable device provided by an embodiment of the present application;

[0025] Figure 6 is a flowchart of another control method of an extended screen applied to a wearable device provided by an embodiment of the present application;

[0026] Figure 7 is a schematic block diagram of a control device of an extended screen provided by an embodiment of the present application;

[0027] Figure 8 is a schematic block diagram of a wearable device provided by an embodiment of the present application;

[0028] Figure 9 is a schematic block diagram of a terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0030] The flowcharts shown in the drawings are only illustrative examples, and do not necessarily include all contents and operations / steps, nor are they necessarily executed in the order described. For example, some operations / steps can also be decomposed, combined, or partially merged, so the actual execution order can be changed according to actual conditions. In addition, although the functional modules are divided in the device schematic diagram, in some cases, the module division can be different from that in the device schematic diagram.

[0031] The term "and / or" used in the present application and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0032] Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The following examples and features in the examples can be combined with each other without conflict.

[0033] To facilitate understanding of the embodiments of the present application, some background related to the embodiments of the present application will be briefly described below.

[0034] Currently, the size of terminal devices such as smart phones has evolved to the limit of one-handed convenient operation, and the development of straight phones has entered a bottleneck, and folding screen phones have become a potential demand, which is a new direction for the development of the industry.

[0035] Taking a terminal device as a mobile phone as an example, the expansion of the physical screen display area is positively related to the cost. The larger the mobile phone screen display area, the higher the cost. In the case that the folding mobile phone technology cannot achieve a leap-forward development, the larger the display area required by the folding mobile phone screen, the larger the size of the folded mobile phone will be, which cannot bring good user experience. The area of the mobile phone screen display cannot exceed the display outside the screen, which will make the content products based on the mobile phone unable to realize more information expansion and interaction expansion.

[0036] Please refer to Figure 1 , Figure 1 is a flowchart of a screen expansion control method provided by an embodiment of the present application. In the embodiment, the screen expansion control method can be applied to a terminal device or a wearable device.

[0037] The terminal device can be a camera, a mobile phone, a computer, a robot, or any other terminal device that can perform a screen display function, which is not specifically limited herein. The wearable device can be smart glasses or an AR helmet, or any other device that can be used to display AR expansion screen display, which is not specifically limited herein. The wearable device can be AR (Augmented Reality) glasses, VR (Virtual Reality) glasses, or any other glasses that can be used to display AR expansion screen display, which is not specifically limited herein.

[0038] For example, the extended screen can be a virtual extended screen projected around the terminal device by the wearable device. When the system or the third-party application in the terminal device needs to expand more display range outside the screen of the terminal device, the system / application of the terminal device sends an instruction to start the AR extended screen to the client of the wearable device, and sends the content displayed on the AR extended screen to the client of the wearable device. The client of the wearable device can display the content on the specified user interface after receiving the content. The client of the wearable device can also display its own content on the specified user interface. It can be understood that the extended screen can display the content of the terminal device or the content of the wearable device. The user interface of the AR extended screen is anchored with the pose of the terminal device after the wearable device recognizes and tracks the position of the terminal device, and becomes an extended display outside the physical screen of the terminal device.

[0039] For example, the control method of the extended screen is applied to the wearable device, and the wearable device is taken as an example of smart glasses to describe the control method of the extended screen provided by the embodiments of the present application. The extended screen of the terminal device is provided by the wearable device.

[0040] As shown in the application, the control method of the extended screen of the wearable device includes steps S101 to S102. Figure 1

[0041] S101, according to the motion type of the terminal device, obtaining the control information corresponding to the extended screen of the terminal device; wherein the motion type of the terminal device is determined according to the motion data collected by the terminal device.

[0042] The motion acquisition device can be a data acquisition device for collecting motion data of the terminal device, such as a gyroscope or an accelerometer. The motion data can include angular velocity and acceleration data. The motion type can include left / right point motion (around z-axis), left / right point motion (around y-axis), up / down point motion (around x-axis), and tapping. Specifically, point motion and tapping can be defined as single, double or multiple. The control information can be used to control the screen content of the extended screen, such as controlling the icon pre-selection cursor of the extended screen to jump to the icon on the left side of the current icon; it can also be used to select icons, open icons, close icons or switch icons, etc. The icon can be a virtual program of the extended screen in the terminal device. The number of the extended screen of the terminal device can be set according to specific needs, such as the number of the extended screen can be one or more, and the size of each extended screen can be the same or different.

[0043] ​It should be noted that the steps of "determining the motion type of the terminal device according to the motion data" and "determining the control information corresponding to the extended screen of the terminal device according to the motion type" can be implemented on the terminal device or on the wearable device, which is not limited here.

[0044] S102, controlling the extended screen according to the control information.

[0045] Specifically, the wearable device such as smart glasses can control the extended screen to display the corresponding screen content according to the control information.

[0046] For example, if the control information is used to control the extended screen to perform a selection action, the smart glasses can display the screen content corresponding to the selection action on the extended screen according to the control information. For example, when the content displayed on the extended screen is a photo album, the selection action can be selecting one photo.

[0047] The method for controlling the extended screen provided by the embodiment of the present application can determine the motion type of the terminal device according to the motion data collected by the motion collection device in the terminal device, obtain the control information corresponding to the extended screen according to the motion type of the terminal device, and control the extended screen according to the control information. Thus, the control of the extended screen of the terminal device can be realized, an intuitive and simple interaction mode for the AR extended display of the terminal device is provided, and the convenience of using the AR extended display of the terminal device is improved.

[0048] Hereinafter, the wearable device is taken as smart glasses, and the terminal device is taken as a mobile phone as an example.

[0049] In the embodiment of the present application, when the user manipulates the terminal device, the terminal device can detect the change of the device state, and the corresponding motion data when the terminal device is in motion can be collected by the motion collection device of the terminal device, so that the corresponding motion data can be accurately collected.

[0050] For example, when the user manipulates the terminal device to move left / right (around the z-axis), the corresponding angular velocity data when the terminal device is in motion can be collected by the gyroscope of the terminal device, so that the corresponding angular velocity data can be accurately collected, that is, the change of the device state of the terminal device can be detected by the angular velocity data. The device state can include the terminal device state reflected by the events such as the point movement or the tapping of the terminal device.

[0051] For example, when the user taps the terminal device, the corresponding acceleration data when the terminal device is in motion can be collected by the accelerometer of the terminal device, so that the corresponding acceleration data can be accurately collected, and the change of the device state of the terminal device can be determined by the acceleration data.

[0052] For example, when the user taps the terminal device, the corresponding acceleration data when the terminal device is in motion can be collected by the accelerometer of the terminal device, so that the corresponding acceleration data can be accurately collected, and the change of the device state of the terminal device can be determined by the acceleration data. Figure 2As shown, exemplary, taking a mobile phone as an example of the terminal device, a user can control the mobile phone to jog left / right (around the z-axis), control the mobile phone to jog left / right (around the y-axis), control the mobile phone to jog up / down (around the x-axis), or knock the mobile phone. Wherein, the above user operations all correspond to a motion type.

[0053] In some embodiments, a data type is determined according to the motion data; and the motion data is calculated according to the data type to obtain the motion type corresponding to the terminal device. Embodiments of the present application can accurately determine the data type, and can accurately calculate the motion type corresponding to the terminal device according to the data type.

[0054] Wherein, the data type can include angular velocity data and acceleration data, etc.

[0055] Exemplary, the data type can be determined by detecting the type of the motion acquisition device, such as if the motion data is obtained from a gyroscope, then the data type of the motion data can be determined as angular velocity data; if the motion data is obtained from an accelerometer, then the data type of the motion data can be determined as acceleration data.

[0056] Exemplary, the data type can also be determined by detecting the numerical characteristics or change curve of the data, etc., which are not limited here.

[0057] In some embodiments, if the data type is angular velocity data, then the rotation angle of the terminal device is determined through the angular velocity data, and the motion type corresponding to the terminal device is determined according to the rotation angle; if the data type is acceleration data, then the acceleration slope corresponding to the acceleration data is determined, and the motion type corresponding to the terminal device is determined according to the acceleration slope. Embodiments of the present application can calculate the motion type corresponding to the terminal device by using corresponding algorithms according to different data types, so as to accurately calculate the motion type corresponding to the terminal device.

[0058] Wherein, the rotation angle of the terminal device is the angle of the terminal device rotating in a direction in a time period; and the acceleration slope can be the slope between two continuous acceleration samples.

[0059] Exemplary, if the data type is angular velocity data, then the rotation angles of the terminal device in the rotation phase and the return phase are calculated through the angular velocity data, and the jog type corresponding to the terminal device is determined according to the rotation angles in the rotation phase and the return phase.

[0060] Wherein, the rotation phase is the process of the terminal device rotating in a first direction; and the return phase is the process of the terminal device rotating in the opposite direction of the first direction. The first direction can be any direction, such as left, right, up, and down, etc.

[0061] For example, if the data type is acceleration data, the acceleration slopes corresponding to two continuous acceleration data are determined, and the type of the knock corresponding to the terminal device is determined according to the acceleration slopes.

[0062] It should be noted that the angular velocity data and the acceleration data can be simultaneously collected by the motion collection device in the terminal device, and the angular velocity data and the acceleration data can be simultaneously calculated, which is not limited herein.

[0063] Therefore, the present application can determine the data type including the angular velocity data and / or the acceleration data by analyzing the motion data, and the corresponding algorithm is proposed to calculate the angular velocity data and the acceleration data according to the angular velocity data and the acceleration data, so as to determine the motion type corresponding to the terminal device.

[0064] The following wearable device is smart glasses, the terminal device is a mobile phone, and the motion data collected by the motion collection device is angular velocity data.

[0065] In some embodiments, the angular velocity data corresponding to the first time period is obtained, the angular velocity data including an angular velocity component and an angular velocity modulus; if the angular velocity component and / or the angular velocity modulus corresponding to the first time period exceeds the respective corresponding preset angular velocity threshold, the angular velocity component and / or the angular velocity modulus corresponding to the first time period is processed to obtain a first rotation angle; if the first rotation angle is within a preset angle range, the angular velocity component corresponding to the second time period is obtained, and the angular velocity component corresponding to the second time period is processed to obtain a second rotation angle; and the motion type corresponding to the terminal device is determined according to the first rotation angle and the second rotation angle. The embodiments of the present application can accurately determine the first rotation angle and the second rotation angle, so as to accurately determine the motion type corresponding to the terminal device according to the first rotation angle and the second rotation angle.

[0066] The angular velocity data includes an angular velocity component and an angular velocity modulus, the angular velocity component can include three-axis angular velocity components, i.e., the angular velocity components corresponding to the x-axis, the y-axis and the z-axis; and the angular velocity modulus can be an absolute value calculated by the three-axis angular velocity components. The preset angular velocity threshold can be any angular velocity value, which is set by a user and is not limited herein. The first time period is a time period corresponding to the rotation stage of the terminal device, and the second time period is a time period corresponding to the return stage of the terminal device. The first rotation angle can be an angle rotated by the terminal device in the first time period, the second rotation angle can be an angle rotated by the terminal device in the second time period, and the preset angle range can be any angle, which is set by a user and is not limited herein.

[0067] It should be noted that each angular velocity component and the angular velocity module value has a corresponding preset angular velocity threshold value, and any angular velocity component or angular velocity module value exceeding the corresponding preset angular velocity threshold value can be processed to obtain the first rotation angle.

[0068] For example, if the angular velocity component corresponding to the x-axis exceeds the corresponding preset angular velocity threshold value, the angular velocity component corresponding to the x-axis can be determined to be processed to obtain the first rotation angle.

[0069] In the embodiment of the present application, the three-axis angular velocity components (ω x , ω y , ω z ) obtained by the gyroscope can be used to calculate the corresponding angular velocity module value.

[0070] For example, the formula for calculating the angular velocity module value can be:

[0071]

[0072] where |ω| is the angular velocity module value, ω x is the angular velocity component corresponding to the x-axis, ω y is the angular velocity component corresponding to the y-axis, and ω z is the angular velocity component corresponding to the z-axis.

[0073] If the angular velocity component and / or the angular velocity module value corresponding to the first time period exceeds the corresponding preset angular velocity threshold value, the specific motion axis of the gyroscope is identified, and the angular velocity component corresponding to the motion axis in the first time period is subjected to Euler integration to obtain the first rotation angle. If the angular velocity component and the angular velocity module value corresponding to the first time period do not exceed the preset angular velocity threshold value, it is determined that the terminal device does not have a rotation phase.

[0074] where the motion axis can be the axis of the terminal device detected by the gyroscope.

[0075] For example, if the preset angular velocity threshold value is 5° / s, if ω x is 10°, ω y is 0°, and ω z is 0°, the angular velocity module value is also 10°, it can be determined that the angular velocity component and the angular velocity module value corresponding to the first time period exceed the preset angular velocity threshold value, the specific motion axis of the gyroscope is identified as the x-axis, and the angular velocity component corresponding to the x-axis in the first time period is subjected to Euler integration to obtain the first rotation angle.

[0076] It should be noted that if the specific motion axis of the gyroscope is multiple axes, such as the x-axis and the y-axis, Euler integration needs to be performed on the angular velocity components corresponding to the x-axis and the y-axis in the first time period respectively to obtain the first rotation angles corresponding to the x-axis and the y-axis.

[0077] Specifically, the formula for calculating the rotation angle can be represented as:

[0078]

[0079] wherein θ is the rotation angle in the time period, which can be the first rotation angle or the second rotation angle; t1 is the start time of rotation around the calculated axis, and t2 is the end time of rotation around the calculated axis; ω(t) is the angular velocity component corresponding to the motion axis at time t.

[0080] If the first rotation angle is within the preset angle range, the angular velocity component corresponding to the second time period is obtained, and Euler integration is performed on the angular velocity component corresponding to the second time period to obtain the second rotation angle. If the first rotation angle is not within the preset angle range, it can be determined that the terminal device does not have a return stage.

[0081] For example, if the preset angle range is [15°, 45°] and the first rotation angle is 30°, it can be determined that the first rotation angle is within the preset angle range, the angular velocity component corresponding to the second time period is obtained, and Euler integration is performed on the angular velocity component corresponding to the second time period to obtain the second rotation angle.

[0082] In some embodiments, after obtaining the first rotation angle, the first angular velocity component and the second angular velocity component in the first time period are obtained; the angular velocity slope is determined according to the first angular velocity component and the corresponding timestamp information and the second angular velocity component and the corresponding timestamp information; if the first rotation angle is within the preset angle range and the angular velocity slope is greater than the preset angular velocity slope, the angular velocity component corresponding to the second time period is obtained, and the angular velocity component corresponding to the second time period is processed to obtain the second rotation angle. The embodiments of the present application can accurately calculate the second rotation angle through the angular velocity component.

[0083] wherein the first angular velocity component and the second angular velocity component can be two consecutive angular velocity components in the first time period; the timestamp information is used to record the timestamps corresponding to the first angular velocity component and the second angular velocity component; and the angular velocity slope can be the slope between two consecutive angular velocity samples. The preset angular velocity slope can be any angular velocity slope, which can be set according to actual conditions and is not limited herein.

[0084] Specifically, according to the first angular velocity component and corresponding timestamp information and the second angular velocity component and corresponding timestamp information, an angular velocity slope is determined; if the first rotation angle is within a preset angle range and the angular velocity slope is greater than a preset angular velocity slope, an angular velocity component corresponding to a second time period is obtained, and the angular velocity component corresponding to the second time period is processed to obtain a second rotation angle; if the first rotation angle is not within the preset angle range or the angular velocity slope is not greater than the preset angular velocity slope, it can be determined that the terminal device does not have a return stage.

[0085] For example, the formula for calculating the angular velocity slope can be expressed as:

[0086]

[0087] Wherein, slopeω(t) is the angular velocity slope, ω3 is the first angular velocity component, ω4 is the second angular velocity component, t3 is the timestamp information corresponding to the first angular velocity component, and t4 is the timestamp information corresponding to the second angular velocity component.

[0088] In some embodiments, a difference between the first rotation angle and the second rotation angle is determined; if the difference between the first rotation angle and the second rotation angle is less than a preset angle threshold, it is determined that the motion type corresponding to the terminal device is point motion. The embodiments of the application can accurately determine the motion type corresponding to the terminal device by comparing the first rotation angle and the second rotation angle.

[0089] Wherein, the preset angle threshold can be any angle, which is determined by actual conditions and is not limited here.

[0090] It should be noted that after determining that the motion type corresponding to the terminal device is point motion, the specific point motion type can be determined by the motion axis and the motion direction read by the gyroscope, such as the motion axis read by the gyroscope being the x-axis and the motion direction being the clockwise direction (such as detecting a positive acceleration value), that is, it can be considered that the motion type corresponding to the terminal device is point motion around the x-axis to the right. The motion axis can also be determined by the angular velocity component and the corresponding preset angular velocity threshold, if the angular velocity component corresponding to the x-axis exceeds the corresponding preset angular velocity threshold, it can be considered that the motion axis includes the x-axis.

[0091] Specifically, a difference between the first rotation angle and the second rotation angle is determined; if the difference between the first rotation angle and the second rotation angle is less than a preset angle threshold, it is determined that the motion type corresponding to the terminal device is point motion; if the difference between the first rotation angle and the second rotation angle is greater than or equal to the preset angle threshold, it is determined that the motion type corresponding to the terminal device is not point motion.

[0092] For example, if the first rotation angle is 30°, the second rotation angle is 20°, and the preset angle threshold is 20°, it can be determined that the difference between the first rotation angle and the second rotation angle is 10°, and it is determined that the motion type corresponding to the terminal device is point motion, because the difference between the first rotation angle and the second rotation angle is less than the preset angle threshold.

[0093] The above is an example of using the motion data collected by the motion collection device as angular velocity data to explain how to determine the motion type corresponding to the terminal device.

[0094] The following wearable device is a smart glasses, and the terminal device is a mobile phone. The motion data collected by the motion collection device is acceleration data.

[0095] In some embodiments, acceleration data corresponding to a plurality of time points is obtained, the acceleration data including acceleration components and acceleration module values; if the acceleration component and / or the acceleration module value corresponding to a first time point exceeds a respective preset acceleration threshold, the acceleration component and / or the acceleration module value corresponding to the first time point and its adjacent time point is processed to obtain a first acceleration slope; if the first acceleration slope exceeds a preset slope threshold, it is determined that the motion type corresponding to the terminal device is single tapping. The embodiments of the present application can accurately determine the motion type corresponding to the terminal device as single tapping through the acceleration data.

[0096] The acceleration data includes acceleration components and acceleration module values. The acceleration components can include three-axis acceleration components, i.e., the acceleration components corresponding to the x-axis, y-axis, and z-axis. The acceleration module value can be an absolute value calculated from the three-axis acceleration components. The preset acceleration threshold can be any angular velocity value, which is specifically set by the user and is not specifically limited herein. The first time point and its adjacent time point can be two consecutive time points in the plurality of time points. The first acceleration slope is the acceleration slope corresponding to the first time point and its adjacent time point. The preset slope threshold can be any acceleration slope value, which is specifically set by the user and is not specifically limited herein.

[0097] It should be noted that each acceleration component and acceleration module value has a corresponding preset acceleration threshold. If any acceleration component or acceleration module value in the first time point exceeds its corresponding preset acceleration threshold, the acceleration component and / or the acceleration module value corresponding to the first time point and its adjacent time point can be processed to obtain the first acceleration slope.

[0098] For example, if the acceleration component corresponding to the x-axis in the first time point exceeds its corresponding preset acceleration threshold, the acceleration component corresponding to the x-axis in the first time point and its adjacent time point (the second time point) can be processed to obtain the first acceleration slope.

[0099] In the embodiments of the present application, the three-axis acceleration components obtained by the accelerometer are (a x , a y , a z , and the acceleration modulus corresponding to the three-axis acceleration components can be calculated.

[0100] For example, the formula for calculating the acceleration modulus can be:

[0101]

[0102] wherein |a| is the acceleration modulus, a x is the acceleration component corresponding to the x-axis, a y is the acceleration component corresponding to the y-axis, and a z is the acceleration component corresponding to the z-axis.

[0103] If the acceleration component and / or the acceleration modulus corresponding to the first time exceeds the preset acceleration threshold, the acceleration components corresponding to the first time and its adjacent time are processed to obtain the first acceleration slope; if the acceleration component and the acceleration modulus corresponding to the first time do not exceed the preset acceleration threshold, it is determined that the terminal device does not detect the knock.

[0104] For example, if the preset acceleration threshold is 1 m / s 2 , if ω x is 2 m / s 2 , ω y is 0 m / s 2 , ω z is 0 m / s 2 , and the angular velocity modulus is also 2 m / s 2 , it can be determined that the angular velocity component and the angular velocity modulus corresponding to the first time period exceed the preset angular velocity threshold, and the slope between two continuous acceleration samples is calculated to obtain the first acceleration slope.

[0105] It should be noted that if the accelerometer detects that the acceleration components corresponding to multiple axes exceed the preset acceleration threshold, such as the x-axis and the y-axis, the acceleration components corresponding to the x-axis and the y-axis at the first time are respectively calculated to obtain the first acceleration slopes corresponding to the x-axis and the y-axis.

[0106] Specifically, the formula for calculating the acceleration slope can be represented as:

[0107]

[0108] wherein slope(t n ) is the slope between two continuous acceleration samples; t n and t (n-1)respectively are the time of two continuous acceleration samples; a(t n ) is the acceleration of the calculated axis at time t n .

[0109] For example, it is determined whether the first acceleration slope exceeds a preset slope threshold value. If the first acceleration slope exceeds the preset slope threshold value, it can be determined that the motion type corresponding to the terminal device is single tapping. If the first acceleration slope does not exceed the preset slope threshold value, it can be determined that the terminal device does not detect tapping.

[0110] In some embodiments, after determining that the motion type corresponding to the terminal device is single tapping, it is determined whether the acceleration component and the acceleration modulus value corresponding to the second time exceed the respective preset acceleration threshold values. If the acceleration component and / or the acceleration modulus value corresponding to the second time exceeds the preset acceleration threshold value, the acceleration component and / or the acceleration modulus value corresponding to the second time and its adjacent time is processed to obtain a second acceleration slope. If the second acceleration slope exceeds the preset slope threshold value, the difference between the second time and the first time is determined. If the difference between the second time and the first time is less than a preset time threshold value, it is determined that the motion type corresponding to the terminal device is multiple tapping. The embodiments of the present application can accurately determine whether the motion type corresponding to the terminal device is multiple tapping, thereby accurately determining the motion type corresponding to the terminal device.

[0111] The second time and its adjacent time can be two continuous times in a plurality of times, and the time of the second time is later than the time of the second time. The second acceleration slope is the acceleration slope corresponding to the second time and its adjacent time. The preset time threshold value can be any time value, such as 5s, which is not limited here.

[0112] Specifically, if it is determined that the second acceleration slope exceeds the preset slope threshold value, the difference between the second time and the first time is determined. If the difference between the second time and the first time is less than a preset time threshold value, it is determined that the motion type corresponding to the terminal device is multiple tapping. If the difference between the second time and the first time is not less than the preset time threshold value, it is determined that the motion type corresponding to the terminal device is single tapping.

[0113] For example, if the first time is 15:10:05, the second time is 15:10:12, and the preset time threshold value is 10s, then the difference between the second time and the first time is 7s, and it can be determined that the difference between the second time and the first time is less than the preset time threshold value, thereby determining that the motion type corresponding to the terminal device is multiple tapping.

[0114] The above is an example of determining the motion type corresponding to the terminal device by taking the motion data collected by the motion collection device as acceleration data.

[0115] It should be noted that the above embodiments are all for how to determine the motion type corresponding to the terminal device, and the above embodiments can be executed alone or simultaneously.

[0116] In some embodiments, a mapping relationship between preset motion types and control information is acquired, and it is determined whether the motion type of the terminal device is the preset motion type; if the motion type of the terminal device is the preset motion type, the control information corresponding to the motion type of the terminal device is determined according to the mapping relationship. The embodiments of the present application can accurately determine the corresponding control information according to the motion type, so as to display the corresponding screen content on the extended screen.

[0117] The mapping relationship between the preset motion types and the control information is used to represent the corresponding relationship between each preset control information and each screen content. The preset motion types can include left-pointing around the x-axis, left-pointing around the y-axis, tapping, etc. The control information can be used to control the screen content of the extended screen. The icon can be a virtual icon of the extended screen in the terminal device.

[0118] For example, if it is determined that the motion type of the terminal device is left-pointing around the x-axis, it is determined whether the motion type of the terminal device is the preset motion type. If it is determined that the motion type of the terminal device is the preset motion type, the control information corresponding to the left-pointing around the x-axis is determined according to the mapping relationship between the preset motion types and the control information, such as the control information for controlling the extended screen to perform a selection action.

[0119] In some embodiments, the control information at least includes one of the following: controlling the extended screen to execute an opening instruction; or, controlling the extended screen to execute a selection instruction; or, controlling the extended screen to execute a closing instruction; or, controlling the extended screen to execute a switching instruction; or, controlling the extended screen to execute an enlargement instruction; or, controlling the extended screen to execute a reduction instruction.

[0120] The control information corresponds to a preset motion type. For example, the instruction for controlling the extended screen to open the corresponding icon corresponds to a left click around the x-axis, and the instruction for controlling the extended screen to close the corresponding icon corresponds to a single tap, and the like. The mapping relationship between the corresponding preset motion type and the control information can be set according to actual conditions, and is not specifically limited herein.

[0121] For example, the control information corresponds to a preset motion type. For example, the instruction for controlling the extended screen to open the corresponding icon corresponds to a left click around the x-axis, and the instruction for controlling the extended screen to close the corresponding icon corresponds to a single tap, and the like. The mapping relationship between the corresponding preset motion type and the control information can be set according to actual conditions, and is not specifically limited herein.

[0122] As shown in FIG. 6, the screen content of the extended screen can include a selection action, a cancellation action, a cursor movement action, and the like. Figure 3 The extended screen can be displayed on the left and right sides and the top space of the terminal device.

[0123] For example, the up / down click (around the x-axis) and the left / right click (around the z-axis) of the terminal device can be associated with the up / down cursor movement and the left / right cursor movement, respectively. When the left click is detected, the cursor is moved to the left of the icon at the current position of the cursor, and the screen content position does not change. For example, the single tap and the double tap are associated with the selection and the cancellation, respectively. When a single tap is detected, the content of the icon at the cursor position is selected.

[0124] For example, when the terminal device is a mobile phone, the action of holding the mobile phone and the action of extending the screen content can be defined by the user in advance and the corresponding mapping relationship can be stored in the wearable device or the terminal device, such as gently shaking the mobile phone screen (equivalent to continuous left and right point motion around the x-axis) associated with multi-component selection action, gently lifting the top of the mobile phone (equivalent to left point motion around the z-axis) associated with confirmation action and opening action, gently lifting the left side of the mobile phone (equivalent to left point motion around the x-axis) associated with return action and closing action, gently lifting the right side of the mobile phone (equivalent to right point motion around the x-axis) associated with switching action, and the like.

[0125] For example, when the terminal device is a mobile phone, if the extended screen displays the corresponding screen content, when the angle of inclination of the extended screen changes with the gravity sensing of the mobile phone (i.e. the amplitude of the point motion type), the extended screen can also present effects such as size magnification and transparent gradient; or after selecting the confirmation operation of the extended screen, the extended screen can also present the effect of component magnification, thereby displaying more detailed content; or when closing and exiting the current component, the extended screen can also present the effect of component reduction and return to the original extended display area; or when switching the extended screen, the extended screen can also present effects such as gradually increasing or decreasing dynamically. The mapping relationship between the inclination angle of the mobile phone and the control information can be set to achieve the above effects.

[0126] In some embodiments, the control method of the extended screen is applied to the terminal device as an example to describe the control method of the extended screen provided by the embodiments of the present application. The extended screen of the terminal device is provided by the wearable device.

[0127] As shown in Figure 4 The control method of the extended screen applied to the terminal device includes steps S201 to S203.

[0128] S201, acquiring motion data collected by a motion acquisition device in the terminal device;

[0129] S202, determining a motion type corresponding to the terminal device according to the motion data;

[0130] S203, sending the motion type to the wearable device, so that the wearable device determines control information corresponding to the extended screen according to the motion type, and controls the extended screen according to the control information.

[0131] It can be understood that "determining the motion type corresponding to the terminal device according to the motion data" and "determining the control information corresponding to the extended screen of the terminal device according to the motion type" can be implemented on the terminal device or on the wearable device. The specific embodiments corresponding to the above steps have been described in the foregoing, and will not be described repeatedly here.

[0132] The method for controlling the extended screen provided by the embodiments of the present application comprises the following steps: acquiring motion data collected by a motion collection device in a terminal device; determining a motion type corresponding to the terminal device according to the motion data; sending the motion type to a wearable device, so that the wearable device determines control information corresponding to the extended screen according to the motion type, and controls the extended screen according to the control information. Thus, the step of determining the motion type corresponding to the terminal device according to the motion data can be implemented on the terminal device, and the step of determining the control information corresponding to the extended screen of the terminal device according to the motion type can be implemented on the wearable device, so as to balance the operation data amount of the terminal device and the wearable device, thereby realizing the control of the extended screen of the terminal device, providing an intuitive and simple interaction mode for the AR extended display of the terminal device, and improving the convenience of using the AR extended display of the terminal device.

[0133] The following will show the method for controlling the extended screen in various application scenarios.

[0134] Please refer to Figure 5 , Figure 5 FIG. 2 is a flowchart of another method for controlling the extended screen provided by the embodiments of the present application.

[0135] As shown in Figure 5 , the method for controlling the extended screen applied to the terminal device comprises steps S301 to S304.

[0136] S301, acquiring motion data collected by a motion collection device in the terminal device.

[0137] S302, determining a motion type corresponding to the terminal device according to the motion data.

[0138] S303, determining control information corresponding to the extended screen according to the motion type.

[0139] S304, sending the control information to the wearable device, so that the wearable device controls the extended screen according to the control information.

[0140] The method for controlling the extended screen provided in the embodiments of the present application comprises the following steps: acquiring motion data collected by a motion collection device in a terminal device; determining a motion type corresponding to the terminal device according to the motion data; sending the motion type to a wearable device, so that the wearable device determines control information corresponding to the extended screen according to the motion type, and controls the extended screen according to the control information. Thus, the steps of "determining a motion type corresponding to the terminal device according to the motion data" and "determining control information corresponding to the extended screen of the terminal device according to the motion type" can be implemented on the terminal device, so as to reduce the operation data amount of the wearable device and the operation burden of the wearable device, thereby realizing the control of the extended screen of the terminal device, providing an intuitive and simple interaction mode for the AR extended display of the terminal device, and improving the convenience of using the AR extended display of the terminal device.

[0141] Please refer to Figure 6 , Figure 6 FIG. 3 is a flowchart of another method for controlling an extended screen provided in the embodiments of the present application.

[0142] As shown in Figure 6 , the method for controlling the extended screen applied to the terminal device comprises steps S401 to S402.

[0143] S401, acquiring motion data collected by a motion collection device in the terminal device.

[0144] S402, sending the motion data to a wearable device, so that the wearable device determines a motion type corresponding to the terminal device according to the motion data, and determines control information corresponding to the extended screen according to the motion type, and controls the extended screen according to the control information.

[0145] It should be noted that the specific embodiments corresponding to the steps of "determining a motion type corresponding to the terminal device according to the motion data" and "determining control information corresponding to the extended screen of the terminal device according to the motion type" have been described in the foregoing, and will not be described herein.

[0146] The method provided by the embodiment of the application comprises the following steps: acquiring motion data collected by a motion acquisition device in a terminal device; determining a motion type corresponding to the terminal device according to the motion data; sending the motion type to a wearable device, so that the wearable device determines control information corresponding to an extended screen according to the motion type, and controls the extended screen according to the control information. Thus, the steps of determining the motion type corresponding to the terminal device according to the motion data and determining the control information corresponding to the extended screen of the terminal device according to the motion type can be implemented on the wearable device, so as to reduce the operation data amount of the terminal device and the operation burden of the terminal device, thereby realizing the control of the extended screen of the terminal device, providing an intuitive and simple interaction mode for the AR extended display of the terminal device, and improving the convenience of using the AR extended display of the terminal device.

[0147] Please refer to Figure 7 , Figure 7 is a schematic block diagram of an extended screen control device applied to a wearable device. As shown in Figure 7 , the extended screen control device 500 comprises a type determination module 501 and a screen control module 502.

[0148] The type determination module 501 is configured to acquire control information corresponding to the extended screen according to a motion type of the terminal device, wherein the motion type of the terminal device is determined according to motion data collected by the terminal device.

[0149] The screen control module 502 is configured to control the extended screen according to the control information.

[0150] In some embodiments, the type determination module 501 is further configured to determine a data type according to the motion data, and calculate the motion data according to the data type to obtain the motion type corresponding to the terminal device.

[0151] In some embodiments, the type determination module 501 is further configured to, if the data type is angular velocity data, determine a rotation angle of the terminal device according to the angular velocity data, and determine the motion type corresponding to the terminal device according to the rotation angle; if the data type is acceleration data, determine an acceleration slope corresponding to the acceleration data, and determine the motion type corresponding to the terminal device according to the acceleration slope.

[0152] In some embodiments, the type determining module 501 is further configured to obtain angular velocity data corresponding to the first time period, the angular velocity data comprising an angular velocity component and an angular velocity modulus; if the angular velocity component and / or the angular velocity modulus corresponding to the first time period exceeds a respective preset angular velocity threshold, processing the angular velocity component and / or the angular velocity modulus corresponding to the first time period to obtain a first rotation angle; if the first rotation angle is within a preset angle range, obtaining an angular velocity component corresponding to a second time period, and processing the angular velocity component corresponding to the second time period to obtain a second rotation angle; and determining the motion type corresponding to the terminal device according to the first rotation angle and the second rotation angle.

[0153] In some embodiments, the type determining module 501 is further configured to obtain a first angular velocity component and a second angular velocity component within the first time period; determine an angular velocity slope according to the first angular velocity component and corresponding timestamp information and the second angular velocity component and corresponding timestamp information; if the first rotation angle is within a preset angle range and the angular velocity slope is greater than a preset angular velocity slope, obtain an angular velocity component corresponding to a second time period, and process the angular velocity component corresponding to the second time period to obtain a second rotation angle.

[0154] In some embodiments, the type determining module 501 is further configured to determine a difference between the first rotation angle and the second rotation angle; and if the difference between the first rotation angle and the second rotation angle is less than a preset angle threshold, determine that the motion type corresponding to the terminal device is point motion.

[0155] In some embodiments, the type determining module 501 is further configured to obtain acceleration data corresponding to a plurality of time points, the acceleration data comprising an acceleration component and an acceleration modulus; if the acceleration component and / or the acceleration modulus corresponding to a first time point exceeds a respective preset acceleration threshold, processing the acceleration component and / or the acceleration modulus corresponding to the first time point and its adjacent time points to obtain a first acceleration slope; and if the first acceleration slope exceeds a preset slope threshold, determining that the motion type corresponding to the terminal device is single tap.

[0156] In some embodiments, the type determining module 501 is further configured to determine whether the acceleration component and the acceleration modulus corresponding to the second time exceed respective preset acceleration thresholds; if the acceleration component and / or the acceleration modulus corresponding to the second time exceeds the preset acceleration threshold, the acceleration component and / or the acceleration modulus corresponding to the second time and its adjacent time are processed to obtain a second acceleration slope; if the second acceleration slope exceeds the preset slope threshold, the difference between the second time and the first time is determined; if the difference between the second time and the first time is less than a preset time threshold, the motion type of the terminal device is determined as multiple taps.

[0157] In some embodiments, the type determining module 501 is further configured to obtain a mapping relationship between a preset motion type and control information, and determine whether the motion type of the terminal device is the preset motion type; if the motion type of the terminal device is the preset motion type, the control information corresponding to the motion type of the terminal device is determined according to the mapping relationship.

[0158] It should be noted that, for the convenience and brevity of description, the specific working process of the above-described device and each module and unit can refer to the corresponding process in the foregoing method embodiments, which will not be described herein.

[0159] The method and device of the present application can be used in many general or special computing system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0160] Please refer to Figure 8 , Figure 8 A structural schematic block diagram of a wearable device 600 provided by an embodiment of the present application is shown.

[0161] As shown in Figure 8 , the wearable device 600 can include a processor 601 and a memory 602, and the processor 601 and the memory 602 are connected through a bus 603, such as an I2C (Inter-integrated Circuit) bus.

[0162] In an example embodiment, the processor 601 can be configured to provide computing and control capabilities, supporting the operation of the entire terminal device. The processor 601 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or can also be any conventional processor.

[0163] Specifically, the memory 602 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a U disk or a mobile hard disk, etc.

[0164] Those skilled in the art can understand that, Figure 8 The structure shown in the figure is only a block diagram of part of the structure related to the embodiment of the present application, and does not constitute a limitation on the terminal device to which the embodiment of the present application is applied. Specifically, the server can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0165] The processor 601 is configured to run the computer program stored in the memory and implement any one of the application methods provided by the embodiments of the present application for controlling the extended screen of the wearable device when executing the computer program.

[0166] In an embodiment, the processor 601 is configured to run the computer program stored in the memory and implement the following steps when executing the computer program: obtaining control information corresponding to the extended screen according to the motion type of the terminal device; wherein the motion type of the terminal device is determined according to the motion data collected by the terminal device; and controlling the extended screen according to the control information.

[0167] In an embodiment, the processor 601 is further configured to determine the data type according to the motion data; and calculate the motion data according to the data type to obtain the motion type corresponding to the terminal device.

[0168] In an embodiment, the processor 601 is further configured to: if the data type is angular velocity data, determine a rotation angle of the terminal device according to the angular velocity data, and determine the motion type corresponding to the terminal device according to the rotation angle; and if the data type is acceleration data, determine an acceleration slope corresponding to the acceleration data, and determine the motion type corresponding to the terminal device according to the acceleration slope.

[0169] In an embodiment, the processor 601 is further configured to: obtain angular velocity data corresponding to a first time period, the angular velocity data including an angular velocity component and an angular velocity modulus; if the angular velocity component and / or the angular velocity modulus corresponding to the first time period exceeds a respective preset angular velocity threshold, process the angular velocity component and / or the angular velocity modulus corresponding to the first time period to obtain a first rotation angle; if the first rotation angle is within a preset angle range, obtain an angular velocity component corresponding to a second time period, and process the angular velocity component corresponding to the second time period to obtain a second rotation angle; and determine the motion type corresponding to the terminal device according to the first rotation angle and the second rotation angle.

[0170] In an embodiment, the processor 601 is further configured to: obtain a first angular velocity component and a second angular velocity component within the first time period; determine an angular velocity slope according to the first angular velocity component and corresponding timestamp information and the second angular velocity component and corresponding timestamp information; if the first rotation angle is within a preset angle range and the angular velocity slope is greater than a preset angular velocity slope, obtain an angular velocity component corresponding to a second time period, and process the angular velocity component corresponding to the second time period to obtain a second rotation angle.

[0171] In an embodiment, the processor 601 is further configured to: determine a difference between the first rotation angle and the second rotation angle; and if the difference between the first rotation angle and the second rotation angle is less than a preset angle threshold, determine that the motion type corresponding to the terminal device is point motion.

[0172] In an embodiment, the processor 601 is further configured to: obtain acceleration data corresponding to a plurality of time points, the acceleration data including an acceleration component and an acceleration modulus; if the acceleration component and / or the acceleration modulus corresponding to a first time point exceeds a respective preset acceleration threshold, process the acceleration component and / or the acceleration modulus corresponding to the first time point and an adjacent time point to obtain a first acceleration slope; and if the first acceleration slope exceeds a preset slope threshold, determine that the motion type corresponding to the terminal device is single tap.

[0173] In an embodiment, the processor 601 is further configured to determine whether the acceleration component and the acceleration module value corresponding to the second time point exceed the respective preset acceleration threshold values; if the acceleration component and / or the acceleration module value corresponding to the second time point exceeds the preset acceleration threshold value, process the acceleration component and / or the acceleration module value corresponding to the second time point and its adjacent time points to obtain a second acceleration slope; if the second acceleration slope exceeds the preset slope threshold value, determine the difference between the second time point and the first time point; and if the difference between the second time point and the first time point is less than a preset time threshold value, determine that the motion type corresponding to the terminal device is multiple taps.

[0174] It should be noted that, for the convenience and brevity of description, the specific working process of the wearable device described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0175] Please refer to Figure 9 , Figure 9 A structural schematic block diagram of a terminal device 700 provided in an embodiment of the present application is shown.

[0176] As shown in Figure 9 , the terminal device 700 can include a processor 701 and a memory 702, which are connected through a bus 703, such as an I2C (Inter-integrated Circuit) bus.

[0177] In an exemplary embodiment, the processor 701 can be configured to provide computing and control capabilities to support the operation of the entire terminal device. The processor 701 can be a central processing unit (CPU), and the processor 701 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0178] Specifically, the memory 702 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a U disk or a mobile hard disk, etc.

[0179] Those skilled in the art can understand that Figure 9 The structure shown in the figure is only a block diagram of part of the structure related to the embodiment of the present application, and does not constitute a limitation on the terminal device to which the embodiment of the present application is applied. Specifically, the server can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0180] The processor 701 is configured to run a computer program stored in the memory and implement any one of the control methods of the extended screen of the terminal device provided in the embodiments of the present application when the computer program is executed.

[0181] In an embodiment, the processor 701 is configured to run a computer program stored in the memory and implement the following steps when the computer program is executed: acquiring motion data collected by a motion collection device in the terminal device; determining a motion type corresponding to the terminal device according to the motion data; and sending the motion type to the wearable device, so that the wearable device determines control information corresponding to the extended screen according to the motion type, and controls the extended screen according to the control information.

[0182] It should be noted that, for the convenience and brevity of description, the specific working process of the terminal device described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0183] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed, any one of the control methods of the extended screen of the terminal device or the control methods of the extended screen of the wearable device provided in the embodiments of the present application is implemented.

[0184] The computer readable storage medium can be an internal storage unit of the wearable device, such as a memory of the wearable device. The computer readable storage medium can also be an external storage device of the wearable device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0185] Further, the computer readable storage medium can mainly include a program storage area and a data storage area. The program storage area can store an operating system, at least one application required by a function, etc.

[0186] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for controlling an extended screen, characterized in that, Applied to wearable devices, the method includes: According to the motion type of the terminal device, control information corresponding to the extended screen is obtained; wherein, the motion type of the terminal device is determined according to the motion data collected by the terminal device; there are multiple extended screens, the extended screens display the content of the terminal device and the wearable device, the extended screens are anchored to the pose of the terminal device, and are an extended display outside the screen of the terminal device; The extended screen is controlled according to the control information.

2. The method according to claim 1, characterized in that, The motion type of the terminal device is determined based on the motion data collected by the terminal device, including: The data type is determined based on the motion data; Based on the data type, the motion data is calculated to obtain the motion type corresponding to the terminal device.

3. The method according to claim 2, characterized in that, The step of calculating the motion type corresponding to the terminal device based on the data type includes: If the data type is angular velocity data, then the rotation angle of the terminal device is determined by the angular velocity data, and the motion type of the terminal device is determined according to the rotation angle. If the data type is acceleration data, then the acceleration slope corresponding to the acceleration data is determined, and the motion type corresponding to the terminal device is determined based on the acceleration slope.

4. The method according to claim 3, characterized in that, The step of determining the rotation angle of the terminal device using the angular velocity data, and determining the corresponding motion type of the terminal device based on the rotation angle, includes: Obtain angular velocity data corresponding to the first time period, wherein the angular velocity data includes angular velocity components and angular velocity magnitude; If the angular velocity component and / or angular velocity magnitude corresponding to the first time period exceeds their respective preset angular velocity thresholds, then the angular velocity component and / or angular velocity magnitude corresponding to the first time period are processed to obtain the first rotation angle. If the first rotation angle is within the preset angle range, the angular velocity component corresponding to the second time period is obtained, and the angular velocity component corresponding to the second time period is processed to obtain the second rotation angle. The motion type of the terminal device is determined based on the first rotation angle and the second rotation angle.

5. The method according to claim 4, characterized in that, The angular velocity data also includes timestamp information. After processing the angular velocity component and / or angular velocity magnitude corresponding to the first time period to obtain the first rotation angle, if the angular velocity component and / or angular velocity magnitude corresponding to the first time period exceeds a preset angular velocity threshold, the method further includes: Obtain the first angular velocity component and the second angular velocity component within the first time period; The angular velocity slope is determined based on the first angular velocity component and its corresponding timestamp information, and the second angular velocity component and its corresponding timestamp information. If the first rotation angle is within a preset angle range and the angular velocity slope is greater than the preset angular velocity slope, then the angular velocity component corresponding to the second time period is obtained, and the angular velocity component corresponding to the second time period is processed to obtain the second rotation angle.

6. The method according to claim 4, characterized in that, Determining the motion type of the terminal device based on the first rotation angle and the second rotation angle includes: Determine the difference between the first rotation angle and the second rotation angle; If the difference between the first rotation angle and the second rotation angle is less than a preset angle threshold, then the motion type corresponding to the terminal device is determined to be jogging.

7. The method according to claim 3, characterized in that, Determining the acceleration slope corresponding to the acceleration data, and determining the motion type corresponding to the terminal device based on the acceleration slope, includes: Acquire acceleration data at multiple time points, wherein the acceleration data includes acceleration components and acceleration magnitude; If the acceleration component and / or acceleration magnitude value corresponding to the first moment exceeds their respective preset acceleration threshold, then the acceleration component and / or acceleration magnitude value corresponding to the first moment and its adjacent moments are processed to obtain the first acceleration slope. If the first acceleration slope exceeds a preset slope threshold, then the motion type corresponding to the terminal device is determined to be a single tap.

8. The method according to claim 7, characterized in that, If the acceleration slope exceeds a preset slope threshold, then after determining that the motion type corresponding to the terminal device is a single tap, the method further includes: Determine whether the acceleration component and acceleration magnitude at the second moment exceed the preset acceleration threshold. If the acceleration component and / or acceleration magnitude value corresponding to the second time moment exceeds their respective preset acceleration threshold, then the acceleration component and / or acceleration magnitude value corresponding to the second time moment and its adjacent time moments are processed to obtain the second acceleration slope. If the second acceleration slope exceeds the preset slope threshold, then the difference between the second moment and the first moment is determined; If the difference between the second time point and the first time point is less than a preset time threshold, then the motion type corresponding to the terminal device is determined to be multiple taps.

9. The method according to claim 1, characterized in that, The step of obtaining control information corresponding to the extended screen of the terminal device based on the motion type of the terminal device includes: Obtain the mapping relationship between preset motion types and control information, and determine whether the motion type of the terminal device is the preset motion type; If the motion type of the terminal device is the preset motion type, then the control information corresponding to the motion type of the terminal device is determined according to the mapping relationship.

10. The method according to claim 1, characterized in that, The control information includes at least one of the following: Control the extended screen to execute the open command; or, Control the extended screen to execute selection instructions; or, Control the extended screen to execute a close command; or, Control the extended screen to execute the switching command; or, Control the extended screen to execute the zoom command; or, Control the extended screen to execute a shrink command.

11. A method for controlling an extended screen, characterized in that, Applied to a terminal device, the method includes: Acquire motion data collected by the motion acquisition device in the terminal device; The motion type corresponding to the terminal device is determined based on the motion data; The motion type is sent to the wearable device so that the wearable device can determine the control information corresponding to the extended screen based on the motion type, and control the extended screen based on the control information; there are multiple extended screens, and the extended screens display the content of the terminal device and the wearable device. The extended screens are anchored to the pose of the terminal device and serve as an extended display outside the screen of the terminal device.

12. A control device for extending a screen, characterized in that, The control device for the extended screen, applied to wearable devices, includes: The motion type determination module is used to obtain control information corresponding to the extended screen based on the motion type of the terminal device; wherein, the motion type of the terminal device is determined based on the motion data collected by the terminal device; there are multiple extended screens, the extended screens display the content of the terminal device and the wearable device, and the extended screens are anchored to the pose of the terminal device, serving as an extended display outside the screen of the terminal device; An extended screen control module is used to control the extended screen according to the control information.

13. A wearable device, characterized in that, The wearable device is used to provide an extended screen for a terminal device. The wearable device includes a memory and a processor. The memory is connected to the processor and is used to store programs. The processor is used to implement the control method for the extended screen as described in any one of claims 1 to 10 by running the programs stored in the memory.

14. A terminal device, characterized in that, The extended screen of the terminal device is provided by a wearable device, and the terminal device includes a memory and a processor; wherein the memory is connected to the processor and is used to store programs, and the processor is used to implement the control method of the extended screen as described in claim 11 by running the programs stored in the memory.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the control method for the extended screen as described in any one of claims 1-10 or the control method for the extended screen as described in claim 11.

Images