A control method and a terminal device

Abstract

The embodiment of the present application provides a control method and a terminal device, a first key is determined according to first posture change data of the terminal device and a first mapping relationship, the first mapping relationship includes a corresponding relationship between the first posture change data and the first key; and an instruction associated with the first key is executed. The method provided by the embodiment of the present application can be applied to a game scene, and by using the method provided by the embodiment of the present application, a user can realize multi-point simultaneous control and improve operation speed, thereby improving the operation experience of the user.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of terminal devices, and more particularly, to a control method and a terminal device. BACKGROUND

[0002] Currently, the control of a terminal device game by a player is based on a screen pressure sensing manner, and the player needs to correctly touch a screen key to accurately control, especially in the case of intensive operation, the player may need to accurately touch multiple screen keys in a short time, and it is difficult for the player to improve his operation speed, or the error probability is higher, and some keys are difficult to reach in the posture of holding the terminal device with both hands, and the game operation experience is not high. Some improvements such as the design of game shoulder keys can increase the number of touch keys, but the essence is still to realize control through pressure sensing operation, and additional devices need to be added to the terminal device. SUMMARY

[0003] Embodiments of the present application provide a control method and a terminal device, which help a user to realize multi-point simultaneous control, thereby improving the operation experience of the user.

[0004] In a first aspect, a control method is provided, which includes: determining a first key according to first posture change data of a terminal device and a first mapping relationship, the first mapping relationship including a corresponding relationship between the first posture change data and the first key; and executing an instruction associated with the first key.

[0005] Optionally, the first key can be a game skill key.

[0006] Optionally, the first key can be a key of other game interfaces, such as a setting key.

[0007] Optionally, the first key can be a physical key, such as a volume key and a lock screen key.

[0008] Optionally, the first mapping relationship can be started in a game battle interface or a game setting interface. It should be understood that when the user starts the first mapping relationship in the game battle interface, it can be more convenient.

[0009] Optionally, the first posture change data is posture change data of the terminal device that can be detected during an auxiliary operation. The auxiliary operation can be a single rotation operation or multiple rotation operations around an x-axis (a bottom edge of the terminal device), a y-axis (a side edge of the terminal device), or a z-axis (perpendicular to the terminal device), and the directions of the multiple rotation operations are the same.

[0010] In the embodiments of the present application, the keys of the terminal device are associated with the posture change data of the terminal device, so that a control method other than touching the keys is provided, which helps the user to realize multi-point simultaneous control, improve the operation speed, and reduce the probability of user misoperation, thereby improving the user's operation experience. Especially for some games that need to be controlled by the user in a two-handed holding posture, the method provided by the embodiments of the present application can be used to conveniently control the game without affecting the user's normal holding posture of the terminal device.

[0011] With reference to the first aspect, in some implementations of the first aspect, the first posture change data is associated with a first icon, and a first interface is displayed, the first interface including the first key and the first icon; and the position of the first key and the position of the first icon have a corresponding relationship.

[0012] Optionally, the area of the region where the first icon is located is smaller than the area of the region where the first key is located, and the display region of the first icon can be located in the display region of the first key. Alternatively, the display region of the first icon and the display region of the first key partially overlap.

[0013] Optionally, the transparency of the first icon is greater than or equal to a preset value.

[0014] Optionally, the position of the first icon is near the position of the corresponding first key.

[0015] In the embodiments of the present application, the first posture change data is associated with the first icon, and the first icon is displayed on the game interface, and the position of the first icon corresponds to the position of the corresponding first key, so that the user can intuitively understand that the first posture change data corresponding to the first icon corresponds to the first key, thereby facilitating the user to use the control method provided by the embodiments of the present application to control the game.

[0016] With reference to the first aspect, in some implementations of the first aspect, in response to executing the instructions, the shape of the first icon is updated.

[0017] In the embodiments of the present application, when the terminal device detects the first posture change data, in addition to executing the instructions associated with the first key corresponding to the first posture change data, the shape of the first icon associated with the first posture change data is also updated, which is used to prompt the user that the corresponding operation of the first posture change data has just been performed.

[0018] With reference to the first aspect, in some implementations of the first aspect, before displaying the first interface, a second interface is displayed, the second interface being a relationship setting interface of the first icon and the first key of the terminal device; and in response to detecting the operation of the user, the corresponding relationship between the first icon and the first key is set.

[0019] Optionally, the first icon can be displayed in the position area corresponding to the first button. It should be understood that the first icon is associated with the first button at this time.

[0020] Optionally, the first icon can be displayed in an area other than the position area corresponding to the first button. It should be understood that the first icon is not associated with any first button at this time.

[0021] Optionally, the second interface is the same interface as the first interface. It should be understood that the user can change the correspondence between the first button and the first icon during the game.

[0022] Optionally, the user clicks the "enable / disable auxiliary operation" icon before setting the relationship between the first icon and the first button. The icon can appear in the game loading interface or in the game interface.

[0023] Optionally, the user needs to determine the available first gesture change data before setting the relationship between the first icon and the first button, in other words, the user needs to determine the available auxiliary operation.

[0024] Optionally, the user needs to determine the available first gesture change data and its corresponding interface before setting the relationship between the first icon and the first button, in other words, the user needs to determine the available auxiliary operation and its corresponding interface.

[0025] It should be understood that the user can change the mapping relationship between the first icon and the first button according to his own needs, thereby changing the relationship between the first gesture data and the first button. Optionally, the user can move the first icon to the position area corresponding to the different first buttons in the setting interface, change the appearance of the first icon to prompt the associable information when the terminal device detects that the first icon is moved to the first button, and retain the last mapping relationship when the terminal device detects that the user releases the hand.

[0026] Through the above scheme, the user can flexibly adjust the mapping relationship between the first gesture change data and the first button, so as to be more in line with his own operation habits or device conditions.

[0027] With reference to the first aspect, in some implementations of the first aspect, before determining the first key according to the first attitude change data of the terminal device and the first mapping relationship, the method further includes: enabling, by an auxiliary operation management program, the first attitude change data, the auxiliary operation management program being configured to manage the first attitude change data and being connected with a target application program (APP) so that the target APP can be controlled by the first attitude change data; receiving, by the auxiliary operation management program, data reported by a sensor; and reporting, by the auxiliary operation management program, a position corresponding to the first attitude change data to the target APP under a preset condition, the preset condition including that the auxiliary operation management program detects that the data sent by the sensor reaches the first attitude change data; and determining, by the target APP, the first key according to the position corresponding to the first attitude change data of the terminal device and a position of the first key, and executing an instruction associated with the first key.

[0028] Directly calling the sensor data by the auxiliary operation management program can make the data transmission path shorter and the running efficiency higher.

[0029] With reference to the first aspect, in some implementations of the first aspect, the first attitude change data is defined by a user.

[0030] Optionally, the user can perform corresponding operations according to the illustrated actions or animations in a setting interface, the terminal device reads corresponding attitude change data, calculates corresponding first, second and third threshold values and time intervals and the like, and changes corresponding parameter settings.

[0031] Through the above scheme, the user can make the setting of the auxiliary operation more suitable for his own habits or the hardware conditions of the terminal device.

[0032] With reference to the first aspect, in some implementations of the first aspect, the first key includes a plurality of keys.

[0033] Optionally, the plurality of keys are independent of each other.

[0034] Through the above scheme, the user only needs one auxiliary operation to perform corresponding control of the plurality of keys, which can further improve his operation speed.

[0035] With reference to the first aspect, in some implementations of the first aspect, the first attitude change data includes a first speed when the terminal device moves in a first direction, and determining the first key according to the first attitude change data of the terminal device and the first mapping relationship includes: when the first speed is greater than or equal to a first threshold value, determining the first key according to the first attitude change data and the first mapping relationship.

[0036] Optionally, the first posture change data comprises a plurality of first speeds when the terminal device moves in the first direction, so that the terminal device is more accurate in recognizing the user operation.

[0037] Optionally, the first speed comprises acceleration, linear speed, angular speed, and angular acceleration, and the first speed can also be a derivative of the above-mentioned speeds.

[0038] Through the above scheme, the user's single rotation of the terminal device posture operation will be recognized by the terminal device as corresponding first posture change data, and the first key corresponding to the user operation is determined according to the first relationship mapping table.

[0039] With reference to the first aspect, in some implementations of the first aspect, the first posture change data comprises a second speed when the terminal device moves in a second direction for the first time and a third speed when the terminal device moves in the second direction for the second time, and the determining of the first key according to the first posture change data of the terminal device and the first mapping relationship further comprises: when a time interval between a starting time of the first movement in the second direction and a starting time of the second movement in the second direction is within a preset range, the second speed is greater than or equal to a second threshold, and the third speed is greater than or equal to a third threshold, the first key is determined according to the first posture change data and the first mapping relationship.

[0040] Optionally, the second threshold is greater than the third threshold.

[0041] Through the above scheme, the user's two rotations of the terminal device posture in the same direction will be recognized by the terminal device as corresponding first posture change data, so that the first key corresponding to the user operation is determined according to the first relationship mapping table. The setting of the two consecutive movements in the same direction helps to reduce the probability that the terminal device recognizes the user's unintentional and small degree of rotation of the terminal device as the first terminal device posture change data.

[0042] With reference to the first aspect, in some implementations of the first aspect, the control bar or the floating window of the terminal device exists a second icon, and the second icon is used to start or stop the first posture change data.

[0043] By providing the second icon in the control bar or the floating window to start or stop the first posture change data, that is, the auxiliary operation, the user can start or stop the auxiliary operation more conveniently, so that the user experience is better.

[0044] With reference to the first aspect, in some implementations of the first aspect, the first key is a game key.

[0045] It should be understood that the game key can be a game skill key or other key of a game interface.

[0046] In a second aspect, a terminal device is provided, comprising one or more memories and one or more processors. The one or more memories store one or more computer programs comprising instructions that, when executed by the one or more processors, cause the terminal device to perform the following steps: the first key is a game skill key.

[0047] Optionally, the first key can be a key of other game interface, such as a setting key.

[0048] Optionally, the first key can be a physical key, such as a volume key and a lock screen key.

[0049] Optionally, the first mapping relationship can be started in a game battle interface or a game setting interface. It should be understood that when the user starts the first mapping relationship in the game battle interface, it can be more convenient.

[0050] Optionally, the first posture change data is posture change data that can be detected by the terminal device when the auxiliary operation is performed. The auxiliary operation can be a single rotation operation or multiple rotation operations around the x-axis (the bottom edge of the terminal device), the y-axis (the side edge of the terminal device), or the z-axis (perpendicular to the terminal device), and the directions of the multiple rotation operations are the same.

[0051] In combination with the second aspect, in some implementations of the second aspect, the first posture change data is associated with a first icon, and the instructions, when executed by the one or more processors, cause the terminal device to perform the following steps: display a first interface, the first interface comprising the first key and the first icon; and wherein the position of the first key and the position of the first icon have a corresponding relationship.

[0052] Optionally, the area of the region where the first icon is located is smaller than the area of the region where the first key is located, and the display region of the first icon can be located in the display region of the first key. Alternatively, the display region of the first icon and the display region of the first key partially overlap.

[0053] Optionally, the transparency of the first icon is greater than or equal to a preset value.

[0054] Optionally, the position of the first icon is near the position of the first key corresponding to the first icon.

[0055] In combination with the second aspect, in some implementations of the second aspect, the instructions, when executed by the one or more processors, cause the terminal device to perform the following steps: in response to executing the instructions, update the shape of the first icon.

[0056] With reference to the second aspect, in some implementations of the second aspect, before displaying the first interface, the instructions, when executed by the one or more processors, cause the terminal device to display a second interface, the second interface being a setting interface of a first icon and a first key of the terminal device; and the one or more processors set the correspondence between the first icon and the first key in response to detecting a user operation.

[0057] Optionally, the first icon can be displayed in a position area corresponding to the first key. It should be understood that the first icon at this time is associated with the first key.

[0058] Optionally, the first icon can be displayed in an area other than the position area corresponding to the first key. It should be understood that the first icon at this time is not associated with any first key.

[0059] Optionally, the second interface is the same interface as the first interface. It should be understood that the user can change the correspondence between the first key and the first icon during the game.

[0060] Optionally, the first mapping relationship can be started in a game battle interface or a game setting interface. It should be understood that it is more convenient for the user to start the first mapping relationship in the game battle interface.

[0061] Optionally, before setting the relationship between the first icon and the first key, the user needs to determine the available first posture change data, in other words, the available auxiliary operation needs to be determined.

[0062] Optionally, before setting the relationship between the first icon and the first key, the user needs to determine the available first posture change data and the interface corresponding thereto, in other words, the available auxiliary operation and the interface corresponding thereto need to be determined.

[0063] With reference to the second aspect, in some implementations of the second aspect, the instructions, when executed by the one or more processors, cause the terminal device to perform the following steps: the first posture change data is user-defined.

[0064] Optionally, the user can perform corresponding operations according to the figure action or animation in the setting interface, the terminal device reads the corresponding posture change data, calculates the corresponding first threshold, second threshold, third threshold and time interval and the like, and changes the corresponding parameter settings.

[0065] With reference to the second aspect, in some implementations of the second aspect, the instructions, when executed by the one or more processors, cause the terminal device to perform the following steps:

[0066] The auxiliary operation management program enables first posture change data, the auxiliary operation management program is used for realizing management to the first posture change data, and is connected with target application program, so that target application program can be manipulated by first posture change data;The auxiliary operation management program receives the data reported by the sensor;The auxiliary operation management program reports the position corresponding to the first change data to the target application program under the preset condition, and the preset condition includes that the auxiliary operation management program detects that the data sent by the sensor reaches the first posture change data;Wherein, according to the first posture change data of the terminal device and the first mapping relationship, the first key is determined, and the instruction associated with the first key is executed, including: the target APP determines the first key according to the position corresponding to the first posture change data of the terminal device and the position of the first key, and executes the instruction associated with the first key.

[0067] With reference to the second aspect, in some implementations of the second aspect, when the instruction is executed by the one or more processors, the terminal device device is caused to perform the following steps: the first key includes a plurality of keys.

[0068] Optionally, the plurality of keys are independent of each other.

[0069] With reference to the second aspect, in some implementations of the second aspect, the first posture change data includes a first speed when the terminal device moves in a first direction, and when the instruction is executed by the one or more processors, the terminal device device is caused to perform the following steps: determining the first key according to the first posture change data and the first mapping relationship includes: when the first speed is greater than or equal to a first threshold, determining the first key according to the first posture change data and the first mapping relationship.

[0070] With reference to the second aspect, in some implementations of the second aspect, the first posture change data includes a second speed when the terminal device moves in a second direction for the first time and a third speed when the terminal device moves in the second direction for the second time, and when the instruction is executed by the one or more processors, the terminal device device is caused to perform the following steps: determining the first key according to the first posture change data and the first mapping relationship further includes: when the time interval between the starting time of the first movement in the second direction and the starting time of the second movement in the second direction is within a preset range, the second speed is greater than or equal to a second threshold, and the third speed is greater than or equal to a third threshold, determining the first key according to the first posture change data and the first mapping relationship.

[0071] In some implementations of the second aspect, when the instructions are executed by the one or more processors, the terminal device is caused to perform the following steps: the control bar or the floating window of the terminal device has a second icon, and the second icon is used to turn on or turn off the first gesture change data.

[0072] In some implementations of the second aspect, when the instructions are executed by the one or more processors, the terminal device is caused to perform the following steps: the first key is a game key.

[0073] In a third aspect, a terminal device is provided, which includes a sensor, a processor, a memory, and a display screen. The sensor is configured to detect a posture of the terminal device, the memory is configured to store a computer program, and the processor is configured to call and run the computer program from the memory, so that the terminal device performs the method in the first aspect and various possible implementations thereof.

[0074] Optionally, the processor is one or more, and the memory is one or more.

[0075] Optionally, the memory can be integrated with the processor, or the memory and the processor can be separately arranged.

[0076] Optionally, the sensor can be one or more, and the one or more sensors can collectively detect the same parameter, or different sensors can be used to detect different parameters.

[0077] Optionally, the terminal device can further include a transmitter and a receiver.

[0078] In a fourth aspect, an apparatus for controlling a terminal device is provided, which includes units for performing the steps of the method in any of the first aspect to the third aspect and various implementations thereof.

[0079] The apparatus for controlling a terminal device can be arranged in the terminal device. In this case, the detecting unit in the apparatus for controlling a terminal device can be implemented using the sensor in the terminal device. The processing unit in the apparatus for controlling a terminal device can be independent of the processor in the terminal device. Alternatively, the processing unit in the apparatus for controlling a terminal device can be implemented using the processor in the terminal device.

[0080] Alternatively, the apparatus for controlling a terminal device itself can be the terminal device.

[0081] In a fifth aspect, a computer storage medium is provided, which includes computer instructions, and when the computer instructions are run on a terminal device, the terminal device performs any of the methods in any of the aspects.

[0082] In a sixth aspect, a computer program product is provided, comprising computer instructions which, when run on a terminal device, cause the terminal device to perform any of the possible methods of any of the aspects described above. BRIEF DESCRIPTION OF DRAWINGS

[0083] Figure 1 A structural schematic diagram of a terminal device to which the game control method provided by the embodiments of the present application is applicable.

[0084] Figure 2 A software framework schematic diagram of the terminal device shown in Figure 1

[0085] Figure 3 A flowchart of the game control method 200 provided by the embodiments of the present application.

[0086] Figure 4A A schematic diagram of a single auxiliary operation provided by the embodiments of the present application Figure 1 .

[0087] Figure 4B A schematic diagram of a single auxiliary operation provided by the embodiments of the present application Figure 2 .

[0088] Figure 4C A schematic diagram of a single auxiliary operation provided by the embodiments of the present application Figure 3 .

[0089] Figure 5 A flowchart of another game control method 400 provided by the embodiments of the present application.

[0090] Figure 6A A schematic diagram of a scenario of the game control method provided by the embodiments of the present application.

[0091] Figure 6B A schematic diagram of another scenario of the game control method provided by the embodiments of the present application.

[0092] Figure 6C A schematic diagram of still another scenario of the game control method provided by the embodiments of the present application.

[0093] Figure 7A A schematic diagram of still another scenario of the game control method provided by the embodiments of the present application.

[0094] Figure 7B A schematic diagram of still another scenario of the game control method provided by the embodiments of the present application.

[0095] Figure 8A A schematic diagram of still another scenario of the game control method provided by the embodiments of the present application.

[0096] Figure 8B ​Still another scenario of the game control method provided by the embodiment of the present application is shown in the figure.

[0097] Figure 8C Still another scenario of the game control method provided by the embodiment of the present application is shown in the figure.

[0098] Figure 8D Still another scenario of the game control method provided by the embodiment of the present application is shown in the figure.

[0099] Figure 8E Still another scenario of the game control method provided by the embodiment of the present application is shown in the figure.

[0100] Figure 9A Still another scenario of the game control method provided by the embodiment of the present application is shown in the figure.

[0101] Figure 9B Still another scenario of the game control method provided by the embodiment of the present application is shown in the figure.

[0102] Figure 9C Still another scenario of the game control method provided by the embodiment of the present application is shown in the figure.

[0103] Figure 10A Still another scenario of the game control method provided by the embodiment of the present application is shown in the figure.

[0104] Figure 10B Still another scenario of the game control method provided by the embodiment of the present application is shown in the figure.

[0105] Figure 10C Still another scenario of the game control method provided by the embodiment of the present application is shown in the figure.

[0106] Figure 11A Still another scenario of the game control method provided by the embodiment of the present application is shown in the figure.

[0107] Figure 11B Still another scenario of the game control method provided by the embodiment of the present application is shown in the figure.

[0108] Figure 11C Still another scenario of the game control method provided by the embodiment of the present application is shown in the figure.

[0109] Figure 12A Still another scenario of the game control method provided by the embodiment of the present application is shown in the figure.

[0110] Figure 12B Still another scenario of the game control method provided by the embodiment of the present application is shown in the figure.

[0111] Figure 12C Still another scenario of the game control method provided by the embodiment of the present application is shown in the figure.

[0112] Figure 12D Another scenario of the game control method provided by the embodiments of the present application is shown.

[0113] Figure 13 Another scenario of the game control method provided by the embodiments of the present application is shown.

[0114] Figure 14 A flowchart of the game control method provided by the embodiments of the present application is shown.

[0115] Figure 15 Another flowchart of the game control method provided by the embodiments of the present application is shown.

[0116] Figure 16 A schematic block diagram of the apparatus 1300 of the terminal device provided by the embodiments of the present application is shown. DETAILED DESCRIPTION

[0117] The technical solutions in the embodiments of the present application will be described below with reference to the drawings.

[0118] The game control method provided by the embodiments of the present application can be applied to a terminal device. For example, the game control method can be applied to a terminal device, a tablet computer, a personal digital assistant (PDA) and the like, and the embodiments of the present application do not limit the specific type of the terminal device.

[0119] For the sake of convenience of description, a smart phone will be taken as an example for description below.

[0120] Figure 1 A schematic diagram of an example of the terminal device is shown. As shown in FIG. 1, the terminal device 100 can include a processor 110, an internal memory 120, an audio module 130, an I / O subsystem 140, a sensor module 150, a display screen 160, other input devices 170, a communication module 180 and a battery module 190 and the like. Figure 1

[0121] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 can include more or fewer components than shown in the diagram, or combine certain components, or split certain components, or different arrangement of components. The components shown in the diagram can be implemented in hardware, software or a combination of software and hardware.

[0122] (1) Processor 110

[0123] ​The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0124] The controller can be the nerve center and command center of the terminal device 100. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of instruction fetching and instruction execution.

[0125] The memory can also be provided in the processor 110, used to store instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can save instructions or data that the processor 110 has just used or repeatedly uses. If the processor 110 needs to use the instructions or data again, it can be directly called from the memory. Avoiding repeated access reduces the waiting time of the processor 110, thus improving the efficiency of the system.

[0126] In some embodiments, the processor 110 can include one or more interfaces. The interface can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a mobile industry processor interface (MIPI), and a GPIO interface, etc.

[0127] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 can include multiple sets of I2C buses. The processor 110 can be coupled to the touch sensor 1510, the charger, the flash, the camera, etc. through different I2C bus interfaces respectively. For example, the processor 110 can be coupled to the touch sensor 1510 through an I2C interface, so that the processor 110 and the touch sensor 1510 communicate through the I2C bus interface, and the touch function of the terminal device 100 is realized.

[0128] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple sets of I2S buses. The processor 110 can be coupled to the audio module 130 through the I2S bus, and communication between the processor 110 and the audio module 130 is realized. In some embodiments, the audio module 130 can deliver an audio signal to the wireless communication module through the I2S interface, and the function of answering a phone through a Bluetooth headset is realized.

[0129] The MIPI interface can be used to connect the processor 110 and the display screen 160, the camera and other peripheral devices. The MIPI interface includes a camera serial interface (CSI), a display screen serial interface (DSI), etc. In some embodiments, the processor 110 and the camera communicate through the CSI interface, and the shooting function of the terminal device 100 is realized. The processor 110 and the display screen 160 communicate through the DSI interface, and the display function of the terminal device 100 is realized.

[0130] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 and the display screen 160, the audio module 130, the sensor module 150, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0131] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative and does not constitute a structural limitation of the terminal device 100. In some other embodiments of the present application, the terminal device 100 can also use different interface connection methods or combinations of multiple interface connection methods in the above embodiments.

[0132] (2) Internal memory 120

[0133] The internal memory 120 can be used to store computer executable program codes including instructions. The processor 110 performs various functional applications and data processing of the terminal device 100 by running the instructions stored in the internal memory 120. The internal memory 120 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required for a function (such as a sound playing function, an image playing function, etc.), and the like. The data storage area can store data created during the use of the terminal device 100 (such as audio data, a phone book, etc.), and the like. In addition, the internal memory 120 can include a high-speed random access memory, and can further include a non-volatile memory such as at least one of a magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like.

[0134] The internal memory 120 can be used to store software programs and modules, and the processor 110 performs various functional applications and data processing of the terminal device 100 by running the software programs and modules stored in the internal memory 120. The internal memory 120 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required for a function (such as a sound playing function, an image playing function, etc.), and the like; and the data storage area can store data (such as audio data, a phone book, etc.) created according to the use of the terminal device 100, and the like. In addition, the internal memory 120 can include a high-speed random access memory, and can further include a non-volatile memory such as at least one of a magnetic disk storage device, a flash memory device, or other volatile solid state storage devices.

[0135] (3) Audio module 130

[0136] The audio module 130 can provide an audio interface between the user and the terminal device 100, and generally includes a speaker 131, a microphone 132, a headphone interface 133, and a receiver 134, etc.

[0137] (4) I / O subsystem 140

[0138] The I / O subsystem 140 is an external device for controlling input and output, and can include an other device input controller 141, a display controller 142, and a sensor controller 143.

[0139] Optionally, one or more other input control device controllers receive signals from and / or send signals to other input devices 170, which can include physical buttons (push buttons, rocker buttons, etc.), dials, slide switches, joysticks, click wheels, optical mice (an optical mouse is a touch-sensitive surface that does not display visual output, or is an extension of a touch-sensitive surface formed by a touch screen), etc. Notably, the other input control device controllers 141 can be connected to any one or more of the above devices.

[0140] A display controller 142 in the I / O subsystem 140 receives signals from and / or sends signals to the display 160. Upon detecting user input, the display controller 142 converts the detected user input into a corresponding interaction with user interface objects displayed on the display 160, i.e., implements human-machine interactions.

[0141] A sensor controller 143 can receive signals from and / or send signals to one or more sensors.

[0142] (5) Sensor Module 150

[0143] The sensor module 150 can include optical sensors, motion sensors, and other sensors. Specifically, the sensor module 150 can include a pressure sensor 151, a gyro sensor 152, an acceleration sensor 153, a touch sensor 154. Although not shown in the figure, the sensor module 150 can also include a magnetic sensor, an air pressure sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, etc.

[0144] The pressure sensor 151 is configured to sense a pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 151 can be disposed on the display screen 160. The pressure sensor 151 can be of various types, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, etc. The capacitive pressure sensor can include at least two parallel plates of conductive material. When a force is applied to the pressure sensor 151, the capacitance between the electrodes changes. The terminal device 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation is applied to the display screen 160, the terminal device 100 detects the intensity of the touch operation based on the pressure sensor 151. The terminal device 100 can also calculate the position of the touch based on the detection signal of the pressure sensor 151. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation instructions. For example, when a touch operation with a touch operation intensity less than a first pressure threshold is applied to a short message application icon, an instruction to view short messages is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold is applied to the short message application icon, an instruction to create a new short message is executed.

[0145] The gyroscope sensor 152 can be configured to determine the motion attitude of the terminal device 100. In some embodiments, the angular velocity of the terminal device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 152. The gyroscope sensor 152 can be used for navigation, camera stabilization, and motion sensing game scenarios.

[0146] The acceleration sensor 153 can detect the magnitude of acceleration of the terminal device 100 in various directions (typically three axes). When the terminal device 100 is stationary, the magnitude and direction of gravity can be detected. The acceleration sensor 153 can also be used to identify the terminal device attitude, applied to landscape / portrait screen switching, pedometer, etc.

[0147] The touch sensor 154, also referred to as a "touch panel". The touch sensor 1510 can be disposed on the display screen 160, and the touch sensor 154 and the display screen 160 form a touch screen, also referred to as a "touch screen". The touch sensor 154 is configured to detect a touch operation applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operation to the application processor to determine the touch event type. Visual output related to the touch operation can be provided through the display screen 160. In other embodiments, the touch sensor 154 can also be disposed on the surface of the terminal device 100, which is different from the position of the display screen 160.

[0148] (6) Display screen 160

[0149] The display screen 160 is configured to display images, videos, and the like. The display screen 160 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diodes (QLED), or the like. In some embodiments, the terminal device 100 can include one or N display screens 160, where N is a positive integer greater than 1.

[0150] The display 160 can be used to display information input by a user or information provided to the user as well as various menus of the terminal device 100, and can also accept user input. In particular, the display 160 can include a display panel and a touch panel (not shown). The display panel can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like. The touch panel, also referred to as a touch screen, a touch-sensitive screen, or the like, can collect a touch or proximity operation (such as an operation of a user using a finger, a stylus, or any suitable object or accessory on or near the touch panel, which can also include a body-sensing operation; the operation includes a single-point control operation, a multi-point control operation, and the like) of a user on or near the touch panel, and drive a corresponding connection device according to a pre-set program. Optionally, the touch panel can include two parts, a touch detection device and a touch controller. The touch detection device detects a touch position and a gesture of a user, and detects a signal resulting from a touch operation, and transmits the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts the touch information into information that can be processed by the processor, and transmits the information to the processor 110, and can also receive a command from the processor 110 and execute the command. In addition, the touch panel can be implemented in various types such as a resistive type, a capacitive type, an infrared type, and a surface acoustic wave type, or any technology developed in the future. Further, the touch panel can cover the display panel, and a user can operate on or near the touch panel covering the display panel according to the content (including but not limited to a soft keyboard, a virtual mouse, a virtual key, an icon, and the like) displayed on the display panel. After the touch panel detects an operation on or near the touch panel, the operation is transmitted to the processor 110 via the I / O subsystem to determine user input, and then the processor 110 provides corresponding visual output on the display panel via the I / O subsystem according to the user input.

[0151] (7) Other input device 170

[0152] The other input device 170 can be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the terminal device 100. In particular, the other input device 170 can include one or more of a physical keyboard, function keys (such as a volume control button, an on / off button, and the like), a trackball, a mouse, a joystick, an optical mouse (an optical mouse is a touch-sensitive surface that does not display visual output, or an extension of a touch-sensitive surface formed by a touch screen), and the like. The other input device 170 is connected to the other input device controller 141 of the I / O subsystem 140, and performs signal interaction with the processor 110 under the control of the other input device controller 141.

[0153] (8) Other

[0154] The terminal device 100 further includes a communication module 180 and a battery module 190.

[0155] In addition, although not shown, the terminal device 100 can further include a camera, a Bluetooth module, and the like, which will not be described herein.

[0156] The software system of the electronic device 100 can employ a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. Embodiments of the present application take the Android system with a layered architecture as an example to exemplarily illustrate the software structure of the electronic device 100.

[0157] Figure 2 is a software structure block diagram of the electronic device 100 of embodiments of the present application. The layered architecture divides the software into several layers, each of which has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, an application layer, an application framework layer, an Android runtime and system library, and a kernel layer. The application layer can include a series of application packages.

[0158] As shown in Figure 2 , the application layer can include a camera, settings, a skin module, a user interface (UI), a third-party application, and the like. Among them, the third-party application can include a gallery, a calendar, a call, a map, navigation, WLAN, Bluetooth, music, video, short message, and the like.

[0159] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications of the application layer. The application framework layer can include some pre-defined functions.

[0160] As shown in Figure 2 , the application framework layer can include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0161] The window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, and intercept the screen, and the like. The content provider is used to store and obtain data, and make the data accessible to the applications. The data can include videos, images, audios, dialed and received calls, browsing history and bookmarks, phone books, and the like.

[0162] The view system includes visual controls, such as controls that display text, controls that display pictures, and the like, such as the indication information used to prompt the virtual shutter key in the embodiments of the present application. The view system can be used to build an application. A display interface can be composed of one or more views. For example, a display interface that includes a short message notification icon can include a view that displays text and a view that displays pictures.

[0163] The phone manager is used to provide the communication function of the electronic device 100. For example, the management of the call state (including call connection, call hang-up, and the like).

[0164] The resource manager provides various resources for an application, such as localized strings, icons, pictures, layout files, video files, and the like.

[0165] The notification manager enables an application to display notification information in the status bar, which can be used to convey a message of the notification type and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify the completion of a download, a message reminder, and the like. The notification manager can also be a notification that appears in the form of a chart or a scroll bar text in the top status bar of the system, such as a notification of an application running in the background, and can also be a notification that appears in the form of a dialogue window on the screen. For example, the display of text information in the status bar, the emission of a prompt sound, the vibration of the electronic device, the flickering of an indicator light, and the like.

[0166] The Android runtime includes a core library and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0167] The core library includes two parts: one part is a function function that needs to be called by the java language, and the other part is the core library of Android.

[0168] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer into binary files. The virtual machine is used to perform the management of the object life cycle, the management of the stack, the management of the thread, the management of security and exceptions, and the functions of garbage collection, and the like.

[0169] The system library can include multiple functional modules. For example: a surface manager, media libraries, a three-dimensional graphics processing library (for example: OpenGL ES), a 2D graphics engine (for example: SGL), and the like.

[0170] The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.

[0171] The media library supports a variety of commonly used audio, video format playback and recording, and static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0172] The three-dimensional graphics processing library is used to implement three-dimensional graphics drawing, image rendering, synthesis, and layer processing, etc.

[0173] The 2D graphics engine is a drawing engine for 2D drawing.

[0174] In addition, the system library can also include a state monitoring service module, etc., such as a physical state identification module, for analyzing and identifying user gestures; a sensor service module for monitoring sensor data uploaded by various sensors of the hardware layer to determine the physical state of the electronic device 100.

[0175] The kernel layer is the layer between hardware and software. The kernel layer at least contains display drivers, camera drivers, audio drivers, and sensor drivers.

[0176] The hardware layer can include various sensors, such as Figure 1 The various sensors introduced in the foregoing, the acceleration sensor, the gyroscope sensor, the touch sensor, etc. involved in the embodiments of the present application. In combination with Figure 1 and Figure 2 The current method of screen pressure sensing control game is described. Generally, the game interface provides one or more game buttons for executing corresponding instructions in response to receiving user touch operations. Specifically, the touch screen of the terminal device 100 receives a touch event and reports the touch event to the input management module of the terminal device 100, and then the input management module of the terminal device 100 forwards the event to the corresponding running game APP; at the same time, the input management module forwards the event to the corresponding UI interface, so that the corresponding button icon can change, thereby prompting the user to control the button.

[0177] After the user enters the game program, the game interface displays screen buttons such as button 1, button 2, button 3, … When the user touches the screen button, the touch screen of the terminal device detects the user's touch or drag action on the screen button and executes the corresponding instruction, and displays the corresponding effect on the screen. Since there can be multiple buttons on the screen of some games, the user needs to perform a large number of operations in a short time, and this control method can make the player have to be very familiar with the positions of the buttons and accurately click different buttons, which can easily lead to user misoperation or difficulty for the user to improve his operation speed. In addition, in some cases, multiple buttons on the game interface occupy a certain percentage of the screen, which can affect the user's observation of the game picture.

[0178] Figure 3 A flow chart of a game control method 200 provided by an embodiment of the present application is shown. For example, the method 200 can be applied to the terminal device 100 described above.

[0179] As shown in S210, according to the first attitude change data of the terminal device and the first mapping relationship, a first key is determined, the first mapping relationship including the corresponding relationship between the first attitude change data and the first key. Figure 3

[0180] In some embodiments, before step S210, the terminal device 100 first acquires sensor data through a sensor, and judges whether the data sent by the sensor reaches the first attitude change data.

[0181] Firstly, the first key is explained. Optionally, the first key includes a key occupying a certain pixel area and having a certain shape at any position of the screen, for receiving the touch of a user and converting it into a corresponding instruction. Optionally, the first key includes a game skill key, which can receive the touch control of a user to release a corresponding skill. The first key can also include a key for opening an interface, which can receive the touch control of a user to open a corresponding interface. The first key can also include a wheel key of a game. Exemplarily, the wheel key generally includes left, right, up and down keys. After the wheel key is touched, a corresponding game control moves a certain unit of distance in a corresponding direction. Optionally, the first key can also include a mechanical key, for receiving the pressing or touch of a user to make the terminal execute a corresponding instruction. Exemplarily, the first key includes a volume key, for receiving the pressing of a user to adjust the volume. Exemplarily, the first key can also include a lock screen key, for receiving the pressing of a user to open a lock screen interface.

[0182] The first attitude change data of the terminal device is explained. Data capable of corresponding to a single auxiliary operation of a user rotating the terminal device or a combination of multiple auxiliary operations of the user rotating the terminal device can be regarded as the first attitude change data. The single auxiliary operation can be a rotating action around an x axis (parallel to the bottom side of the terminal device), a y axis (parallel to the side of the terminal device) or a z axis (perpendicular to the terminal device). The combination of the multiple auxiliary operations can be an action of rotating multiple times in the same rotating direction.

[0183] Figure 4A An illustration of the single auxiliary operation provided by an embodiment of the present application Figure 1 .

[0184] Figure 4B An illustration of the single auxiliary operation provided by an embodiment of the present application Figure 2 . ​

[0185] Figure 4C A schematic diagram of a single auxiliary operation provided in the embodiments of this application. Figure 3 .

[0186] Combination Figures 4A to 4C The above auxiliary operations will be explained. For example, as follows: Figure 4A As shown, the user-assisted operations include auxiliary operation U1 (rotating counterclockwise around the y-axis once) and auxiliary operation D1 (rotating clockwise around the y-axis once), as follows: Figure 4B As shown, auxiliary operation L1 (rotates counterclockwise around the x-axis once), and auxiliary operation R1 (rotates clockwise around the x-axis once), as follows: Figure 4C As shown, auxiliary operation P1 (rotating counterclockwise around the z-axis once) and auxiliary operation T1 (rotating clockwise around the z-axis once) are examples. Optionally, the user's auxiliary operation can also be a combination of multiple of the above auxiliary operations, such as auxiliary operation U2 (containing two auxiliary operations U1), auxiliary operation D2 (containing two auxiliary operations D1), auxiliary operation L2 (containing two auxiliary operations L1), auxiliary operation R2 (containing two auxiliary operations R1), auxiliary operation P2 (containing two auxiliary operations P1), and auxiliary operation T2 (containing two auxiliary operations T1). Optionally, the number of auxiliary operations included can also be three or more, and this application does not impose any limitation in this regard. It should be understood that the above-described auxiliary operations are designed to facilitate user operation by simultaneously touching buttons and rotating the terminal device.

[0187] It should be understood that the terminal device can determine whether the user's operation corresponds to the above-mentioned auxiliary operation through various attitude change data, such as angular velocity, acceleration, angular acceleration, linear velocity, etc., and this application does not make any limitation in this regard.

[0188] The following explanation uses the attitude change data of the terminal device, including acceleration and angular velocity, as an example.

[0189] For simplicity, the following description assumes the terminal device is in landscape orientation with the top camera facing the user's left. This application does not specify the sign of the clockwise rotation angle of the terminal device under different landscape orientations. When the terminal device's landscape orientation is opposite to the preset orientation, the sign of the rotation value is reversed.

[0190] For example, the y-axis gyroscope sensor can detect the angular velocity λ of the terminal device rotating around the y-axis (parallel to the side of the terminal device). When the rotating direction is counterclockwise around the y-axis, λ is greater than 0; when the rotating direction is clockwise around the y-axis, λ is less than 0. The x-axis gyroscope sensor can detect the angular velocity β of the terminal device rotating around the x-axis (parallel to the bottom of the terminal device). When the rotating direction is counterclockwise around the y-axis, β is greater than 0; when the rotating direction is clockwise around the y-axis, β is less than 0. The z-axis gyroscope sensor can detect the angular velocity α of the terminal device rotating around the z-axis (perpendicular to the front of the terminal device). When the rotating direction is counterclockwise around the z-axis, α is greater than 0; when the rotating direction is clockwise around the z-axis, α is less than 0. The values of the above angular velocities can be used to determine the magnitude of the rotating motion and the direction of the terminal device motion.

[0191] In addition to the gyroscope sensor, the terminal device can also obtain the acceleration of the terminal device in the x-axis, y-axis and z-axis through the acceleration sensor. When the terminal device moves according to the above auxiliary operation, the terminal device will also generate acceleration components in the x-axis, y-axis and z-axis, and the terminal device can determine the rotating direction and the rotating magnitude through the acceleration values. For example, the y-axis gyroscope sensor can detect the acceleration ay of the terminal device in the y-axis (parallel to the side of the terminal device), the x-axis gyroscope sensor can detect the acceleration ax of the terminal device in the x-axis (parallel to the bottom of the terminal device), and the z-axis gyroscope sensor can detect the acceleration az of the terminal device in the z-axis (perpendicular to the front of the terminal device).

[0192] The "determining the first key according to the first attitude change data and the first mapping relationship" is described.

[0193] For example, when the terminal device rotates counterclockwise around the x-axis, β is the angular velocity change around the x-axis and the value is positive. At this time, the terminal device will also generate corresponding accelerations in the y-axis and z-axis, and the terminal device can obtain these data through the motion sensor, for example, the terminal device can obtain the angular velocity of the x-axis through the x-axis gyroscope sensor, and obtain the acceleration of the y-axis through the y-axis acceleration sensor, and obtain the acceleration of the z-axis through the z-axis acceleration sensor. In this way, the terminal device can obtain the attitude change data of itself, and determine the first key according to the corresponding relationship between the attitude change data and the first key. It should be understood that the first terminal device attitude change set by the embodiment of the present application is associated with the first key, which is equivalent to that the first key is associated with the above auxiliary operation.

[0194] Case 1

[0195] The first attitude change data includes a first speed of the terminal device moving in a first direction. When the first speed is greater than or equal to a first threshold, the first key is determined according to the first attitude change data and the first mapping relationship. At this time, the first attitude change data corresponds to the single auxiliary operation, for example, the auxiliary operation D1.

[0196] The first direction is explained. The first direction can be determined by the sign of the first speed and the type of the first speed. For example, when λ is greater than 0, the moving direction is counterclockwise around the y axis.

[0197] Optionally, the first direction is the direction of the angular speed, and the first speed is the absolute value of the angular speed. For example, when λ is greater than 0 and the absolute value of λ is greater than or equal to a corresponding first threshold λ1, the first key is determined to be the key 1 according to the first mapping relationship (Table 1). The first attitude change data also corresponds to the auxiliary operation D1, that is, the user can realize the instruction of the key 1 through the rotating action around the y axis.

[0198] The first threshold is explained. Optionally, the first threshold is a relatively large value, for example, 5 rad / 50 ms or 5 rad / 100 ms. The relatively large first threshold is set to avoid the "misjudgment" caused by other actions of the user (for example, the user changes the posture of the body, and the posture of the terminal device also changes). Optionally, the values of the first thresholds corresponding to different motions can be the same or different, which is not limited in the present application.

[0199] Table 1

[0200]

[0201] Optionally, the first direction is the direction of the acceleration, and the first speed is the absolute value of the acceleration. For example, when ax is greater than 0 and the absolute value of ax is greater than or equal to a corresponding first threshold ax1, the first key is determined to be the key 1 according to the first mapping relationship (Table 2). The first attitude change data also corresponds to the auxiliary operation D1, that is, the user can realize the instruction of the key 1 through the rotating action around the y axis.

[0202] Table 2

[0203]

[0204] Optionally, the first direction is a direction of angular velocity, and the first speed is an absolute value of acceleration. For example, when the terminal device rotates counterclockwise around the y axis, the angular velocity of the y axis is greater than 0, and a larger acceleration component is generated on the x axis, the terminal device can determine the direction of rotation by combining the positive and negative of the angular velocity of the y axis, and determine the amplitude of rotation by the absolute value of the acceleration of the x axis. For example, the terminal device detects that λ is positive, determines that the terminal device is operated with a motion of rotating counterclockwise around the y axis, and combines the absolute value of ax greater than or equal to ax1, the terminal device determines that the amplitude of the rotation operation is greater than a preset value, and the time interval between two detections should be less than a first time, which is a relatively small time interval, so as to determine the same operation. For example, the first time is 200 ms, or the first time is 500 ms, according to the first mapping relationship (Table 3), the terminal device determines that the corresponding first key is key 1.

[0205] Table 3

[0206]

[0207] Optionally, the first direction is a direction of angular velocity, and the first speed includes acceleration and angular velocity. For example, the terminal device detects that λ is positive and greater than or equal to λ1, and the absolute value of ax is greater than or equal to ax1, according to the first determination relationship (Table 4), the terminal device determines that the first attitude change data corresponds to key 1. In order to ensure that the terminal device detects the same operation, the time interval between the above two detections should be less than a first time. The first time can be a relatively small value, for example, 10 ms.

[0208] Table 4

[0209]

[0210] Optionally, the first direction includes the direction of acceleration and the direction of angular velocity, and the first speed includes acceleration and angular velocity. For example, λ and ax are both positive, and the absolute values of λ and ax are greater than or equal to the corresponding first threshold value, and the terminal device can determine the first key as key 1 according to the first mapping relationship (Table 5).

[0211] Table 5

[0212]

[0213] Case 2

[0214] Optionally, the first attitude change data includes a second speed when the terminal device moves for the first time in a second direction and a third speed when the terminal device moves for the second time in the second direction, and the determining the first key according to the first attitude change data of the terminal device and the first mapping relationship further includes:

[0215] when a time interval between the starting moment of the first movement in the second direction and the starting moment of the second movement in the second direction is within a preset range, the second speed is greater than or equal to a second threshold value, and the third speed is greater than or equal to a third threshold value, the first key is determined according to the first attitude change data and the first mapping relationship.

[0216] Optionally, the second direction is as described above for the first direction, which will not be repeated here. The second speed and the third speed are as described above for the first speed, which will not be repeated here.

[0217] Optionally, the first attitude change data includes the second speed when the terminal device is moved in the second direction for the first time and the third speed when the terminal device is moved in the second direction for the second time, and further includes a fourth speed when the terminal device is moved in a third direction for the first time, the second speed is greater than or equal to a second threshold value, the third speed is greater than or equal to a third threshold value, the third direction is opposite to the second direction, or the fourth speed is 0, and the first key is determined according to the first attitude change data and the first mapping relationship.

[0218] It should be understood that the movement in the third direction is a stop or reset movement between the two movements, and the corresponding user action is to rotate the terminal device in the second direction, pause for a period of time, or rotate the terminal device in the opposite direction, and then rotate the terminal device in the second direction again. The terminal device detects the fourth speed to avoid the terminal device being unable to distinguish between the first operation and the second operation.

[0219] Optionally, the second threshold value and the third threshold value can be different, for example, the second threshold value is 3 rad / 50 ms, and the third threshold value is 2 rad / 50 ms.

[0220] Optionally, the auxiliary operation D3, the auxiliary operation L3, and the like can be similarly set.

[0221] Optionally, the first mapping relationship can be preset or user-defined.

[0222] Optionally, when the terminal device is a device with a flexible screen, the same operation of a user can correspond to two or more auxiliary operations, and then the terminal device executes the instructions associated with the first keys corresponding to the auxiliary operations. For example, the terminal device simultaneously detects the first attitude change data corresponding to the auxiliary operation L1 and the auxiliary operation U1, and then the terminal device simultaneously executes the instructions of the first keys corresponding to the auxiliary operation L1 and the instructions of the first keys corresponding to the auxiliary operation U1.

[0223] It should be understood that the same first button is set to correspond to different first terminal posture change data, that is, the same first button is set to correspond to different auxiliary operations, and the instruction corresponding to the first button has multiple triggering modes. For example, button 1 corresponds to auxiliary operation U1 and auxiliary operation L1. When the terminal device detects the first posture change data corresponding to the auxiliary operation U1, the instruction corresponding to the button 1 is executed. When the terminal device detects the first posture change data corresponding to the auxiliary operation L1, the instruction corresponding to the button 1 is executed. In other words, touching the button 1 or performing the operation corresponding to the auxiliary operation L1 or the operation corresponding to the auxiliary operation U1 can make the terminal execute the instruction corresponding to the button 1. S220, execute the instruction associated with the first button.

[0224] For example, the terminal device determines that the operation of the user corresponds to the button 1, and executes the instruction associated with the button 1.

[0225] When the user plays the game, the user needs to complete the input of multiple instructions in a short time. The method of rotating the terminal device to issue the instruction can be used to operate the game, instead of needing to accurately click the corresponding multiple buttons on the screen with the fingers, thereby improving the operation accuracy of the user, and also helping the user to realize multi-point simultaneous operation and improve the operation speed, thereby improving the user experience. Alternatively, some button positions are difficult to reach, and the method provided in the embodiments of the present application can facilitate user operation.

[0226] Optionally, the first posture change data can also be associated with a first icon, and the first icon has a corresponding relationship with the position of the first button. In other words, the first posture change data has a second mapping relationship with the first icon, and the first button has a third mapping relationship with the first icon. Optionally, the second mapping relationship is pre-set. For example, the first icon includes icon U1 and icon D2, wherein the icon U1 is associated with the first posture change data corresponding to the auxiliary operation U1, and the icon D2 is associated with the first posture change data corresponding to the auxiliary operation D2.

[0227] Optionally, the first icon is displayed on a first interface, and the first interface further includes the first button corresponding to the first icon. It should be understood that the first icon, the first posture change data, and the first button one-to-one correspond.

[0228] Optionally, the position of the first button and the position of the first icon have a corresponding relationship.

[0229] Optionally, the area of the region where the first icon is located is smaller than the area of the region where the first button is located, and the display region of the first icon can be located in the display region of the first button. Alternatively, the display region of the first icon and the display region of the first button partially overlap.

[0230] Optionally, the transparency of the first icon is greater than or equal to a preset value.

[0231] Optionally, the third mapping relationship is preset or user-defined, and the second mapping relationship is fixed. It should be understood that the first mapping relationship is changed when the third mapping relationship is changed. For example, the first key includes key 1, the first icon includes icon U1, and a third mapping relationship between key 1 and icon U1 is newly established. Since icon U1 is associated with the first terminal device posture change data corresponding to the auxiliary operation U1, the first terminal device posture change data corresponding to key 1 and the auxiliary operation U1 also form a corresponding relationship, that is, the first mapping relationship. For details of the method for changing the third mapping relationship, see Figure 5 and related descriptions.

[0232] Figure 6A the game interface before the instruction corresponding to key 1 is executed, Figure 6B and Figure 6C the game interface after the instruction corresponding to key 1 is executed.

[0233] Optionally, after step 220, the shape of the first icon is not updated in response to the execution of the instruction. For example, as shown in Figure 6A , the trees and grasses are randomly arranged before the instruction corresponding to key 1 is executed, and the trees and grasses are arranged in order after key 1 is touched or the auxiliary operation corresponding to key 1 is detected, as shown in Figure 6B .

[0234] Optionally, after step 220, the shape of the first icon is updated in response to the execution of the instruction. For example, as shown in Figure 6C , the trees and grasses are arranged in order, and the shape of icon U1 corresponding to key 1 is updated in response to the instruction corresponding to key 1, so that the user can more intuitively understand the operation performed by the user.

[0235] Optionally, updating the shape of the first icon can be that the first icon dynamically flashes for a period of time. For example, the period of time can be 1 second.

[0236] Optionally, updating the shape of the first icon can be that the first icon becomes darker or lighter.

[0237] Figure 5 a flowchart of another game control method 400 provided by the embodiment of the application.

[0238] In combination with Figure 5 the third mapping relationship is user-defined and the control method of the embodiment of the application is used.

[0239] S410, a first icon is associated with a first key.

[0240] This step is described in detail in combination with FIG. 7-FIG. 12.

[0241] FIG. 7 shows a set of graphical user interfaces (GUIs) of the terminal device, wherein, from Figure 7A to FIG. 7 shows the steps of the user entering the game interface. Figure 7A The main desktop of the terminal device includes one or more icons of application (APP). The icons can include an icon of clock, an icon of calculator, an icon of weather, an icon of music, an icon of photo album and an icon of game 610. When the terminal device detects the operation of the user clicking the icon 610, the GUI as shown in Figure 7B can be displayed.

[0242] Referring to the GUI as shown in Figure 7B , which is a loading interface of the game. The loading interface includes a control icon 620 for opening the game setting interface, and a “start game” icon for entering the game battle interface.

[0243] Figures 8A to 8E The steps of enabling the auxiliary operation are shown.

[0244] Optionally, the terminal device detects the operation of the user clicking the icon 620, and enters the game setting interface as shown in Figure 8A , which includes a plurality of setting options and a plurality of settings, and an icon 710 of “auxiliary operation setting”. When the terminal detects that the icon 710 is clicked, the interface as shown in Figure 8B can be displayed. Figure 8B , which shows a schematic diagram of an auxiliary operation setting interface, such as Figure 8BAs shown, the enabled first gesture change data includes first gesture change data corresponding to auxiliary operation U1, auxiliary operation D1, auxiliary operation R2 and auxiliary operation L2. When the user clicks the "auxiliary operation switch", the button corresponding to the "auxiliary operation switch" displays "ON", and in the game APP, the first gesture change data corresponding to the auxiliary operation is in an available state. Alternatively, when the terminal device detects that the user clicks the "auxiliary operation switch" again, the button corresponding to the "auxiliary operation switch" displays "OFF", and in the game APP, the first gesture change data corresponding to the auxiliary operation is in an unavailable state. The initial state of the preset "auxiliary operation switch" is not limited to be on or off in the present application. It should be understood that at this time, the auxiliary operations corresponding to the first gesture change data available in the APP are auxiliary operation U1, auxiliary operation D1, auxiliary operation R2 and auxiliary operation L2, and this setting cannot be changed, i.e., the user cannot add other auxiliary operations or delete any of the above auxiliary operations.

[0245] Alternatively, the first icon corresponding to the auxiliary operation is not mapped to the first key, and the user needs to establish a third mapping relationship between the first icon and the first key so that the first gesture change data is mapped to the first key.

[0246] Alternatively, after the terminal detects that the icon 710 is clicked, an interface as shown in Figure 8C may be displayed, Figure 8C which shows a schematic diagram of an auxiliary operation setting interface, as shown in Figure 8C As shown, the available first gesture change data includes first gesture change data corresponding to auxiliary operation U1, auxiliary operation D1, auxiliary operation R2 and auxiliary operation L2. The icon corresponding to the auxiliary operation can be touched and selected to be enabled. For example, the terminal device detects that the auxiliary operation icon U1 is touched, and the shape of the icon is updated to prompt the user that the auxiliary operation U1 corresponding to the icon is enabled. Alternatively, after the icon is clicked again, the auxiliary operation U1 corresponding to the icon is in an unavailable state.

[0247] Alternatively, the first icon corresponding to the enabled auxiliary operation is not mapped to the first key, and the user needs to establish a third mapping relationship between the first icon and the first key so that the first gesture change data is mapped to the first key.

[0248] Alternatively, after the terminal device detects that the icon 710 is clicked, an interface as shown in Figure 8D may be displayed, Figure 8D which shows a schematic diagram of an auxiliary operation setting interface, as shown in Figure 8DAs shown, exemplary, the available first gesture change data includes first gesture change data corresponding to auxiliary operation U1, auxiliary operation D1, auxiliary operation R2 and auxiliary operation L2, the icons corresponding to the auxiliary operations can be touched to be selected and enabled, for example, the terminal device detects that the auxiliary operation icon U1 is touched, the icon shape is updated to prompt the user that the auxiliary operation U1 corresponding to the icon has been enabled, optionally, after the icon is clicked again, the auxiliary operation U1 corresponding to the icon is in an unavailable state. Optionally, the above-mentioned auxiliary operations can be used in different interfaces. As Figure 8D As shown, exemplary, the auxiliary operation U1 is selected to interface 1, the auxiliary operation D1, the auxiliary operation R2 and the auxiliary operation L2 are selected to interface 2.

[0249] Optionally, the first icon corresponding to the enabled auxiliary operation is not mapped with the first key, and the user needs to establish a third mapping relationship between the first icon and the first key so as to form a mapping relationship between the first gesture change data and the first key.

[0250] Fig. 9 and Fig. 10 show two different methods of setting the first icon corresponding to the first key.

[0251] The "setting the first icon corresponding to the first key" is described in combination with Fig. 9 and Fig. 10.

[0252] Optionally, the terminal device detects that the icon 750 is clicked, and displays an interface as shown in Figure 8E . Figure 8E A schematic diagram of selecting a modification interface is shown. As shown in Figure 8E , exemplary, the interface includes the key "interface 1", the key "interface 2", the key "interface 3" and the key "interface 4", the interface 1 is a setting interface corresponding to a game interface, for example, the terminal device detects that the key "interface 1" is clicked, and displays an interface as shown in Figure 9A or Figure 10A .

[0253] Optionally, the terminal device detects that the icon 740 is clicked, and the icon 750 is clicked, and also displays an interface as shown in Figure 8C or Figure 8D . Figure 9A or Figure 10A .

[0254] Figure 9A A schematic diagram of an auxiliary operation mapping setting interface is shown. As shown in Figure 9A , the icon U1 appears at the position corresponding to the key 1, at this time, the default icon U1 has formed a third mapping relationship with the key 1. When the terminal device detects that the icon U1 is clicked or pressed, the icon U1 can be moved. As shown in Figure 9BAs shown, icon U1 is moved along the illustrated path, and when icon U1 is moved away from the position area corresponding to key 1, the third mapping relationship between icon U1 and key 1 is terminated. Alternatively, when icon U1 is moved away from the position area corresponding to key 1, the appearance of icon U1 is updated to prompt the user that the position of icon U1 no longer corresponds to the position of key 1, and at this time, the state of icon U1 changes to connectable, as shown in FIG. 10B. Figure 9C As shown, when icon U1 is moved to the position area corresponding to key 3, the appearance of icon U1 is updated to prompt the user that the position of icon U1 corresponds to the position of key 3, and after the terminal device detects that the user releases the hand, the third mapping relationship between icon U1 and key 3 is established. At this time, the first gesture change data corresponding to icon U1 is associated with key 3.

[0255] Figure 10A Another example of a screen interface for setting the mapping of auxiliary operations is shown. Figure 10A As shown, icon U1 appears at any position on the screen and is in a connectable state, i.e., is not associated with any key. When the terminal device detects that icon U1 is clicked or pressed, icon U1 can be moved, as shown in FIG. 10A. Figure 10B As shown, icon U1 is moved along the illustrated path, and when icon U1 is moved away from the position area corresponding to key 1, the third mapping relationship between icon U1 and key 1 is terminated. Alternatively, when icon U1 is moved away from the position area corresponding to key 1, the appearance of icon U1 is updated to prompt the user that the position of icon U1 no longer corresponds to the position of key 1, and at this time, the state of icon U1 changes to connectable, as shown in FIG. 10B. Figure 10C As shown, when icon U1 is moved to the position area corresponding to key 3, the appearance of icon U1 is updated to prompt the user that the position of icon U1 corresponds to the position of key 3, and after the terminal device detects that the user releases the hand, the third mapping relationship between icon U1 and key 3 is established.

[0256] Alternatively, when a first key has been associated with a first icon, if the user wants to move another first icon to the position area corresponding to the first key, the system reports an error. Although the position of the other first icon corresponds to the position of the first key, the appearance of the other first icon is not updated to prompt the user that the other first icon cannot correspond to the first key.

[0257] Alternatively, when a first key has been associated with a first icon, if the user wants to move another first icon to the position area corresponding to the first key, the other first icon is also associated with the first key. It should be understood that at this time, the first key corresponds to two auxiliary operations, in other words, the first key corresponds to two first gesture change data.

[0258] FIG. 11 shows a set of screen interfaces for setting the association between a first key and a first icon.

[0259] Figure 11A An example of a screen interface for setting the mapping of auxiliary operations is shown.

[0260] Figure 11B Another example of a screen interface for setting the mapping of auxiliary operations is shown.

[0261] Figure 11C Another schematic diagram of an interface for setting up auxiliary operation mapping is shown.

[0262] The first button associated with the first icon will be explained with reference to Figure 11.

[0263] Optionally, the first button can be a single button. For example, such as... Figure 11A As shown, the enabled auxiliary operations include auxiliary operation D1, auxiliary operation R1, auxiliary operation L1 and auxiliary operation U1. Optionally, the default positions of the above auxiliary operations correspond to the positions of the four buttons on the steering wheel, that is, the above auxiliary operations have a third mapping relationship with the four directional buttons (up, down, left and right) by default.

[0264] Optionally, the first button can be multiple buttons, and these multiple buttons are interconnected. For example, such as... Figure 11B As shown, this interface can be detected by the terminal device. Figure 8E The "Interface 2" button shown is pressed to display the game settings interface. Figure 7B The mapping settings interface. For example, Figure 7B The settings interface has settings options 1, 2, and 3 linked together. For example, if setting option 1 is on by default, its corresponding setting is also enabled. When the terminal device detects that the user clicks the toggle switch for setting option 2, setting option 1 is automatically disabled, and setting option 2 is enabled. If the terminal device detects that the user clicks the toggle switch for setting option 3, setting option 2 is automatically disabled, and setting option 3 is enabled. When this combination of settings options forms a third mapping with icon U1—for example, placing icon U1 in the position corresponding to any of the options in the combination—a third mapping relationship is formed between the combination of settings options and icon U1. For instance, the terminal device's default initial setting is the setting for setting option 1 or the previously retained setting. When the terminal device detects a first posture change data for the corresponding auxiliary operation U1, it changes the enabled setting for this combination to the setting for setting option 2. When the terminal device detects another first posture change data for the corresponding auxiliary operation U1, it changes the enabled setting for this combination to the setting for setting option 3.

[0265] Optionally, the first button can be multiple buttons, and these multiple buttons are independent of each other. For example, such as... Figure 11C As shown, the first button includes button 3, button 4, and button 5, all corresponding to icon U1. One method to set button 3, button 4, and button 5 to simultaneously correspond to icon U1 is to display on the screen as shown... Figure 9AAlternatively, in the 10A interface, double-clicking the first icon copies it. The copied icon is displayed in the original icon's position, and its layer is above the original icon's layer. When the user moves the copied icon, the original icon remains in its original position. It should be understood that when the user moves the first icon, the topmost icon on the layer is moved first. Alternatively, setting buttons 3, 4, and 5 to simultaneously correspond to icon U1 can also be achieved by long-pressing and dragging the first icon to copy it. For example, the long press duration is 5 seconds. After copying icon U1, the user can use the methods described in Figures 9 and 10 to move multiple icons U1 to the corresponding positions of buttons 3, 4, and 5 respectively.

[0266] Figure 12 illustrates another method for initiating auxiliary operations.

[0267] Optionally, users can activate auxiliary operations on the game battle interface or the game loading screen.

[0268] Optionally, the game loading screen includes an icon to enable / disable assistive functions, allowing users to choose whether to activate assistive functions in that match. For example... Figure 12A As shown, after the terminal device detects that the user clicks on icon 1110, it activates the corresponding pre-set auxiliary operation during the game.

[0269] Optionally, the icon for enabling / disabling accessibility can also be displayed as follows: Figure 12B The game battle interface shown.

[0270] Optionally, when accessibility is enabled, the appearance of the "Enable / Disable Accessibility" icon differs from its appearance when accessibility is disabled.

[0271] Optionally, the terminal device has accessibility enabled by default, and only displays a first prompt message (not shown in the figure, such as "Accessibility enabled"), which can appear on the game loading screen or the game battle screen. Optionally, the first prompt message disappears after flashing for 5 seconds. Optionally, the first prompt message can also remain permanently.

[0272] Optionally, after the "Enable / Disable Assistive Operations" icon is turned on, not only is the first posture change data corresponding to the assisted operation enabled, but the first icon is also movable, thereby changing the third mapping relationship, that is, changing the mapping relationship between the first button and the first icon. At this time, a second prompt message can also appear to inform the user that the mapping between the first icon and the first button can be changed, such as... Figure 12C As shown, the interface displays a second prompt: "Move the icon to change the mapping."

[0273] Optionally, when the terminal device's default accessibility is enabled, the first icon is movable, thereby changing the third mapping relationship, i.e., changing the mapping relationship between the first button and the first icon. In this case, a second prompt message can appear to inform the user that the mapping between the first icon and the first button can be changed, such as... Figure 12C As shown, the interface displays a second prompt: "Move the icon to change the mapping."

[0274] Optional, such as Figure 12D As shown, a toggle icon for the first attitude data, which is also the toggle icon for the auxiliary operation, can be added to the control bar. When the icon is clicked and displayed in a dark color, it indicates that the auxiliary operation is on; clicking it again will turn the icon light-colored, indicating that the auxiliary operation is off.

[0275] In some embodiments, the floating window may also display the above icons and have the same functionality.

[0276] This makes it easier for users to choose to turn auxiliary operations on or off.

[0277] Optionally, users can still access the settings interface by clicking icon 620 in the game battle interface or game loading interface, and change the mapping relationship between the first icon and the first button as shown in Figures 8-10.

[0278] Optionally, the mapping between the first icon and the first button can be pre-set by the service provider. This way, users can directly use the default settings after entering the game app.

[0279] Optionally, the first icon may not be displayed on the game interface. For example, after setting the relationship between the first button and the first icon in the settings interface, the user chooses not to display auxiliary operations. When the user enters the game interface, the first icon is not displayed. It should be understood that the first button corresponding to the first icon means that the first button is associated with the first posture change data. Although the first icon is hidden, the user can still operate according to the relationship between the first button and the first icon that they set.

[0280] S420, determine the first icon based on the first posture change data.

[0281] For example, the terminal detects the first terminal posture change data corresponding to the auxiliary operation U1, and determines the first icon as icon U1 based on the icon U1 corresponding to the first terminal posture change data of the auxiliary operation U1.

[0282] Since icon U1 corresponds to button 1 (which is set in S410), the first button is determined to be button 1.

[0283] S430, execute the instruction associated with the first button.

[0284] The description of this step is similar to that of S220, which will not be repeated here.

[0285] Figure 13 An example of a scene for customizing the "first gesture change data".

[0286] In combination with Figure 13 The step of customizing the "first gesture change data" by the user is described. It should be understood that customizing the "first gesture change data" is equivalent to customizing the auxiliary operation. For example, the terminal device detects that the user clicks Figure 8D After the user customizes 730 as shown, the terminal device displays a user customization interface as shown in Figure 13 The interface includes an example action image or animation, a drop-down menu 1260 for customizing the auxiliary operation, a start button 1270, and a confirm button 1280. After the terminal device detects that the user selects through the drop-down menu 1260, it displays the corresponding image or animation to prompt the user how to operate the terminal device. After the terminal device detects that the user clicks the button 1270, it reads the gesture change data of the terminal device until it detects that the user clicks the button 1280, stops reading the gesture change data of the terminal device, and calculates the first threshold value, the second threshold value, the preset time, and other parameters according to the gesture change data.

[0287] Optionally, the button 1280, i.e., the confirm button, can be omitted. For example, the terminal device displays countdown information to prompt the user of the time when the operation can be performed. The countdown time can be preset, for example, 10 seconds.

[0288] If the terminal device detects that the user's action is beyond the range allowed by the system or the difference from the data of the previous action is greater than a preset value, it prompts the user to re-operate.

[0289] By customizing the above-mentioned first gesture change data, the user can change the settings to adapt to personal operation habits or hardware conditions (for example, different sizes of different terminal devices or different sensor conditions of different terminal devices).

[0290] In combination with Figure 14 and Figure 15 The specific implementation of the method 300 and the method 400 introduced in the above embodiments and the execution subject involved are introduced.

[0291] In the following, two possible implementation modes are introduced.

[0292] Mode 1,

[0293] Referring to Figure 14In some embodiments, the steps before step S210, including: obtaining sensor data by the sensor, and determining whether the sensor data reaches the first gesture change data, can be performed by the motion management module of the terminal device 100.

[0294] Specifically, steps 1401 to 1408 introduce the enabling of the first gesture change data (i.e., the aforementioned auxiliary operation), which can be understood as an initialization step and is performed in response to the target application being started. It should be understood that the steps do not need to be repeated before the corresponding application is closed.

[0295] Step 1401: The target application is started. This step can be in response to the application icon of the main interface in the terminal device 100 being touched.

[0296] Step 1402: The auxiliary operation management program is activated, i.e., in response to step 1401, the auxiliary operation management program is started.

[0297] Step 1403: The auxiliary operation is started. This step can be understood as opening and closing the operation in the interface of the auxiliary operation management program itself. In one possible implementation, this step can be directly performed by default, i.e., when the target application is started, the auxiliary operation management program can automatically start the corresponding auxiliary operation. In another possible implementation, referring to the foregoing Figure 12A 、 Figure 12B and Figure 12D , the terminal device 100 can start the auxiliary operation in response to one or more of the keys 1110, 1120, and 1160.

[0298] According to the target application activating the auxiliary operation management program, the auxiliary operation management program can correspond the first gesture change data to the first key of the target application, so that the first gesture change data can be used to implement the control of the application.

[0299] Step 1404: In response to the auxiliary operation management program being started, the auxiliary operation management program sends the position of the first icon to the UI, so that the UI can display the first icon at the corresponding position when subsequently receiving the display command of the first icon.

[0300] Step 1405: In response to the auxiliary operation management program being started, the auxiliary operation management program sends the mapping relationship between the first gesture change data and the first icon to the input management service module, so that when the input management module receives the first gesture change data, it can determine that the user operation corresponds to the first icon, and forwards the first icon to the UI and the target application.

[0301] Step 1406: In response to the auxiliary operation manager being started, the auxiliary operation manager sends a notification to the input management module, for notifying it to start subscribing to the motion management.

[0302] Optionally, the order of steps 1404 to 1406 is not limited.

[0303] Step 1407: The input management service subscribes to the first gesture change data (corresponding to the auxiliary operation) from the motion module.

[0304] Step 1408: In response to step 1407 being executed, the motion module subscribes to the sensor data.

[0305] The following steps (steps 1409 to 1416) occur in response to the terminal device 100 sensor reporting data each time, and this series of steps can occur multiple times during the running of the target application.

[0306] Step 1409: The sensor reports sensor data to the motion module,

[0307] This step can be performed by the sensor in the hardware layer, and the reported data can be understood as raw data.

[0308] This step can occur in response to the user's manipulation of the terminal device 100.

[0309] Step 1410: The motion module determines whether the gesture data reported by the sensor meets the first gesture change data (i.e., the auxiliary operation) according to the subscription content.

[0310] It should be understood that this step also determines the type of auxiliary operation, for example, auxiliary operation D2.

[0311] Step 1411: The motion module sends the first gesture change data (i.e., the auxiliary operation) to the input management service.

[0312] Step 1412: The input management service module determines the first icon according to the auxiliary operation and the mapping relationship between the auxiliary operation and the first icon, and in addition, since the input management module also knows the position corresponding to the first icon, this step also determines the position of the first icon.

[0313] Step 1413: The input management service module sends a notification to the UI, for notifying the UI that a point has occurred at the position of the first icon, so that the UI can display the first icon at the position of the first icon. This display is used to prompt the type of operation performed.

[0314] Step 1414, the UI displays the first icon, for prompting the user to perform the type of operation.

[0315] Step 1415: The input management service module sends the location of the pointing event to the target application.

[0316] Step 1416: The target application executes corresponding instructions according to the location of the pointing location.

[0317] It should be understood that when the input management service module determines the location corresponding to the pointing event, the steps thereof are the same as those of the general touch operation. Thus, step 1412 can be understood as that the input management module converts the auxiliary operation input into a touch operation input, and only needs to confirm the location of the input operation. In step 1416, the program to be executed by the target application is determined according to the pointing location, and it is determined whether the pointing location corresponds to the first key. In some possible implementation manners, the first icon corresponding to the auxiliary operation has a location range corresponding to the location range of the first key, that is, the location range of the first icon corresponding to the auxiliary operation has a mapping relationship with the first key.

[0318] Method 2,

[0319] Please refer to Figure 15 the schematic flowchart shown in FIG. 15.

[0320] Compared with method 1, the improvement of method 2 mainly lies in canceling the call to the motion module. Moreover, the step of determining whether the sensor data reaches the first attitude change data is changed to be performed by the auxiliary operation management program.

[0321] Specifically, steps 1501-1504 can refer to steps 1401-1404 in the foregoing method 1,

[0322] Step 1505: In response to the auxiliary operation management program starting the auxiliary operation, the auxiliary operation management program subscribes to the sensor data.

[0323] The above steps can also be understood as initialization steps.

[0324] Step 1506: The sensor sends sensor data, that is, raw data, to the auxiliary operation management program.

[0325] Step 1507: The auxiliary operation management program determines whether the sensor data reaches the first attitude change data (corresponding to the auxiliary operation), and determines the first icon corresponding to the auxiliary operation and the location of the first icon.

[0326] Step 1508: The auxiliary operation management program notifies the UI that a pointing event occurs at the location of the first icon.

[0327] Step 1509: In response to the execution of step 1508, the UI displays the first icon at the location of the first icon.

[0328] Step 1510: the auxiliary operation management program sends the location of the report point event to the input management module;

[0329] This step is similar to the general touch screen operation, and the input management module receives the report point event and its location reported by the touch screen.

[0330] Steps 1511 to 1513 can be understood as subsequent steps of the touch screen operation, and can also refer to the description of steps 1415 and 1416 in Figure 14 .

[0331] Compared with mode 1, since the call to the motion module is reduced, the called modules and programs are less, the interfaces involved are less, and the error rate is reduced, thereby improving the stability. In addition, in mode 2, since the step of subscribing data from the motion module to the sensor by the input management module is avoided, the data transmission path is reduced, thereby the overall running time is shorter and the efficiency is higher.

[0332] Figure 15 is a schematic block diagram of the terminal device apparatus 1300 provided by the embodiment of the present application.

[0333] As shown in Figure 15 , the apparatus 1300 comprises:

[0334] The detection unit 1310 is configured to detect first attitude change data.

[0335] The processing unit 1320 is configured to determine a first key according to the first attitude change data of the terminal device and a first mapping relationship, the first mapping relationship comprising a corresponding relationship between the first attitude change data and the first key.

[0336] The processing unit 1320 is further configured to execute an instruction associated with the first key.

[0337] The apparatus 1300 for operating the terminal device can correspond to the terminal device described in the above method 200, and each module or unit in the apparatus 1300 is respectively configured to perform each action and processing process performed by the terminal device in the above method 200. Here, to avoid redundancy, the detailed description is omitted.

[0338] The embodiment of the present application further provides a terminal device, comprising one or more memories and one or more processors. The one or more memories store one or more computer programs, the one or more computer programs comprising instructions, when the instructions are executed by the one or more processors, causing the terminal device to perform the operation method in the above embodiment.

[0339] The embodiment of the present application further provides a terminal device, comprising a sensor, a processor, a memory and a display screen. The sensor is used for detecting the posture of the terminal device, the memory is used for storing a computer program, and the processor is used for calling and running the computer program from the memory, so that the terminal device executes the operation method in the above embodiment.

[0340] The present application further provides an apparatus for operating a terminal device, comprising units for performing the steps of the above operation method.

[0341] The apparatus for operating the terminal device can be configured in the terminal device. In this case, the detection unit in the apparatus for operating the terminal device can be implemented by using the sensor in the terminal device. The processing unit in the user identification apparatus can be independent of the processor in the terminal device. Alternatively, the processing unit in the apparatus for operating the terminal device can be implemented by using the processor in the terminal device.

[0342] Alternatively, the apparatus for operating the terminal device itself can be the terminal device.

[0343] The embodiment further provides a computer storage medium, which stores computer instructions. When the computer instructions are run on the terminal device, the terminal device executes the related method steps to implement the operation method in the above embodiment.

[0344] The embodiment further provides a computer program product, which, when run on a computer, causes the computer to execute the related steps to implement the operation method in the above embodiment.

[0345] Those skilled in the art can understand that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by terminal device hardware or a combination of computer software and terminal device hardware. Whether the functions are realized in hardware or software mode depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0346] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, apparatus and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0347] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic. The division of the units is merely logical function division. There can be other division manners in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0348] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0349] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can be a physically separate unit, or two or more units can be integrated into one unit.

[0350] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0351] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for controlling a terminal device, characterized in that, include: The first button is determined based on the first posture change data of the terminal device and the first mapping relationship, wherein the first mapping relationship includes the correspondence between the first posture change data and the first button; Execute the instruction associated with the first button; The method further includes associating the first posture change data with the first icon: The first interface is displayed, which includes the first button and the first icon; wherein the positions of the first button and the first icon are in a corresponding relationship. The first interface also includes a second icon, which is associated with second posture change data; In response to moving the second icon to the position of the first button, a correspondence is established between the position of the first button and the position of the second icon, wherein the position of the first button corresponds to the position of the first icon and the position of the second icon, respectively.

2. The method according to claim 1, characterized in that, The method further includes: In response to executing the instruction, the shape of the first icon is updated.

3. The method according to claim 1 or 2, characterized in that, The method further includes: Before displaying the first interface, a second interface is displayed, which is the interface for setting the correspondence between the first icon and the first button of the terminal device; In response to detecting a user's action, a correspondence is established between the first icon and the first button.

4. The method according to claim 1 or 2, characterized in that, The first icon is movable, and the correspondence between the first icon and the first button can be changed by moving the first icon.

5. The method according to claim 1 or 2, characterized in that, Before determining the first button based on the first posture change data and the first mapping relationship of the terminal device, the method further includes: The auxiliary operation management program enables the first posture change data. The auxiliary operation management program is used to manage the first posture change data and connects with the target application (APP) so that the target APP can be controlled through the first posture change data. The auxiliary operation management program receives data reported by the sensors; Under preset conditions, the auxiliary operation management program reports the location corresponding to the first posture change data to the target APP. The preset conditions include the auxiliary operation management program detecting that the data sent by the sensor reaches the first posture change data. The steps of determining the first button based on the first posture change data and the first mapping relationship of the terminal device, and executing the instruction associated with the first button, include: The target app determines the first button based on the position corresponding to the first posture change data of the terminal device and the position of the first button, and the target app executes the instruction associated with the first button.

6. The method according to claim 1 or 2, characterized in that, The first attitude change data is user-defined.

7. The method according to claim 1 or 2, characterized in that, The first button includes multiple buttons.

8. The method according to claim 7, characterized in that, The plurality of buttons are independent of each other, and the method further includes: In response to the user's action, multiple copies of the first icon are obtained; In response to moving the plurality of first icons to the positions of the plurality of buttons respectively, the positions of the plurality of buttons and the positions of the plurality of first icons are respectively in a corresponding relationship.

9. The method according to claim 1 or 2, characterized in that, The first attitude change data includes the first velocity of the terminal device when it moves in the first direction. The step of determining the first button based on the first posture change data and the first mapping relationship of the terminal device includes: When the first speed is greater than or equal to the first threshold, the first button is determined based on the first attitude change data and the first mapping relationship.

10. The method according to claim 1 or 2, characterized in that, The first attitude change data includes the second velocity when the terminal device moves in the second direction for the first time and the third velocity when the terminal device moves in the second direction for the second time. The step of determining the first button based on the first posture change data and the first mapping relationship of the terminal device further includes: When the time interval between the start time of the first movement in the second direction and the start time of the second movement in the second direction is within a preset range, and the second speed is greater than or equal to the second threshold and the third speed is greater than or equal to the third threshold, the first button is determined based on the first posture change data and the first mapping relationship.

11. The method according to claim 1 or 2, characterized in that, The method further includes: The terminal device has a switch icon in its control bar or floating window, which is used to turn the first posture change data on or off.

12. The method according to claim 1 or 2, characterized in that, The first button is the game button.

13. A terminal device, characterized in that, One or more processors; One or more memory units; The one or more memories store one or more computer programs, the one or more computer programs including instructions that, when executed by the one or more processors, cause the terminal device to perform the following steps: The first button is determined based on the first posture change data of the terminal device and the first mapping relationship, wherein the first mapping relationship includes the correspondence between the first posture change data and the first button; Execute the instruction associated with the first button; associate the first posture change data with the first icon, and when the instruction is executed by the one or more processors, cause the terminal device to perform the following steps: The first interface is displayed, which includes the first button and the first icon; wherein the positions of the first button and the first icon are in a corresponding relationship. The first interface also includes a second icon, which is associated with second posture change data; In response to moving the second icon to the position of the first button, a correspondence is established between the position of the first button and the position of the second icon, wherein the position of the first button corresponds to the position of the first icon and the position of the second icon, respectively.

14. The terminal device according to claim 13, characterized in that, When the instruction is executed by the one or more processors, the terminal device performs the following steps: In response to executing the instruction, the processor updates the shape of the first icon.

15. The terminal device according to claim 13 or 14, characterized in that, When the instruction is executed by the one or more processors, the terminal device performs the following steps: Before displaying the first interface, a second interface is displayed, which is the interface for setting the correspondence between the first icon and the first button of the terminal device; In response to detecting a user's action, a correspondence is established between the first icon and the first button.

16. The terminal device according to claim 13 or 14, characterized in that, The first icon is movable, and the correspondence between the first icon and the first button can be changed by moving the first icon.

17. The terminal device according to claim 13 or 14, characterized in that, When the instruction is executed by the one or more processors, the terminal device performs the following steps: The auxiliary operation management program enables the first posture change data. The auxiliary operation management program is used to manage the first posture change data and connects with the target application (APP) so that the target APP can be controlled through the first posture change data. The auxiliary operation management program receives data reported by the sensors; Under preset conditions, the auxiliary operation management program reports the location corresponding to the first posture change data to the target APP. The preset conditions include the auxiliary operation management program detecting that the data sent by the sensor reaches the first posture change data. The steps of determining the first button based on the first posture change data and the first mapping relationship of the terminal device, and executing the instruction associated with the first button, include: The target app determines the first button based on the position corresponding to the first posture change data of the terminal device and the position of the first button, and the target app executes the instruction associated with the first button.

18. The terminal device according to claim 13 or 14, characterized in that, The first attitude change data is user-defined.

19. The terminal device according to claim 13 or 14, characterized in that, The first button includes multiple buttons.

20. The terminal device according to claim 19, characterized in that, The terminal device is also used to perform the following steps: In response to the user's action, multiple copies of the first icon are obtained; In response to moving the plurality of first icons to the positions of the plurality of buttons respectively, the positions of the plurality of buttons and the positions of the plurality of first icons are respectively in a corresponding relationship.

21. The terminal device according to claim 13 or 14, characterized in that, When the instruction is executed by the one or more processors, the terminal device performs the following steps: The first posture change data includes a first velocity when the terminal device moves in a first direction, and the step of determining the first button based on the first posture change data of the terminal device and a first mapping relationship includes: When the first speed is greater than or equal to the first threshold, the first button is determined based on the first attitude change data and the first mapping relationship.

22. The terminal device according to claim 13 or 14, characterized in that, When the instruction is executed by the one or more processors, the terminal device performs the following steps: The first posture change data includes the second velocity when the terminal device moves in the second direction for the first time and the third velocity when the terminal device moves in the second direction for the second time. Determining the first button based on the first posture change data and the first mapping relationship of the terminal device further includes: When the time interval between the start time of the first movement in the second direction and the start time of the second movement in the second direction is within a preset range, and the second speed is greater than or equal to the second threshold and the third speed is greater than or equal to the third threshold, the first button is determined based on the first posture change data and the first mapping relationship.

23. The terminal device according to claim 17, characterized in that, The terminal device has a switch icon in its control bar or floating window. The switch icon is used to turn the auxiliary operation of the auxiliary operation management program on or off. The auxiliary operation corresponds to the first posture change data.

24. The terminal device according to claim 13 or 14, characterized in that, The first button is the game button.

25. A computer storage medium, characterized in that, It includes computer instructions that, when executed on a terminal device, cause the terminal device to perform the method as described in any one of claims 1 to 12.

Images