Application program control method and device, electronic equipment and computer storage medium

Abstract

The embodiment of the invention discloses an application program control method which is applied to electronic equipment and comprises the steps that when the electronic equipment plays spatial audios of at least two application programs, target control instructions for the application programs are determined according to obtained rotation data of the head of a user of the electronic equipment, and the target control instructions are executed. The embodiment of the invention further provides an application program control device, electronic equipment and a computer storage medium.

Description

Technical Field

[0001] The present application relates to human-computer interaction technology in holographic audio, and more particularly to a control method, device, electronic device and computer storage medium for an application. Background Art

[0002] At present, the interaction modes of mobile phones include touch interaction, voice interaction and peripheral interaction. Among them, touch interaction means that users achieve interactive control by touching the mobile phone screen; voice interaction means that users achieve interactive control of the mobile phone by speaking; peripheral interaction, for example, the buttons of a flip phone, the mobile phone achieves interaction by issuing instructions to the mobile phone through a wired connection or Bluetooth connection.

[0003] However, under touch control, the user has to walk towards the phone and touch the phone, and the distance between the person and the phone is 0; during voice interaction, the phone needs to be equipped with a neural network model and pre-set several sets of voice command execution controls to match the user's sentence to a certain command for execution. At this time, there are certain requirements for the user's pronunciation standard and environmental noise to complete the correct matching of sentence instructions; peripheral interaction requires the use of peripherals to achieve interaction with the phone; it can be seen that in the existing human-computer interaction mode, there is a technical problem of low control efficiency of electronic devices over application programs. Summary of the invention

[0004] The embodiments of the present application provide a method and device for controlling an application program, an electronic device, and a computer storage medium, which can improve the efficiency of the electronic device in controlling the application program.

[0005] The technical solution of this application is implemented as follows:

[0006] The present application embodiment provides a method for controlling an application program, the method being applied to an electronic device, comprising:

[0007] When the electronic device plays the spatial audio of the application, determining a target control instruction for the application according to the acquired rotation data of the head of the user of the electronic device;

[0008] Execute target control instructions.

[0009] An embodiment of the present application provides a control device for an application program, wherein the device is arranged in an electronic device and includes:

[0010] a determination module, configured to determine a target control instruction for the application program according to the acquired head rotation data of the user of the electronic device when the electronic device plays the spatial audio of the application program;

[0011] An execution module is used to execute the target control instruction.

[0012] An embodiment of the present application provides an electronic device, including:

[0013] A processor and a storage medium storing instructions executable by the processor, wherein the storage medium relies on the processor to perform operations through a communication bus, and when the instructions are executed by the processor, the control method of the application described in one or more of the above embodiments is executed.

[0014] An embodiment of the present application provides a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the control method of the application program as described in one or more embodiments.

[0015] The embodiments of the present application provide a method, device, electronic device and computer storage medium for controlling an application. The method is applied to an electronic device, comprising: when the electronic device plays the spatial audio of the application, according to the rotation data of the head of the user of the electronic device obtained, determining the target control instruction for the application and executing the target control instruction; that is, in the embodiments of the present application, when the electronic device plays the spatial audio of the application, according to the rotation data of the head of the user obtained, the target control instruction for the application can be determined, and the target control instruction can be executed on the application. In this way, the target control instruction can be executed on the application by monitoring the rotation of the user's head, that is, the control of the target application can be achieved by only monitoring the rotation of the head, thereby improving the control efficiency of the electronic device over the application under holographic audio. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of an optional application control method provided in an embodiment of the present application;

[0017] Figure 2 A flowchart of an example 1 of controlling an optional application program provided in an embodiment of the present application;

[0018] Figure 3 A flowchart of a second example of controlling an optional application program provided in an embodiment of the present application;

[0019] Figure 4 A schematic diagram of an optional user's head rotation process provided in an embodiment of the present application;

[0020] Figure 5 A schematic diagram of the structure of a control device for an optional application provided in an embodiment of the present application;

[0021] Figure 6 A schematic diagram of the structure of an optional electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0023] In the related art, mobile phone interaction can be carried out by touch interaction, voice interaction and peripheral interaction, however, these interaction methods make the control of the electronic device over the application program inefficient.

[0024] In addition, holographic audio (also known as holographic sound effect) is a technology that simulates auditory perception, that is, 3D sound effect technology, which uses complex algorithms and processing methods to transform sound from a single plane into three-dimensional sound, so that the audience can clearly perceive the position and distance of the sound in space; then, for the control of applications under holographic audio, the embodiment of the present application provides a control method for an application, which is applied to an electronic device, Figure 1 A flowchart of an optional application control method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the control method of the application may include:

[0025] S101: when the electronic device plays the spatial audio of the application, determining a target control instruction for the application according to the acquired rotation data of the head of the user of the electronic device;

[0026] In S101, all or part of the applications in the electronic device may adopt holographic audio technology, so that the electronic device can play spatial audio of all or part of the applications, wherein the application adopting holographic audio technology is one application with audio, or may be at least two applications with audio.

[0027] When an electronic device plays the spatial audio of an application, the spatial position of the spatial audio of this application is generally located at the center of the sound field space formed by the user listening device connected to the electronic device; when the electronic device plays the spatial audio of at least two applications, here, the spatial position of the spatial audio of each of the at least two applications can be different. In addition, when the above-mentioned electronic device adopts holographic audio technology, the spatial position of the spatial audio is within a spatial range of a preset distance from the user.

[0028] Among them, when the electronic device plays the spatial audio of an application, the electronic device determines the target control instructions for the application based on the obtained rotation data of the user's head. Here, the electronic device can obtain the rotation data of the head collected by the wearable device from the wearable device on the head. The acquisition here can be obtained from the wearable device or from the memory of the electronic device. Here, the embodiment of the present application does not make specific limitations on this.

[0029] In addition, with respect to obtaining the rotation data from the wearable device, the wearable device is provided with an inertial measurement unit (IMU), and the IMU is used to measure the three-axis attitude angle and acceleration of the head in real time.

[0030] In this way, when the electronic device meets the need to play the spatial audio of an application, it can determine the target control instructions for the application based on the rotation data that has been acquired, and control the target control instructions, thereby achieving control over the application.

[0031] S102: Execute control of the target control instruction.

[0032] After determining the target control instruction for the application through S101, the target control instruction can be executed on the application. When there is only one application, it can be a control instruction for this one application. When there are at least two applications, it can be a control instruction for the target application in the application. The control instruction can be a fixed control instruction or a control instruction determined based on the rotation action information corresponding to the rotation data. Here, the embodiment of the present application does not make any specific limitation on this.

[0033] Since the number of the above application programs may be one, or two or more, then, for the case where there is only one application program, in an optional embodiment, S101 may include:

[0034] When the electronic device plays spatial audio of an application, determining first rotation action information of the user according to the rotation data;

[0035] For the application, determining a control instruction corresponding to the first rotation action information;

[0036] The control instruction corresponding to the first rotation action information is determined as the target control instruction.

[0037] It can be understood that when an electronic device plays the spatial audio of an application, the first rotation action information can be determined based on the rotation data. For example, the determined first rotation action information is the action information of a head shaking action. Then, if the spatial audio of a music application is currently being played, and the control instruction corresponding to the head shaking action information is to play the next track, the instruction to play the next track will be determined as the target control instruction, and the music application will be controlled to play the next track.

[0038] In this way, the application currently playing the spatial audio can be controlled by the user's head rotation, so as to improve the control efficiency of the electronic device over the applications participating in the holographic audio.

[0039] In addition, for at least two or more application programs, in an optional embodiment, S101 may include:

[0040] When the electronic device plays spatial audio of at least two applications, determining the orientation of the user based on the rotation data;

[0041] A target application is determined from among at least two applications to which the spatial audio toward which the user is directed belongs, and a target control instruction for the target application is determined.

[0042] It can be understood that when an electronic device plays spatial audio of at least two applications, the electronic device can determine the user's orientation based on the obtained rotation data of the user's head, and the user's orientation is mainly the direction the user is facing, wherein the rotation data may include: position coordinates before rotation and motion vectors during rotation, wherein a three-dimensional rectangular coordinate system is established with a fixed spatial position as the origin, and the position coordinates of the user's current head in the three-dimensional rectangular coordinate system are marked as position coordinates before rotation. In addition, the motion vector of the user's head can be collected by the IMU in the wearable device, so that the position coordinates before rotation and the motion vector during rotation can be obtained. After knowing the position coordinates before rotation and the motion vector during rotation, the position coordinates after rotation can be calculated, and thus the user's orientation can be determined based on the position coordinates after rotation.

[0043] After determining the user's direction, the electronic device selects the application to which the spatial audio that the user is heading belongs from at least two applications, and determines a target control instruction for the target application. Here, when there is only one application to which the spatial audio that the user is heading belongs, the application is determined as the target application. When there are multiple applications to which the spatial audio that the user is heading belongs, the application to which the spatial audio that the user is heading belongs that is closest to the user in the spatial position of the spatial audio that the user is heading can be determined as the target application. The application to which the spatial audio that the user is heading belongs that is within a preset distance range from the user in the spatial position of the spatial audio that the user is heading can also be determined as the target application. The embodiments of the present application do not specifically limit this.

[0044] After the electronic device determines the target application, it can control the target application. For example, the electronic device continues to play the spatial audio of the target application and mutes at least two applications other than the target application. In this way, only the spatial audio of the target application is played in the entire space, so that the user can immerse himself in the spatial audio of the target application.

[0045] In addition, in addition to the above-mentioned control method for the target application, the spatial audio of the target application can be played at a volume of A%, and the spatial audio of at least two applications other than the target application can be played at a volume of B%, where A is greater than B. In this way, the volume of the spatial audio of the target application is greater than the volume of the spatial audio of at least two applications other than the target application, so that the user can hear both the spatial audio of the target application at a larger volume and the spatial audio of the at least two applications other than the target application at a smaller volume.

[0046] The spatial audio of the target application can also be played at a volume of A%. The volume of at least two applications other than the target application can be determined according to the priority of the at least two applications other than the target application, and the spatial audio of at least two applications other than the target application can be played at the determined volume, and A% is greater than other determined volume values. In this way, the spatial audio can be played at different volumes in combination with the user's orientation and the priority of the application, thereby realizing intelligent control of holographic audio.

[0047] Furthermore, the target application may be controlled based on the rotation action information determined by the rotation data, for example, closing the target application, entering the next page of the target application, etc. Here, the embodiment of the present application does not make any specific limitation on this.

[0048] In order to control the target application by monitoring the rotation data of the user's head under holographic audio, in an optional embodiment, when the electronic device plays the spatial audio of the application, determining the user's orientation according to the rotation data may include:

[0049] When the electronic device plays spatial audio of at least two applications and the priorities of the at least two applications meet a preset priority condition, the user's orientation is determined according to the rotation data.

[0050] It can be understood that when the electronic device plays the spatial audio of at least two applications, it is also necessary to check whether the priorities of the at least two applications meet the preset priority conditions. For example, each time the spatial audio of an application is started, the priority of the already opened application is compared with the priority of the application opened this time. Only when the priority of the application opened this time is higher than the lowest priority among the priorities of the already opened applications, it is determined that the priorities of at least two applications including the already opened application and the application opened this time meet the preset priority conditions, and then the user's orientation is determined according to the rotation data, that is, the control method of the application proposed in the embodiment of the present application is started. Otherwise, it is determined that the priorities of at least two applications including the already opened application and the application opened this time do not meet the preset priority conditions. For at least two applications including the already opened application and the application opened this time, the control method of the application proposed in the embodiment of the present application is prohibited from being started; if the already opened application includes at least two applications, and the priority of the already opened application meets the preset priority condition, then for at least two applications including the already opened application and the application opened this time, even if the control method of the application proposed in the embodiment of the present application is prohibited from being started, the control method of the application proposed in the embodiment of the present application is still executed for the already opened application.

[0051] In this way, only when the priorities of the applications that have been opened and the application that is opened this time meet the preset priority conditions, the user's orientation is determined based on the rotation data to achieve control of the target application. That is to say, only when the electronic device plays spatial audio of at least two applications, it is also necessary to determine that the priorities of at least two applications meet the preset priority conditions, that is, there are certain requirements for the priorities of at least two applications before the control of the target application is achieved by monitoring the rotation data of the user's head, thereby improving the control efficiency of electronic devices over applications under holographic audio.

[0052] Furthermore, in order to improve the control efficiency of the electronic device over the application under holographic audio, in an optional embodiment, the method may further include:

[0053] When the electronic device is playing the spatial audio of at least one application, starting to play the spatial audio of a new application;

[0054] When the priority of the new application is higher than the lowest priority among the priorities of all applications in at least one application, it is determined that the electronic device plays spatial audio of at least two applications, and the priorities of the at least two applications meet a preset priority condition.

[0055] It can be understood that when an electronic device plays spatial audio of at least one application, for example, when the electronic device has already started playing spatial audio of an application, when starting to play spatial audio of a new application, it is necessary to compare whether the priority of the new application is higher than the lowest priority of the already opened application. If so, it means that the priority of the new application is not the lowest priority among the opened priorities. For both the new application and the already opened application, it is possible to start monitoring the user's head rotation data to achieve control of the target application in the new application and the already opened application.

[0056] If the priority of the newer application is not higher than the priority of the already opened application, then, for the new application and the already opened application, the control of the target application among the new application and the already opened application by monitoring the rotation data of the user's head is not enabled. Therefore, when the priority of the new application is not higher than the lowest priority among the priorities of the already opened applications, it is determined that the electronic device plays spatial audio of at least two applications, and the priorities of at least two applications do not meet the preset priority condition; wherein the at least two applications include: a new application and at least one application (the already opened application).

[0057] When the electronic device has already started playing the spatial audio of two applications, it starts playing the spatial audio of a new application. Similarly, it is necessary to compare whether the priority of the new application is higher than the lowest priority of all applications in the two already opened applications. If so, it means that the priority of the new application is not the lowest priority among the opened priorities. For the three applications, the new application and the already opened application, it is possible to start monitoring the user's head rotation data to achieve control of the new application and the target application in the two already opened applications.

[0058] If the priority of the new application to be compared is not higher than the lowest priority of the two applications that have been opened, for the three applications, the new application and the application that have been opened, the control of the target application among the new application and the two applications that have been opened by monitoring the rotation data of the user's head is not enabled. Therefore, when the priority of the new application is not higher than the lowest priority of the two applications that have been opened, it is determined that the electronic device plays spatial audio of at least two applications, and the priorities of at least two applications do not meet the preset priority conditions; wherein the at least two applications include: a new application and at least one application (an application that has been opened); it should be noted that, for the two applications that have been opened and the application that is opened this time, even if the control method of the application proposed in the embodiment of the present application is prohibited, the control method of the application proposed in the embodiment of the present application is still executed for the two applications that have been opened.

[0059] In this way, by judging whether the priority of the new application is higher than the lowest priority among the already opened applications, it is determined whether to enable the control of the new application and the target application among the already opened applications by monitoring the rotation data of the user's head. This allows the application control method proposed in the embodiment of the present application to be reasonably enabled, thereby improving the control efficiency of the electronic device over the application under holographic audio.

[0060] In order to determine the orientation of the user, in an optional embodiment, determining the orientation of the user according to the rotation data may include:

[0061] Determine the position coordinates after the rotation according to the position coordinates before the rotation in the rotation data and the motion vector in the rotation data;

[0062] The user's orientation is determined based on the position coordinates after rotation.

[0063] It can be understood that after the electronic device obtains the rotation data of the user's head, it can obtain the position coordinates and motion vector before the rotation from the rotation data, wherein a three-dimensional rectangular coordinate system is established with a fixed spatial position as the origin, and the position coordinates of the user's current head in the three-dimensional rectangular coordinate system are marked as the position coordinates before the rotation. In addition, the motion vector of the user's head can be collected by the IMU in the wearable device. In this way, the position coordinates before the rotation and the motion vector during the rotation can be obtained, and the position coordinates before the rotation are added to the motion vector, and the result is the position coordinates after the rotation. The position coordinates after the rotation can be used to determine the user's orientation.

[0064] In this way, by obtaining the user's head rotation data, the user's orientation can be determined, thereby determining the target application, and then controlling the target application, thereby improving the control efficiency of the electronic device over the application under holographic audio.

[0065] In order to determine the target application from the application to which the spatial audio that the user is directed toward belongs among the at least two applications, in an optional embodiment, determining the target application from the application to which the spatial audio that the user is directed toward belongs among the at least two applications, and determining the target control instruction for the target application may include:

[0066] For each of the spatial audios in the at least two applications toward which the user is directed, calculating a distance between the user and a spatial position of each of the spatial audios toward which the user is directed;

[0067] Selecting the shortest distance from the distances between the user and the spatial position of each spatial audio that the user is facing;

[0068] The application to which the spatial audio corresponding to the selected shortest distance belongs is determined as the target application, and a target control instruction for the target application is determined.

[0069] It can be understood that before playing the spatial audio of an application, the electronic device needs to first determine the spatial position of each spatial audio. After determination, it can play the spatial audio based on the determined spatial position corresponding to each application, so that the spatial positions of the spatial audio of at least two applications are stored in the electronic device.

[0070] Then, based on knowing the spatial positions of the spatial audios of at least two applications, the application to which the spatial audio to which the user is heading belongs can be determined among the at least two applications, and for each spatial audio in the spatial audio to which the user is heading, the distance between the user and the spatial position of each spatial audio to which the user is heading is calculated, and then the shortest distance is selected from the distances between the user and the spatial position of each spatial audio to which the user is heading, and the application to which the spatial audio corresponding to the shortest distance belongs is determined as the target application. In this way, the target application can be determined.

[0071] After determining the target application, the electronic device can determine the target control instruction of the target application. The control instruction of the target application can be a preset instruction or an instruction determined based on the rotation data. Here, the embodiment of the present application does not make specific limitations on this.

[0072] In this way, the application closest to the user's front is determined from the applications to which the spatial audio to which the user is facing belongs, and is controlled, and the target application is accurately determined to achieve control of the target application, thereby improving the control efficiency of the electronic device over the application under holographic audio.

[0073] In order to achieve control over the target application, in an optional embodiment, S102 may include:

[0074] Play the spatial audio of a target application and mute the spatial audio of at least two applications other than the target application.

[0075] It can be understood that after determining the target application, the electronic device continues to play the spatial audio of the target application and mutes the spatial audio of at least two applications other than the target application. In this way, only the spatial audio of the target application is played, so that the user can turn his head to the spatial position of the spatial audio of the target application, thereby immersing the user in the spatial audio of the target application.

[0076] In this way, the user can play only the spatial audio of the target application by turning his head to the spatial position of the spatial audio of the target application, thereby improving the control efficiency of the electronic device over the application under holographic audio.

[0077] In addition, in order to achieve control over the target application, in an optional embodiment, S102 may include:

[0078] Play the spatial audio of the target application at a first preset volume, and play the spatial audio of at least two applications other than the target application at a second preset volume;

[0079] It can be understood that after determining the target application, the electronic device can play the corresponding spatial audio of the target application according to the first preset volume, and play the spatial audio of at least two applications other than the target application according to the second preset volume, wherein the first preset volume is greater than the second preset volume.

[0080] In this way, the spatial audio of the target application and the spatial audio of at least two applications other than the target application are played in the user's space, and the spatial audio of the target application is mainly played, and the spatial audio of the at least two applications other than the target application is supplemented, thereby improving the sound effect of the spatial audio.

[0081] Further, in order to achieve control over the target application, in an optional embodiment, determining the target application from the application to which the spatial audio toward which the user is directed belongs among the at least two applications, and determining a target control instruction for the target application may include:

[0082] Determine a target application from among at least two applications to which the spatial audio that the user is facing belongs;

[0083] Determining second rotation action information of the user according to the rotation data;

[0084] For the target application, determining a control instruction corresponding to the second rotation action information;

[0085] The control instruction corresponding to the second rotation action information is determined as the target control instruction.

[0086] It can be understood that after the electronic device obtains the rotation data of the user's head, it can also determine the user's second rotation action information based on the rotation data, such as head shaking information or nodding information. Then, for the target application, each second rotation action information corresponds to a different control instruction. Here, the control instruction corresponding to the second rotation action information can be determined for the target application, and the target application can be controlled according to the control instruction corresponding to the second rotation action information.

[0087] For example, for music software, the control instruction corresponding to the second rotation action information is a close instruction, so the target application is closed after the target application is determined. The control instruction corresponding to the second rotation action information is an instruction to play the next track, so the next track is played after the target application is determined.

[0088] In this way, by monitoring the user's head rotation data, different controls on the target application can be achieved, thereby improving the control efficiency of the electronic device over the application under holographic audio.

[0089] In order to enable the electronic device to obtain the rotation data of the user's head, in an optional embodiment, when the user wears a Bluetooth headset, the above method may further include:

[0090] Get rotation data from Bluetooth headset.

[0091] It can be understood that when the user of an electronic device wears a Bluetooth headset, the Bluetooth headset has a Bluetooth connection with the electronic device, and an IMU is provided in the Bluetooth headset, which can be used to collect the rotation data of the user's head and transmit the rotation data of the user's head to the electronic device via Bluetooth, so that the electronic device can determine the target control instructions for the application to achieve control of the target application.

[0092] In this way, the rotation data of the head of the user of the electronic device is obtained from the Bluetooth headset so that the electronic device can be used to control the application under holographic audio, which helps to improve the control efficiency of the electronic device over the application under holographic audio.

[0093] In addition, in addition to the Bluetooth headsets mentioned above, wearable devices can also use other wearable devices such as smart glasses.

[0094] The following is an example to describe the control method of the application in one or more of the above embodiments.

[0095] Figure 2 A flowchart of an example 1 of an optional application control method provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the control method of the application may include:

[0096] S201: The application needs to start playing;

[0097] Specifically, when the electronic device plays the spatial audio of at least one application, the electronic device starts playing the spatial audio of a new application, which is called the application starting to play.

[0098] S202: Determine whether the current scene is interactive. If yes, execute S203;

[0099] Specifically, when the application starts playing, the electronic device needs to determine whether the current scene is interactive. Specifically, Figure 3 A flowchart of Example 2 of an optional application control method provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the control method of the application may include:

[0100] S301: The application starts playing;

[0101] S302: Determine whether the current scenario is a multi-application scenario. If yes, execute S303; if no, execute S304;

[0102] Here, S301 is equivalent to the above S201, and S302-S305 are equivalent to S202. Specifically, it is first determined whether the currently played spatial audio is the spatial audio of at least two applications. If yes, S303 is executed, and if no, S304 is executed;

[0103] S303: Determine whether the priority of the application currently opened is higher than the lowest priority among the priorities of the applications already opened. If yes, execute S305; if no, execute S304.

[0104] Specifically, it is compared whether the priority of the application currently opened is higher than the lowest priority of the priorities of the applications already opened. The applications in this example are all audio applications played using holographic audio technology. If the judgment is yes, S305 is executed, and if not, S304 is executed.

[0105] S304: Switch to a non-multi-application interactive scene, and end.

[0106] Specifically, for the scenario where only one application is opened or there are multiple applications, and the priority of the application opened this time is not higher than the lowest priority of the applications already opened, for all applications including the application opened this time and the applications already opened, the electronic device switches to a non-multi-application interactive scenario, that is, the holographic audio is not turned on, and the control of the target application is achieved by monitoring the rotation of the user's head to obtain IMU data. If there are at least two applications opened and in a multi-application interactive scenario, the scenario remains unchanged.

[0107] S305: Switch to a multi-application interactive scenario, and end.

[0108] Specifically, for at least two applications, where the priority of the application currently opened is higher than the lowest priority of the applications already opened, the electronic device switches to a multi-application interactive scenario for all applications including the application currently opened and the applications already opened, i.e., when holographic audio is turned on, the target application is controlled by monitoring the rotation of the user's head.

[0109] S203: Acquire IMU data;

[0110] Specifically, when it is determined that the holographic audio is turned on and the target application is controlled by monitoring the rotation of the user's head, the electronic device first obtains IMU data (equivalent to the rotation data of the user's head) from the Bluetooth headset worn on the user's ear.

[0111] S204: Determine whether it is a control action? If yes, execute S205;

[0112] Specifically, after acquiring the IMU data, the electronic device first determines whether the IMU data has a control action, that is, whether it is a control action for the application under the holographic audio. Here, the IMU data is judged, for example, to determine whether the action corresponding to the IMU data falls into the preset action. If so, execute S205.

[0113] S205: Calculate control actions;

[0114] Specifically, the control action is calculated based on the IMU data. Figure 4 A schematic diagram of an optional user's head rotation provided in an embodiment of the present application, such as Figure 4 As shown, assuming that the user's current position is A, the three-dimensional spatial coordinates are (x1, y1, z1). When the user's head moves to position B, the motion vector is (x2, y2, z2). The two vectors are added together to obtain the coordinates of the user's position after movement (x1+x2, y1+y2, z1+z2), which is the user's current orientation.

[0115] S206: Selecting a target application according to the spatial position of the spatial audio;

[0116] Specifically, after determining the user's direction, the electronic device calculates the distance between the user and the spatial position of each spatial audio that the user is facing based on knowing the spatial position of each spatial audio, selects the shortest distance from the calculated distances, and determines the application to which the spatial audio corresponding to the shortest distance belongs as the target application.

[0117] S207: Control the target application.

[0118] Here, the electronic device continues to play the spatial audio of the target application and mutes the spatial audio of applications other than the target application in all spatial audios.

[0119] On the product side, this example is applied to the spatial audio module of the audio service framework layer of the mobile terminal. It uses headphones with an IMU sensor to call back the user's head rotation vector data in real time, calculates the vector data to get the user's orientation, and then obtains the spatial position of the spatial audio of the application being played. The application that belongs to the spatial audio closest to the user's orientation is selected. If the scene is in focus mode at this time, other applications except the selected application are muted.

[0120] For example, when a user is listening to music, the music application (Application, APP) is in static mode. After a call comes in, it enters focus mode. The user turns his head toward the side of the incoming call ringtone, and the Bluetooth headset transmits the IMU data of the human ear rotation to the electronic device. The electronic device selects the application through calculation. When it is determined to be an incoming call application, all other applications are muted.

[0121] From the technical side, the process of electronic devices controlling audio streams through IMU data is divided into the following three steps: receiving IMU data through Bluetooth and determining user orientation, multi-application interactive scenario determination, and controlling audio stream behavior; among them, for multi-application interactive scenario determination, the IMU data calculation module provides the user orientation to the service layer of the electronic device, and the service layer of the electronic device needs to combine whether the current scenario is multi-application interactive, and if so, match the next control behavior; for controlling audio stream behavior, according to the application selected by the user, keep its audio stream playing normally and mute other audio streams.

[0122] It can be seen that this example uses the user's head motion data as input, which is different from the traditional method of using touch screen and voice as input. The head motion data is obtained through the IMU data calculation of the Bluetooth headset, and transmitted to the electronic device through Bluetooth. After data analysis, calculation, and matching control commands, each application is finally controlled.

[0123] In this way, based on holographic audio to distribute different sounds in all directions of space, this example can perform various control actions on mobile phone applications through the user's head movement, so that the user does not need to speak, touch, etc. to achieve human-computer interaction at a long distance.

[0124] An embodiment of the present application provides a method for controlling an application, which is applied to an electronic device, comprising: when the electronic device plays the spatial audio of the application, determining a target control instruction for the application based on the rotation data of the head of the user of the electronic device obtained, and executing the target control instruction; that is, in an embodiment of the present application, when the electronic device plays the spatial audio of the application, based on the rotation data of the user's head obtained, the target control instruction for the application can be determined, and the target control instruction can be executed on the application. In this way, the target control instruction can be executed on the application by monitoring the rotation of the user's head, that is, the control of the target application can be achieved by only monitoring the rotation of the head, thereby improving the control efficiency of the electronic device over the application under holographic audio.

[0125] Based on the same inventive concept as the above-mentioned embodiment, the embodiment of the present application provides a control device for an application program, wherein the device is arranged in an electronic device. Figure 5 A schematic diagram of a control device for an optional application provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the control device of the application includes:

[0126] A determination module 51, configured to determine a target control instruction for the application program according to the acquired rotation data of the head of the user of the electronic device when the electronic device plays the spatial audio of at least two applications;

[0127] The execution module 52 is used to execute the target control instruction.

[0128] In an optional embodiment, the determination module 51 is specifically configured to:

[0129] When the electronic device plays spatial audio of an application, determining first rotation action information of the user according to the rotation data;

[0130] For the application, determining a control instruction corresponding to the first rotation action information;

[0131] The control instruction corresponding to the first rotation action information is determined as the target control instruction.

[0132] In an optional embodiment, the determination module 51 is specifically configured to:

[0133] When the electronic device plays spatial audio of at least two applications, determining the orientation of the user based on the rotation data;

[0134] A target application is determined from among at least two applications to which the spatial audio toward which the user is directed belongs, and a target control instruction for the target application is determined.

[0135] In an optional embodiment, when the electronic device plays the spatial audio of the application, the determination module 51 determines the user's orientation according to the rotation data, including:

[0136] When the electronic device plays spatial audio of at least two applications and the priorities of the at least two applications meet a preset priority condition, the user's orientation is determined according to the rotation data.

[0137] In an optional embodiment, the device is further used for:

[0138] When the electronic device is playing the spatial audio of at least one application, starting to play the spatial audio of a new application;

[0139] When the priority of the new application is higher than the lowest priority among the priorities of all applications in at least one application, it is determined that the electronic device plays spatial audio of at least two applications, and the priorities of the at least two applications meet a preset priority condition; wherein the at least two applications include: at least one application and the new application.

[0140] In an optional embodiment, the determining module 51 determines the user's orientation according to the rotation data, including:

[0141] The position coordinates after the rotation are determined according to the position coordinates before the rotation in the rotation data and the motion vector in the rotation data; and the user's orientation is determined according to the position coordinates after the rotation.

[0142] In an optional embodiment, the determination module 51 determines a target application from the application to which the spatial audio that the user is facing belongs among the at least two applications, and determines a target control instruction for the target application, including:

[0143] For each of the spatial audios that the user is facing in the at least two applications, calculating a distance between the user and a spatial position of each of the spatial audios that the user is facing;

[0144] Selecting the shortest distance from the distances between the user and the spatial position of each spatial audio that the user is facing;

[0145] The application to which the spatial audio corresponding to the selected shortest distance belongs is determined as the target application, and a target control instruction for the target application is determined.

[0146] In an optional embodiment, the execution module 52 is specifically configured to:

[0147] Play the spatial audio of a target application and mute the spatial audio of at least two applications other than the target application.

[0148] In an optional embodiment, the execution module 52 is specifically configured to:

[0149] Play the spatial audio of the target application at a first preset volume, and play the spatial audio of at least two applications other than the target application at a second preset volume;

[0150] The first preset volume is greater than the second preset volume.

[0151] In an optional embodiment, the determination module 51 determines a target application from the application to which the spatial audio that the user is facing belongs among the at least two applications, and determines a target control instruction for the target application, including:

[0152] Determine a target application from among at least two applications to which the spatial audio that the user is facing belongs;

[0153] Determining second rotation action information of the user according to the rotation data;

[0154] For the target application, determining a control instruction corresponding to the second rotation action information;

[0155] The control instruction corresponding to the second rotation action information is determined as the target control instruction.

[0156] In an optional embodiment, when the user wears a Bluetooth headset, the device is used to:

[0157] Get rotation data from Bluetooth headset.

[0158] In practical applications, the above-mentioned determination module 51 and execution module 52 can be implemented by a processor located on the control device of the application, specifically a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA).

[0159] Figure 6 A schematic diagram of the structure of an optional electronic device provided in an embodiment of the present application, such as Figure 6 As shown, an embodiment of the present application provides an electronic device 600, including:

[0160] A processor 61 and a storage medium 62 storing executable instructions of the processor 61, wherein the storage medium 62 relies on the processor 61 to perform operations through a communication bus 63, and when the instructions are executed by the processor 61, the control method of the application executed in one or more of the above embodiments is executed.

[0161] It should be noted that in actual application, the various components in the electronic device are coupled together through the communication bus 63. It is understandable that the communication bus 63 is used to realize the connection and communication between these components. In addition to the data bus, the communication bus 63 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 6 Various buses are labeled as communication buses 63.

[0162] An embodiment of the present application provides a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the control method of the application program executed by the control device in one or more of the above embodiments.

[0163] Among them, the computer-readable storage medium can be a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) and other memories.

[0164] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0165] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0166] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0167] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0168] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. A method for controlling an application, characterized in that: The method is applied to an electronic device, comprising: When the electronic device plays the spatial audio of the application, determining a target control instruction for the application according to the acquired rotation data of the head of the user of the electronic device; Execute the target control instruction.

2. The method according to claim 1, characterized in that When the electronic device plays the spatial audio of an application, determining a target control instruction for the application according to the acquired rotation data of the head of the user of the electronic device includes: When the electronic device plays a spatial audio of an application, determining first rotation action information of the user according to the rotation data; For the application, determining a control instruction corresponding to the first rotation action information; The control instruction corresponding to the first rotation action information is determined as the target control instruction.

3. The method according to claim 1, characterized in that: When the electronic device plays the spatial audio of an application, determining a target control instruction for the application according to the acquired rotation data of the head of the user of the electronic device includes: When the electronic device plays spatial audio of at least two applications, determining the orientation of the user according to the rotation data; A target application is determined from the at least two applications to which the spatial audio toward which the user is directed belongs, and a target control instruction for the target application is determined.

4. The method according to claim 3, characterized in that When the electronic device plays the spatial audio of at least two applications, determining the orientation of the user according to the rotation data includes: When the electronic device plays spatial audio of at least two applications and the priorities of the at least two applications meet a preset priority condition, the orientation of the user is determined according to the rotation data.

5. The method according to claim 4, characterized in that The method further comprises: When the electronic device is playing the spatial audio of at least one application, starting to play the spatial audio of a new application; When the priority of the new application is higher than the lowest priority among the priorities of all applications in the at least one application, it is determined that the electronic device plays spatial audio of at least two applications, and the priorities of the at least two applications meet a preset priority condition; wherein the at least two applications include: the at least one application and the new application.

6. The method according to claim 3, characterized in that The determining the orientation of the user according to the rotation data includes: Determining the position coordinates after the rotation according to the position coordinates before the rotation in the rotation data and the motion vector in the rotation data; The orientation of the user is determined according to the position coordinates after the rotation.

7. The method according to claim 3, characterized in that The step of determining a target application from among the at least two applications to which the spatial audio that the user is facing belongs, and determining a target control instruction for the target application includes: For each of the spatial audios in the at least two applications that the user is facing, calculating a distance between the user and a spatial position of each of the spatial audios that the user is facing; Selecting the shortest distance from the distances between the user and the spatial position of each spatial audio toward which the user is directed; The application to which the spatial audio corresponding to the selected shortest distance belongs is determined as the target application, and a target control instruction for the target application is determined.

8. The method according to claim 3, characterized in that The executing the target control instruction comprises: The spatial audio of the target application is played, and the spatial audio of the application other than the target application among the at least two applications is muted.

9. The method according to claim 3, characterized in that: The executing the target control instruction comprises: Play the spatial audio of the target application at a first preset volume, and play the spatial audio of an application other than the target application among the at least two applications at a second preset volume; Wherein, the first preset volume is greater than the second preset volume.

10. The method according to claim 3, characterized in that The step of determining a target application from among the at least two applications to which the spatial audio that the user is facing belongs, and determining a target control instruction for the target application includes: Determine a target application from among the at least two applications to which the spatial audio toward which the user is directed belongs; determining second rotation action information of the user according to the rotation data; For the target application, determining a control instruction corresponding to the second rotation action information; The control instruction corresponding to the second rotation action information is determined as the target control instruction.

11. The method according to claim 1, characterized in that When the user wears a Bluetooth headset, the method further includes: The rotation data is obtained from the Bluetooth headset.

12. A control device for an application, characterized in that: The device is arranged in an electronic device and comprises: a determination module, configured to determine a target control instruction for the application program according to the acquired rotation data of the head of the user of the electronic device when the electronic device plays the spatial audio of the application program; An execution module is used to execute the target control instruction.

13. An electronic device, characterized in that: include: A processor and a storage medium storing instructions executable by the processor, wherein the storage medium relies on the processor to perform operations via a communication bus, and when the instructions are executed by the processor, the control method of the application described in any one of claims 1 to 11 is executed.

14. A computer storage medium, characterized in that: Executable instructions are stored, and when the executable instructions are executed by one or more processors, the processors execute the control method of the application program according to any one of claims 1 to 11.

Images