Method and apparatus for generating combined instructions during application operation

Abstract

The present disclosure relates to a method and device for generating a combination instruction during application operation. The method comprises: generating an instruction record layer corresponding to a target application, the instruction record layer covering a target interface corresponding to the target application; receiving a plurality of trigger operations for the target interface through the instruction record layer, and generating a combination instruction corresponding to the target interface according to a trigger sequence and a trigger position of the plurality of trigger operations; and issuing the plurality of trigger operations to the target application to execute operations corresponding to the plurality of trigger operations by the target application.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of touch control, and in particular, to a method and device for generating a combination instruction in an application operation process. BACKGROUND

[0002] In order to facilitate user operation, major manufacturers provide a function of generating a combination instruction in an electronic device. The function can record a user operation process by generating a combination instruction when the user operates any interface, so as to complete a series of operations included in the operation process by one-key triggering when the user needs to repeat the operation process.

[0003] However, in the related art, the function of generating a combination instruction must be implemented on the premise of starting an accessibility service. However, the accessibility service needs to open a large number of permissions to all applications, which is prone to privacy leakage. SUMMARY

[0004] The present disclosure provides a method and device for generating a combination instruction in an application operation process, which can generate a combination instruction corresponding to any application program without starting an accessibility service.

[0005] According to a first aspect of the present disclosure, a method for generating a combination instruction in an application operation process is provided, comprising:

[0006] generating an instruction recording layer corresponding to a target application program, the instruction recording layer covering a target interface corresponding to the target application program;

[0007] receiving a plurality of trigger operations for the target interface through the instruction recording layer, and generating a combination instruction corresponding to the target interface according to a trigger order and a trigger position of the plurality of trigger operations;

[0008] issuing the plurality of trigger operations to the target application program to execute operations corresponding to the plurality of trigger operations by the target application program.

[0009] According to a second aspect of the present disclosure, a device for generating a combination instruction in an application operation process is provided, comprising:

[0010] a generating unit for generating an instruction recording layer corresponding to a target application program, the instruction recording layer covering a target interface corresponding to the target application program;

[0011] a receiving unit for receiving a plurality of trigger operations for the target interface through the instruction recording layer, and generating a combination instruction corresponding to the target interface according to a trigger order and a trigger position of the plurality of trigger operations;

[0012] The issuing unit issues the plurality of trigger operations to the target application program, so that the target application program performs operations corresponding to the plurality of trigger operations.

[0013] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0014] a processor;

[0015] a memory for storing processor-executable instructions;

[0016] The processor implements the method according to the first aspect by running the executable instructions.

[0017] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, which stores computer instructions. The instructions are executed by a processor to implement the steps of the method according to the first aspect.

[0018] In the technical solution of the present disclosure, when it is determined that the user needs to generate a combined instruction for the target application program, an instruction record layer is generated and overlaid on the target interface corresponding to the target application program. The instruction record layer is used to receive a plurality of trigger operations of the user for the target application program. On the one hand, the plurality of trigger operations are recorded for generating a corresponding combined instruction; on the other hand, the trigger information of the plurality of trigger operations is issued to the target application program, so that the target application program can perform corresponding operations according to the trigger operations of the user. It can be seen that, by generating the instruction record layer, the present disclosure can generate a combined instruction for the target application program on the premise that the trigger operations of the user are normally responded by the target application program.

[0019] It should be understood that in related technologies, accessibility services are enabled, allowing other applications to passively receive user-triggered operations on the target application. Since this process is not considered a consumption of a single trigger operation, the trigger operation is consumed by the target application while simultaneously being recorded by other applications. This effectively circumvents the mechanism in the touch field where a user's trigger at a software layer can only be consumed by one application. In this disclosure, however, an additional command recording layer is generated to receive user-triggered operations on the target interface. The trigger information is recorded to generate combined commands and also sent to the target application so that it can trigger the corresponding location on the target interface. This method is equivalent to the user triggering the operation separately at the command recording layer and at the software layer corresponding to the target interface. The trigger at the command recording layer is recorded to generate combined commands, while the trigger at the target interface is consumed by the target application to execute the corresponding operation. In other words, this disclosure is equivalent to creating an additional software layer to add a consumable trigger operation. Without violating the aforementioned consumption mechanism, it satisfies the need to record the trigger operation while the user is operating the target application, so as to generate combined instructions. This avoids the user privacy leakage caused by the need to enable accessibility services in related technologies to generate combined instructions. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0021] Figure 1 This is a flowchart illustrating an exemplary embodiment of the present disclosure of a method for generating combined instructions during application operation;

[0022] Figure 2 This is a flowchart illustrating another method for generating combined instructions during application operation, as shown in an exemplary embodiment of this disclosure;

[0023] Figure 3 This is a schematic diagram illustrating an instruction recording layer according to an exemplary embodiment of the present disclosure;

[0024] Figure 4 This is a block diagram illustrating an apparatus for generating combined instructions during application operation, as shown in an exemplary embodiment of this disclosure;

[0025] Figure 5 This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0026] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The following description is made with reference to the accompanying drawings in which like reference numerals refer to like elements, unless the context clearly shows otherwise. The following description of exemplary embodiments is not representative of all embodiments consistent with the present disclosure. Rather, it is merely an example of apparatus and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0027] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0028] It will be understood that, although the terms first, second, third, etc. can be used herein to describe various information, these terms are not intended to denote a particular order or hierarchy. These terms are used only to distinguish one from another. For example, a first information can be termed a second information, and, similarly, a second information can be termed a first information, without departing from the scope of the present disclosure. Depending on the context, the word "if' as used herein can be interpreted as meaning "when" or "in response to determining."

[0029] The accessibility service was initially introduced to make it easier for people with disabilities to use electronic devices. After an electronic device enables the accessibility service, applications installed on the electronic device can obtain a large number of permissions to help people with disabilities use the electronic device using the obtained permissions. For example, an application can obtain a permission to detect a user's trigger operation and a permission to voice broadcast to help a blind person broadcast the type and function of any operation control when the blind person triggers the operation control.

[0030] As user needs continue to increase, the accessibility service is gradually applied to other fields in addition to the above functions. For example, to make it easier for ordinary users to operate, a method of generating a combined instruction using the accessibility service is proposed in the related art.

[0031] In actual operation, when the accessibility service is started, the service broadcasts the detected trigger operation (or touch event) to each application. At this time, the application responsible for generating the combined instruction can record the multiple trigger operations of the user on the interface corresponding to any other application, and then generate a combined instruction corresponding to the any application. Therefore, when the user needs to perform repeated operations on the any application next time, the user only needs to initiate an execution instruction for the generated combined instruction to complete the above-mentioned multiple trigger operations, thereby simplifying the user's operation.

[0032] It can be understood that, in the above process of generating the combined instruction, the accessibility service needs to be started so that the application responsible for generating the combined instruction can obtain the multiple trigger operations of the user on any application. However, after the accessibility service is started, other permissions except the broadcast instruction are also opened to each application, resulting in leakage of user privacy.

[0033] In order to avoid the leakage of user privacy, the present disclosure provides a method for generating a combined instruction during application operation, so as to generate a combined instruction for any application without starting the accessibility service.

[0034] Figure 1 A method for generating a combined instruction during application operation is shown in an exemplary embodiment of the present disclosure. As shown in Figure 1 The method can include the following steps:

[0035] Step 102, generating an instruction recording layer corresponding to a target application, the instruction recording layer covering a target interface corresponding to the target application.

[0036] In the present disclosure, the application that needs to generate a combined instruction is referred to as a target application, and the interface corresponding to the target application is referred to as a target interface. As known from the above, the related art needs to start the accessibility service when a combined instruction for the target application is needed, so as to broadcast the trigger operation of the user on the target interface to the application responsible for generating the combined instruction. Since the accessibility service opens a large number of permissions to each application, it is easy to cause the problem of leakage of user privacy.

[0037] Therefore, the present disclosure proposes a method for generating a combined instruction without starting the accessibility service. Before introducing the present solution, the technical basis of the present solution is first introduced:

[0038] In the field of touch control, in the hardware layer, the touch control operation in the electronic device is detected by the touch screen and reported to the operating system; and in the software layer, all trigger operations are detected by the software layer, which is also called View. In actual application, the trigger operation detected by any View is only allowed to be actively acquired and used by one application program (which can also be regarded as being consumed by the application program, and in the following text, this mechanism is referred to as the consumption mechanism of the trigger operation). For example, the above target interface can be regarded as a View. When a trigger operation is detected in the target interface, if the trigger operation is acquired by the target application program to perform a corresponding operation, the trigger operation cannot be acquired and used by other applications.

[0039] Under this mechanism, since the related technology broadcasts the detected trigger operation to each application program by the accessibility service, each application program passively receives the trigger operation, which is not regarded as the consumption of the trigger operation, so it does not violate the mechanism. In the present disclosure, in order to generate a combined instruction without starting the accessibility service, an additional View specially used for generating a combined instruction is generated, which is referred to as the instruction recording layer in the present disclosure.

[0040] After generating the instruction recording layer, the present disclosure can overlay it on the target interface to receive the trigger operation of the user for the target interface. Since the trigger operation cannot penetrate the View in the software layer, when the user triggers on the touch screen, only the instruction recording layer overlaid on the target interface can receive the trigger operation. In order to enable the target application program to detect the trigger operation to perform a corresponding operation, after the instruction recording layer receives the trigger operation and sends the trigger information of the trigger operation to the operating system, the trigger information is recorded on one hand to generate a combined instruction; on the other hand, the trigger information of the trigger operation is re-issued to the target application program to determine the trigger position on the target interface corresponding to the trigger operation by the target application program, and execute the operation instruction corresponding to the trigger position (or respond to the trigger operation), which is equivalent to performing a trigger operation on the View of the target interface again. In other words, this method is equivalent to the user performing a trigger operation on two Views of the instruction recording layer and the target interface, respectively. The trigger operation in the instruction recording layer is used to generate a combined instruction, and the trigger operation in the target interface is used to instruct the target application program to perform a corresponding operation.

[0041] As can be seen, through the technical solution of the present disclosure, a corresponding combined instruction can be generated according to the trigger operation of the user when the user operates the target application program. Neither does it violate the consumption mechanism in the field of touch control, nor does it need to start the accessibility service, avoiding the privacy leakage caused by the need to start the accessibility service in the related technology.

[0042] In the present disclosure, the user can convey the indication of the need to generate the combined instruction to the electronic device in various ways.

[0043] In an embodiment, a combined instruction generation control can be preset in the target interface, and the user can trigger the combined instruction generation control to inform the electronic device of the need to generate the combined instruction for the target application program. Then, the electronic device can generate the instruction record layer corresponding to the target application program for generating the combined instruction after detecting the triggering operation.

[0044] In another embodiment, a condition can be set for the electronic device to determine by itself whether the user needs to generate the combined instruction. For example, the frequency of the user triggering the touch screen can be used as the basis for the determination, and specifically, a preset value of a time interval can be preset to determine whether the time interval between the triggering operations at multiple positions in the target interface is less than the preset value. If yes, it is determined that the user needs to generate the combined instruction, and the instruction record layer corresponding to the target application program is generated for generating the combined instruction.

[0045] In yet another embodiment, the indication of the generation of the combined instruction can be bound to any gesture instruction, so that the electronic device generates the instruction record layer corresponding to the target application program when detecting the any gesture instruction. In the present disclosure, the any gesture instruction can be referred to as a preset gesture instruction corresponding to the combined instruction generation operation. For example, the preset gesture instruction can be a sliding gesture, a flipping gesture, a tilting gesture, etc., which can be determined by those skilled in the art according to actual needs, and the present disclosure does not limit this.

[0046] In step 104, multiple triggering operations for the target interface are received through the instruction record layer, and a combined instruction corresponding to the target interface is generated according to the triggering order and the triggering positions of the multiple triggering operations.

[0047] In the present disclosure, a coordinate system can be established for recording the triggering operations of the user for the target interface. Specifically, when the instruction record layer detects multiple triggering operations, the multiple triggering coordinates corresponding to the multiple triggering operations can be determined based on the above-mentioned coordinate system, and the multiple triggering coordinates can be recorded according to the triggering order of the multiple triggering operations as the combined instruction corresponding to the target application program.

[0048] It should be understood that after the combination instruction is generated, the combination instruction can be triggered to be completed when it is required to repeatedly perform a plurality of trigger operations consistent with the combination instruction. In actual operation, the combination instruction can be triggered in various ways. For example, the combination instruction can be bound to any gesture instruction, so that the user can trigger the combination instruction through the corresponding gesture; for example, the combination instruction can also be generated in the target interface in the form of a corresponding combination instruction execution control, so that the user can trigger the combination instruction through the combination instruction execution control. How to trigger the generated combination instruction can be determined by those skilled in the art according to actual needs.

[0049] In actual application, the actual operation instruction usually needs to be executed by the corresponding application program. Therefore, in the present disclosure, when the execution indication for the generated combination instruction is received, the combination instruction can be issued to the corresponding target application program, so that the target application program triggers at the corresponding position of the target interface according to the trigger position and the trigger sequence contained in the combination instruction. It needs to be declared that the combination instruction records the trigger positions of a plurality of trigger operations and the trigger sequence between the plurality of trigger positions. Therefore, after receiving the combination instruction, the target application program triggers each trigger position recorded in the combination instruction in turn according to the trigger sequence recorded in the combination instruction, so as to ensure that the plurality of trigger operations performed by the user for the first time can be completely executed again.

[0050] Step 106, issuing the plurality of trigger operations to the target application program to execute the operations corresponding to the plurality of trigger operations by the target application program.

[0051] It should be emphasized that the present disclosure generates a combined instruction based on a plurality of trigger operations of a user during the operation of the target interface by the user. Therefore, as described above, in addition to the need to record the trigger positions and trigger sequences of the plurality of trigger operations, the trigger information of the trigger operations is also issued to the target application to ensure that the target application can perform the corresponding operation. Specifically, after the plurality of trigger operations are issued to the target application, the target application can determine the positions corresponding to the trigger positions of each trigger operation from the target interface and perform the operations corresponding to the plurality of positions. Of course, in actual operation, in order to ensure that the target application can respond to the plurality of trigger operations in time, the instruction recording layer will upload any trigger operation to the operating system after receiving it. After receiving the any trigger operation, the operating system will record the trigger information (mainly the trigger position) of the any trigger operation, or send it to the third-party application responsible for generating the combined instruction for recording. On the other hand, the operating system will also issue it to the target application so that the target application can determine the position to be triggered by the user in the target interface according to the trigger information, and then perform the corresponding instruction according to the determined position. Further, after the operating system or the third-party application records the trigger information of the plurality of trigger operations, the combined instruction corresponding to the plurality of trigger operations can be generated.

[0052] In the present disclosure, the instruction recording layer can be generated by the operating system of the electronic device, in other words, the instruction recording layer is a system-level instruction recording layer; or the instruction recording layer can be generated by a third-party application that is different from the target application, which means that the third-party application is responsible for generating the instruction recording layer. It should be understood that whether the instruction recording layer is generated by the operating system or the third-party application, the combined instruction can be generated while the user is operating the target interface, which does not affect the normal response of the target application to the user operation, and can generate the combined instruction for the continuous operation that needs to be repeated.

[0053] In the present disclosure, in order to make each position on the above-mentioned instruction recording layer correspond to each position in the target interface, and further more conveniently determine which position on the target interface the user aims at according to the trigger operation received on the instruction recording layer, the size of the instruction recording layer can be designed to be consistent with the target interface.

[0054] Correspondingly, in order not to hinder the trigger operation of the user on the target interface, the generated instruction recording layer can be a transparent layer, so that the user can directly see each operation control on the target interface through the instruction recording layer. In other words, designing the instruction recording layer as a transparent layer can enable the user to operate the target interface without feeling the instruction recording layer and generate the corresponding combined instruction.

[0055] It should be noted that the technical solution of the present disclosure can be applied to any type of electronic device, for example, the electronic device can be a mobile terminal such as a smart phone, a tablet computer, etc., or a fixed terminal such as a smart television, a PC (Personal Computer), etc. It should be understood that as long as the electronic device is controlled by the user's touch operation, it can be used as the electronic device applied by the present disclosure, and the specific type of electronic device to which the present disclosure is applied can be determined by the person skilled in the art according to the actual needs, and the present disclosure does not limit it.

[0056] From the above technical solution, it can be known that the present disclosure can generate an instruction recording layer when it is determined that the user needs to generate a combination instruction for the target application program, and overlay it on the target interface corresponding to the target application program. On this basis, the user's multiple trigger operations for the target application program can be received through the instruction recording layer. On the one hand, the corresponding combination instruction can be generated by recording the multiple trigger operations; on the other hand, the target application program can perform the corresponding operation according to the user's trigger operation by issuing the trigger information of the multiple trigger operations to the target application program. It can be seen that the present disclosure generates an instruction recording layer, so that the present disclosure can ensure that the user's trigger operation is normally responded while generating the corresponding combination instruction.

[0057] It should be understood that in the related art, the other application program can passively receive the user's trigger operation for the target application program by starting the accessibility service. Since this process is not considered as the consumption of a trigger operation, the trigger operation can be consumed by the target application program while being recorded by the other application program. It is equivalent to avoiding the mechanism in the touch field that the trigger of a user on a software layer can only be consumed by one application program by starting the accessibility service. In the present disclosure, the trigger operation of the user for the target interface is received by generating an additional instruction recording layer, and the trigger information of the trigger operation is recorded on the one hand to generate a combination instruction, and on the other hand is issued to the target application program to trigger the corresponding position of the target interface. This method is equivalent to that the user triggers once on the software layer corresponding to the instruction recording layer and the target interface respectively. The trigger on the instruction recording layer is recorded to generate a combination instruction, and the trigger in the target interface is consumed by the target application program to perform the corresponding operation. In other words, the present disclosure is equivalent to creating an additional software layer to increase a trigger operation that can be consumed, so as to meet the need of recording the trigger operation for generating a combination instruction without violating the above consumption mechanism, and avoid the user privacy leakage caused by starting the accessibility service to generate a combination instruction in the related art.

[0058] In addition, the disclosure records the multiple trigger operations of the user for the target interface by using the generated instruction recording layer, and further issues the trigger information of the trigger operation to the target application, which is equivalent to triggering on the software layer of the target interface. In the field of touch, in the multiple software layers overlapped with each other, only the uppermost software layer can receive a trigger operation, and the disclosure can make the overlapped software layer also receive the trigger operation by the above-mentioned method of re-issuing the trigger information, which is equivalent to providing a penetration method of the trigger operation.

[0059] Next, taking the generation of the combined instruction by the user in the process of playing the mobile game as an example, the technical solutions of the disclosure are introduced.

[0060] Figure 2 Another method for generating a combined instruction in the process of application operation is shown in an exemplary embodiment of the disclosure. As shown in Figure 2 The method can include the following steps:

[0061] Step 201, starting the mobile game APP when detecting the trigger operation of the user for the icon of the mobile game APP.

[0062] In this embodiment, the user can pre-install the mobile game APP which is often played by himself on the smart phone. Then, when the user wants to play the mobile game, the mobile game APP can be started by triggering the icon of the mobile game APP.

[0063] Step 202, displaying the mobile game interface corresponding to the mobile game APP.

[0064] In this embodiment, after the mobile game APP is started, the operation interface corresponding to the mobile game APP can be displayed.

[0065] Taking the mobile game as a shooting game M for example, after the mobile game M is started, the preparation interface corresponding to the mobile game M can be displayed. After the user finishes the preparation, the operation control of starting the game can be triggered to enter the game interface. In the game interface of the shooting game, multiple operation controls such as the moving control, the shooting control, and the opening control are usually included. At this time, the user may have the need to generate a combined instruction for the convenience of operation. For example, the user may have the need to combine the opening control and the shooting control into the combined instruction of “opening + shooting” for the convenience and rapidity of shooting.

[0066] Step 203, generating the instruction recording layer corresponding to the mobile game APP when detecting the preset gesture of the user.

[0067] In this embodiment, the function of generating the combined instruction can be bound to any gesture instruction in advance, so that the user can enable the function of "generating the combined instruction" through the gesture instruction when the user needs to generate the combined instruction. For example, the flip gesture of the mobile phone can be bound to the function of generating the combined instruction in advance, so as to enable the function of generating the combined instruction.

[0068] Taking the above example, after the user enters the performing interface of the mobile game M, if the user wants to generate a combined instruction, the user can enable the function of generating the combined instruction by flipping the mobile phone. After the function is enabled, the corresponding instruction recording layer can be generated to receive the trigger operation of the user on the performing interface.

[0069] For example, the generated instruction recording layer can be as shown in Figure 3 The lower layer is the performing interface corresponding to the mobile game M, and the upper layer is the generated instruction recording layer. In the performing interface, the "opening lens" control 31 and the "shooting" control 32 are included. It should be noted that other operation controls can also be included in the actual performing interface. Since this embodiment takes the generation of the combined instruction of "opening lens + shooting" as an example, only the related controls are shown in the figure.

[0070] Step 204, in the case of detecting the trigger operation of the user on the mobile game interface, the trigger operation of the user is received through the instruction recording layer.

[0071] In this embodiment, the instruction recording layer is usually designed as a transparent layer, so that the user can directly see the operation controls in the mobile game performing interface through the instruction recording layer.

[0072] In actual operation, after the instruction recording layer receives the trigger operation of the user, the operation system processes it to realize different functions. In this embodiment, the third-party application N is responsible for generating the combined instruction, so the operation system sends the trigger operation received by the instruction recording layer to the third-party application N to record and generate the corresponding combined instruction, and on the other hand, the operation system sends the trigger operation to the APP of the mobile game M to make it perform the corresponding operation.

[0073] Step 205A, the third-party application records the received trigger operation.

[0074] Step 206A, the third-party application generates the combined instruction corresponding to the mobile game APP based on the recorded multiple trigger operations.

[0075] The embodiment corresponds to a third-party application specially pre-installed for generating the combined instruction. It should be noted that the embodiment is only used as an example in this case, and in actual operation, the operating system can also generate the above instruction record layer at the system level for generating the combined instruction. The specific way is not limited by the disclosure.

[0076] In step 205B, the trigger operation received by the instruction record layer is sent to the mobile game APP.

[0077] In step 206B, the mobile game APP performs the corresponding operation based on the received trigger operation.

[0078] It should be noted that after the instruction record layer receives any trigger operation, the operating system will not only send the trigger information of the any trigger operation to the third-party application N, but also send the any trigger operation to the APP of the mobile game M in real time, so as to respond in real time and ensure the normal operation of the mobile game.

[0079] Based on the above example, after the function of generating the combined instruction is started, when the user triggers the "open lens" control in the above performing interface (in the user's view, the "open lens" control is triggered, and the trigger operation is actually received by the generated instruction record layer, such as the user actually triggering the A area in the instruction record layer), the operating system sends the trigger operation to the third-party application N for recording on the one hand, and sends it to the APP of the mobile game M on the other hand, so as to determine that the user triggers the "open lens" control according to the trigger position corresponding to the trigger operation, and then display the picture under the open lens scene in the performing interface. Subsequently, when the user further triggers the "shooting" control in the above performing interface (similarly, in the user's view, the "shooting" control is triggered, and the trigger operation is actually received by the generated instruction record layer, such as the user actually triggering the B area in the instruction record layer), the operating system sends the trigger operation to the third-party application N for recording on the one hand, and sends it to the APP of the mobile game M on the other hand, so as to determine that the user triggers the "shooting" control according to the trigger position corresponding to the trigger operation, and then display the shooting special effect in the performing interface.

[0080] After the above actions are completed, the user can flip the smart phone again, and the operating system can send the corresponding indication information to the third-party application N after detecting the gesture instruction, so that the third-party application N generates the "open lens + shooting" combined instruction according to the trigger information related to the "open lens" control and the trigger information related to the "shooting" control which have been recorded.

[0081] In actual application, the execution control corresponding to the "open lens + shooting" combined instruction can be further generated in the above performing interface, so that the user can trigger the "open lens + shooting" two instructions at one key.

[0082] According to the technical solution, the application program in the smart phone can generate a combined instruction by generating an instruction recording layer, so as to simplify the operation of the user. After receiving the trigger operation, the trigger information of the trigger operation is sent to the third-party application program, so that the third-party application program generates a combined instruction according to the recorded trigger information, and the trigger information is also sent to the target application program, so that the target application program performs a corresponding response. It can be seen that, by using the technical solution of the present disclosure, a combined instruction can be generated without starting the accessibility service, thereby avoiding the problem of privacy leakage caused by starting the accessibility service in the related art.

[0083] Figure 4 is a block diagram of an apparatus for generating a combined instruction in an application operation process according to an example embodiment of the present disclosure. Referring to Figure 4 The apparatus includes a generating unit 401, a receiving unit 402, and a sending unit 403.

[0084] The generating unit 401 generates an instruction recording layer corresponding to a target application program, and the instruction recording layer is overlaid on a target interface corresponding to the target application program.

[0085] The receiving unit 402 receives a plurality of trigger operations for the target interface through the instruction recording layer, and generates a combined instruction corresponding to the target interface according to a trigger sequence and a trigger position of the plurality of trigger operations.

[0086] The sending unit 403 sends the plurality of trigger operations to the target application program, so that the target application program performs an operation corresponding to the plurality of trigger operations.

[0087] Optionally, the generating unit 401 is further configured to:

[0088] generate the instruction recording layer corresponding to the target application program in a case where a combined instruction generation control in the target interface is detected to be triggered; or

[0089] generate the instruction recording layer corresponding to the target application program in a case where a time interval at which a plurality of positions in the target interface are detected to be triggered is less than a preset value; or

[0090] generate the instruction recording layer corresponding to the target application program in a case where a preset gesture instruction is detected.

[0091] Optionally, the receiving unit 402 is further configured to:

[0092] In a case that a plurality of trigger operations are detected on the instruction recording layer, a plurality of trigger coordinates corresponding to the plurality of trigger operations respectively are determined based on a coordinate system corresponding to the target interface;

[0093] The plurality of trigger coordinates are recorded in a trigger sequence of the plurality of trigger operations as a combined instruction corresponding to the target interface.

[0094] Optionally, the issuing unit 403 is further configured to:

[0095] The plurality of trigger operations are issued to the target application program, so that the target application program determines a position in the target interface corresponding to a trigger position of each trigger operation and performs an operation corresponding to the position.

[0096] Optionally, the issuing unit 403 is further configured to:

[0097] In a case that an execution instruction for the combined instruction is received, the combined instruction is issued to the target application program, so that the target application program triggers at a corresponding position of the target interface according to the trigger position and the trigger sequence of the plurality of trigger operations.

[0098] Optionally, the instruction recording layer is a system-level instruction recording layer; or,

[0099] The instruction recording layer is generated by a third-party application program which is different from the target application program.

[0100] Optionally, the instruction recording layer is consistent with a size of the target interface.

[0101] Optionally, the instruction recording layer is a transparent layer.

[0102] For the device embodiment, since it basically corresponds to the method embodiment, the related parts can be referred to the part of the method embodiment. The device embodiment described above is merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. According to actual needs, part or all of the modules can be selected to achieve the purpose of the present disclosure. Those skilled in the art can understand and implement it without creative labor.

[0103] Accordingly, the disclosure also provides an apparatus for generating a combined instruction in an application operation process, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement a method for generating a combined instruction in an application operation process as described in any of the above embodiments, such as the method can comprise: generating an instruction recording layer corresponding to a target application program, the instruction recording layer covering a target interface corresponding to the target application program; receiving a plurality of trigger operations for the target interface through the instruction recording layer, and generating a combined instruction corresponding to the target interface according to the trigger order and trigger position of the plurality of trigger operations; and issuing the plurality of trigger operations to the target application program to execute operations corresponding to the plurality of trigger operations by the target application program.

[0104] Accordingly, the disclosure also provides an electronic device, comprising a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, and the one or more programs contain instructions for implementing a method for generating a combined instruction in an application operation process as described in any of the above embodiments, such as the method can comprise: generating an instruction recording layer corresponding to a target application program, the instruction recording layer covering a target interface corresponding to the target application program; receiving a plurality of trigger operations for the target interface through the instruction recording layer, and generating a combined instruction corresponding to the target interface according to the trigger order and trigger position of the plurality of trigger operations; and issuing the plurality of trigger operations to the target application program to execute operations corresponding to the plurality of trigger operations by the target application program.

[0105] Figure 5 is a block diagram of an apparatus 500 for implementing a process scheduling method according to an exemplary embodiment. For example, the apparatus 500 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0106] Referring to Figure 5 , the apparatus 500 can include one or more of the following components: a processing component 502, a memory 504, a power supply component 506, a multimedia component 508, an audio component 510, an input / output (I / O) interface 512, a sensor component 514, and a communication component 516.

[0107] The processing component 502 generally controls the overall operations of the device 500, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 502 can include one or more processors 520 to execute instructions delivered from the memory 504 to complete all or part of the steps of the methods described above. In addition, the processing component 502 can include one or more modules to facilitate the interaction between the processing component 502 and other components. For example, the processing component 502 can include a multimedia module to facilitate the interaction between the multimedia component 508 and the processing component 502.

[0108] The memory 504 is configured to store various types of data to support operations of the device 500. Examples of these data include instructions for any application or method operating on the device 500, contact data, phonebook data, messages, pictures, videos, and so on. The memory 504 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0109] The power component 506 provides power to the various components of the device 500. The power component 506 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 500.

[0110] The multimedia component 508 includes a screen providing an output interface between the device 500 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes the touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensors can not only sense a boundary of a touching or sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 508 includes a front camera and / or a back camera. The front camera and / or the back camera can receive external multimedia data when the device 500 is in an operating mode, such as a shooting mode or a video mode. Each of the front and back cameras can be a fixed optical lens system or have a focal length and optical zoom capability.

[0111] The audio component 510 is configured to output and / or input audio signals. For example, the audio component 510 includes a microphone (MIC) that is configured to receive an external audio signal when the device 500 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 504 or transmitted via the communication component 516. In some embodiments, the audio component 510 also includes a speaker for outputting audio signals.

[0112] The I / O interface 512 provides an interface between the processing component 502 and peripheral interface modules, which can be a keypad, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0113] The sensor component 514 includes one or more sensors for providing status assessments of various aspects of the device 500. For example, the sensor component 514 can detect an open / closed position of the device 500, relative positioning of components, such as a display and a keypad of the device 500, a change of position of the device 500 or a component of the device 500, presence or absence of user contact with the device 500, a change in orientation or acceleration / deceleration of the device 500, and a temperature change of the device 500, among other possibilities. The sensor component 514 can include a proximity sensor configured to detect presence of an object in proximity to the device 500 without any physical touch. The sensor component 514 can also include a light sensor (e.g., a CMOS or CCD image sensor) configured to work in an imaging application. In some embodiments, the sensor component 514 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0114] The communication component 516 is configured to facilitate wired or wireless communication between the device 500 and another device. The device 500 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR (New Radio), or a combination thereof. In an example embodiment, the communication component 516 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 516 also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technology.

[0115] In exemplary embodiments, the apparatus 500 can be implemented using one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic devices, for performing the above methods.

[0116] In exemplary embodiments, a non-transitory computer readable storage medium including instructions, such as the memory 504 including instructions, is also provided, which can be executed by the processor 520 of the apparatus 500 to complete the above methods. For example, the non-transitory computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0117] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure following, in general, the principles of the present disclosure and including such features that are evident to those skilled in the art to which the present disclosure pertains. The specification and examples are to be regarded as illustrative only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0118] It should be understood that the present disclosure is not limited to the precise structures herein described and illustrated above, and that various modifications and changes in the herein disclosed structures can be affected, without departing from the scope of the present disclosure. The scope of the present disclosure is indicated by the appended claims, and all changes that come within the meaning and range of equivalents are intended to be embraced therein.

[0119] The above-described embodiments are merely possible implementations of the present disclosure, and are not intended to limit the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure should be included in the scope of the present disclosure.

Claims

1. A method of generating a combined instruction during application operation, characterized by, The method comprises the following steps: generating an instruction recording layer corresponding to a target application, the instruction recording layer covering a target interface corresponding to the target application; receiving a plurality of trigger operations for the target interface through the instruction recording layer; generating a combined instruction corresponding to the target interface according to a trigger sequence and trigger positions of the plurality of trigger operations, and issuing the plurality of trigger operations to the target application in real time to execute operations corresponding to the plurality of trigger operations by the target application, when the plurality of trigger operations are received.

2. The method of claim 1, wherein, The generating an instruction recording layer corresponding to a target application comprises: generating an instruction recording layer corresponding to a target application when a combined instruction generation control in the target interface is detected to be triggered; or generating an instruction recording layer corresponding to a target application when a time interval at which a plurality of positions in the target interface are detected to be triggered is less than a preset value; or generating an instruction recording layer corresponding to a target application when a preset gesture instruction is detected.

3. The method of claim 1, wherein: the receiving a plurality of trigger operations for the target interface through the instruction recording layer comprises: determining a plurality of trigger coordinates corresponding to the plurality of trigger operations respectively based on a coordinate system corresponding to the target interface when the plurality of trigger operations are detected in the instruction recording layer; and the generating a combined instruction corresponding to the target interface according to a trigger sequence and trigger positions of the plurality of trigger operations comprises: recording the plurality of trigger coordinates as a combined instruction corresponding to the target interface according to the trigger sequence of the plurality of trigger operations.

4. The method of claim 1, wherein, The issuing the plurality of trigger operations to the target application to execute operations corresponding to the plurality of trigger operations by the target application comprises: issuing the plurality of trigger operations to the target application to determine a position in the target interface corresponding to a trigger position of each trigger operation and execute an operation corresponding to the position by the target application.

5. The method of claim 1, wherein, The method further comprises: issuing the combined instruction to the target application to trigger the target application at a corresponding position of the target interface according to the trigger positions and the trigger sequence of the plurality of trigger operations when an execution instruction for the combined instruction is received.

6. The method of claim 1, wherein: the instruction recording layer is a system-level instruction recording layer; or the instruction recording layer is generated by a third-party application distinguished from the target application.

7. The method of claim 1, wherein, The instruction recording layer is consistent in size with the target interface.

8. The method of claim 1, wherein, The instruction recording layer is a transparent layer.

9. An apparatus for generating a combined instruction during application operation, the apparatus comprising: The method comprises the following steps: generating an instruction recording layer corresponding to a target application, the instruction recording layer covering a target interface corresponding to the target application; receiving a plurality of trigger operations for the target interface through the instruction recording layer; In a case that the plurality of trigger operations are received, the receiving unit is further configured to generate a combined instruction corresponding to the target interface according to a trigger sequence and a trigger position of the plurality of trigger operations, and the delivering unit is configured to deliver the plurality of trigger operations to the target application in real time, so that the target application performs operations corresponding to the plurality of trigger operations.

10. An electronic device, comprising: Comprise: a processor; a memory for storing processor-executable instructions; wherein the processor implements the method of any one of claims 1-8 by running the executable instructions.

11. A computer readable storage medium having stored thereon computer instructions, wherein, The instructions, when executed by the processor, implement the steps of the method of any one of claims 1-8.

Images