Application operation method and device

Abstract

The invention discloses an application operation method and device, and belongs to the technical field of artificial intelligence. The method comprises the steps that a first interface is displayed, the first interface comprises an operation link corresponding to a first message, the operation link is generated based on the message content of the first message, and the first message is a session message; receiving a first input; in response to the first input, executing a first operation corresponding to the operation link; and replying the first message based on an operation result corresponding to the first operation.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and specifically relates to an application operation method and device. Background Art

[0002] Currently, with the development of communication technology, electronic devices have more and more functions. For example, in order to allow users to use electronic devices more conveniently, electronic devices can provide accessibility assistance functions.

[0003] In the related art, when an electronic device has an accessibility function turned on, if the electronic device receives a message, the electronic device can voice broadcast the message content so that a visually impaired user can know the message content.

[0004] However, when the above message instructs the visually impaired user to perform certain complex operations, for example, instructing the visually impaired user to complete the submission operation in application A, since the visually impaired user cannot clearly see or see the information displayed on the electronic device, the visually impaired user may not be able to complete the above complex operations through the electronic device, for example, reply to the message. Therefore, how the visually impaired user can conveniently perform operations through the electronic device is an urgent problem to be solved. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide an application operation method and device, which can enable visually impaired users to conveniently perform operations through electronic devices.

[0006] In a first aspect, an embodiment of the present application provides an application operation method, which includes: displaying a first interface, the first interface including an operation link corresponding to a first message, the operation link being generated based on the message content of the first message, wherein the first message is a conversation message; receiving a first input; in response to the first input, executing a first operation corresponding to the operation link; and replying to the first message based on the operation result corresponding to the first operation.

[0007] In a second aspect, embodiments of the present application provide an application operation device, comprising: a display module, a receiving module, and a processing module. The display module is configured to display a first interface, wherein the first interface includes an operation link corresponding to a first message, the operation link being generated based on the message content of the first message, wherein the first message is a conversation message. The receiving module is configured to receive a first input. The processing module is configured to, in response to the first input received by the receiving module, execute a first operation corresponding to the operation link; and reply to the first message based on an operation result corresponding to the first operation.

[0008] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method described in the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the method described in the first aspect.

[0012] In an embodiment of the present application, a first interface is displayed, which includes an operation link corresponding to a first message, the operation link being generated based on the message content of the first message, wherein the first message is a conversation message; then, a first input is received; then, in response to the first input, a first operation corresponding to the operation link is performed; and finally, based on the operation result corresponding to the first operation, a reply is made to the first message. In this solution, since the operation link can be automatically generated based on the message content of the first message, a visually impaired user only needs one input to trigger the electronic device to immediately perform the first operation corresponding to the first message and automatically reply to the first message based on the operation result of the first operation. This simplifies the process of visually impaired users performing complex operations on electronic devices and the process of replying to messages, thereby allowing visually impaired users to conveniently perform operations on electronic devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is one of the flow charts of an application operation method provided in an embodiment of the present application;

[0014] Figure 2 This is one of the example schematic diagrams of a conversation interface provided in an embodiment of the present application;

[0015] Figure 3 This is the second flowchart of an application operation method provided in an embodiment of the present application;

[0016] Figure 4A This is one of the example schematic diagrams of an accessibility setting interface provided in an embodiment of the present application;

[0017] Figure 4BThis is a second example schematic diagram of an accessibility setting interface provided in an embodiment of the present application;

[0018] Figure 5 This is the third flowchart of an application operation method provided in an embodiment of the present application;

[0019] Figure 6 This is a second example diagram of a conversation interface provided in an embodiment of the present application;

[0020] Figure 7 This is the fourth flowchart of an application operation method provided in an embodiment of the present application;

[0021] Figure 8 This is a third example diagram of a conversation interface provided in an embodiment of the present application;

[0022] Figure 9 This is a fourth example diagram of a conversation interface provided in an embodiment of the present application;

[0023] Figure 10 This is a fifth example diagram of a conversation interface provided in an embodiment of the present application;

[0024] Figure 11 This is the fifth flow chart of an application operation method provided in an embodiment of the present application;

[0025] Figure 12 This is the sixth example diagram of a conversation interface provided in an embodiment of the present application;

[0026] Figure 13 This is the seventh example diagram of a conversation interface provided in an embodiment of the present application;

[0027] Figure 14 This is the sixth flowchart of an application operation method provided in an embodiment of the present application;

[0028] Figure 15 This is the seventh flowchart of an application operation method provided in an embodiment of the present application;

[0029] Figure 16 This is one of the example schematic diagrams of a resume interface provided in an embodiment of the present application;

[0030] Figure 17 This is the eighth flowchart of an application operation method provided in an embodiment of the present application;

[0031] Figure 18 This is the second example diagram of a resume interface provided in an embodiment of the present application;

[0032] Figure 19 This is the third example diagram of a resume interface provided in an embodiment of the present application;

[0033] Figure 20 This is a structural diagram of an application operating device provided in an embodiment of the present application;

[0034] Figure 21 This is one of the hardware structure diagrams of an electronic device provided in an embodiment of the present application;

[0035] Figure 22 This is the second hardware structure diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The following will be combined with the accompanying drawings in the embodiments of this application to clearly describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0037] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0038] The terms "at least one" and "at least one of" in the specification and claims of this application refer to any one, any two, or a combination of more than two of the objects included. For example, at least one of a, b, and c can be represented by: "a", "b", "c", "a and b", "a and c", "b and c", and "a, b, and c", where a, b, and c can be single or multiple. Similarly, "at least two" means two or more, and its meaning is similar to "at least one".

[0039] The following is a detailed explanation of the professional terms provided in the embodiments of this application.

[0040] Action chaining: This refers to linking multiple steps or tasks together to form an automated process. For example, a trigger condition: When event A, such as receiving an email, occurs, action B, such as saving an attachment to a cloud drive, is automatically triggered. Chaining services: This allows data flow between different platforms, such as Excel and email, by linking functions.

[0041] Atomic Instruction: An atomic instruction is a term widely used in fields such as computer science, system design, and microservices architecture. Its core meaning is to break down a complex operation into the smallest, indivisible independent units. Each instruction has independence, integrity, and indivisibility, and can be executed alone or combined into larger operation instructions.

[0042] Grammatical Structure: Grammatical structure refers to the ways and rules of organizing words, phrases, or sentences in a language. Specifically, it includes the following types: Coordinate Phrase: There is a coordinate relationship between words. Modifying Phrase: The preceding word modifies or restricts the following noun, verb, or adjective. Verb-Object Phrase: The preceding verb dominates or concerns the following word. Complementary Phrase: The following word supplements and explains the preceding verb or adjective. Subject-Predicate Phrase: The following word states the preceding word, with a relationship of being stated and stating. Locative Phrase: A locative noun is attached after a noun or verb. Prepositional Phrase: It is formed by combining a preposition with a noun or pronoun, etc. Phrase with "de": It is formed by attaching the structural particle "de" after a noun, verb, adjective, or pronoun, etc. Appositive Phrase: Two or several words refer to the same object. Fixed Phrase: A proper name or idiom with a fixed structure.

[0043] Control Name: A control name refers to the name used to identify and reference a control in software development. A control is an object with which a user can interact to input or operate on data, and it usually appears in a dialog box or on a toolbar. The control name can help developers reference and operate these controls in the code. For example, in Visual Basic (VB), the name of the Frame control is Frame. You can reference and operate this control in the code through this name. For example, you can set the properties of the Frame control or call its methods.

[0044] Next, in combination with the accompanying drawings, the application operation method and device provided by the embodiments of the present application will be described in detail through specific embodiments and their application scenarios.

[0045] The application operation method and device provided by the embodiments of the present application can be applied in scenarios where a user performs an operation based on a session message and replies to the session message. Specifically, after receiving a session message, an electronic device can voice broadcast the message content of the session message; then, the user can trigger the electronic device to perform an operation according to their own needs based on the message content, and then reply to the session message through the operation result of this operation.

[0046] Exemplarily, taking the session message as "Fill in your personal information in the resume", the user fills in their personal information in the resume form according to this session message and replies to this session message, as specifically shown in Scenario 1.

[0047] In Scenario 1, suppose Xiao Wang is a user with limited vision. The electronic device displays a conversation interface between Xiao Wang and Xiao Hong. Xiao Wang's electronic device receives a conversation message (i.e., the first message) sent by Xiao Hong's electronic device and displays the conversation message, "Please fill in your personal information in your resume," in the conversation interface, announcing the conversation message by voice. Xiao Wang can then use voice to trigger the electronic device to open a recruitment application and display a recruitment interface with a resume form. At this point, Xiao Wang can fill in his personal information in the resume form and click the submit control, allowing the electronic device to submit his personal information. Finally, Xiao Wang can trigger the electronic device to return to the conversation interface and reply to Xiao Hong by voice, "Submitted," in the conversation interface. However, since Xiao Wang cannot see clearly, the submitted personal information may contain errors, and Xiao Wang may have to repeat the same operation several times, making the user's operation of the electronic device more cumbersome.

[0048] For example, the conversation message is "Send your employment experience", and the user copies the employment experience text from the resume document according to the conversation message, copies it in the conversation interface, and replies to the conversation message, as shown in Scenario 2.

[0049] In Scenario 2, the electronic device displays a conversation interface between Xiao Wang and Xiao Hong. Xiao Wang's electronic device receives a conversation message from Xiao Hong's electronic device and displays the message "Send me your employment history" on the conversation interface, announcing the message via voice. Xiao Wang can then use voice to trigger the electronic device to open a document application and display a document interface with a resume document. At this point, Xiao Wang can click and input the resume document, causing the electronic device to display the resume's contents. Xiao Wang can then search for the employment history text in the resume document and copy it. Xiao Wang then triggers the electronic device to return to the conversation interface, copy the employment history text there, and send it to Xiao Hong's electronic device via the electronic device. However, since Xiao Wang can't see clearly, searching for the employment history text in the resume document may take a long time and may be less accurate.

[0050] For example, with the conversation message being "Send me your employment experience", Xiao Wang orally narrates it to Xiao Hong via a voice call or video call, as shown in Scenario 3.

[0051] In Scenario 3, assume Xiao Wang is a completely blind user. His electronic device displays a conversation interface between Xiao Wang and Xiao Hong. Xiao Wang's electronic device receives a conversation message from Xiao Hong's electronic device and displays the message "Send me your employment history" on the conversation interface, announcing the message. Xiao Wang can then trigger his electronic device to initiate a voice call with Xiao Hong and verbally narrate his employment history to her. However, Xiao Hong may prefer a text version of her employment history, which Xiao Wang's verbal narration may not satisfy.

[0052] In the application operation method provided in the embodiment of the present application, since an operation link can be automatically generated through the message content of the first message, the visually impaired user only needs one input to trigger the electronic device to immediately execute the first operation corresponding to the first message, and automatically reply to the first message with the operation result of the first operation; this simplifies the process of visually impaired users performing complex operations in electronic devices and the process of replying to messages, thereby allowing visually impaired users to conveniently perform operations through electronic devices.

[0053] The execution subject of the application operation method provided in the embodiment of the present application may be an application operation device, which may be an electronic device or a functional module in an electronic device. The following uses an electronic device as an example to illustrate the technical solution provided in the embodiment of the present application.

[0054] The present application embodiment provides an application operation method, Figure 1 FIG. 1 shows a flow chart of an application operation method provided by an embodiment of the present application. Figure 1 As shown, the application operation method provided in the embodiment of the present application may include the following steps 201 to 204.

[0055] Step 201: The electronic device displays a first interface.

[0056] In an embodiment of the present application, the above-mentioned first interface includes an operation link corresponding to the first message, and the operation link is generated based on the message content of the first message, and the first message is a conversation message.

[0057] Optionally, in the embodiment of the present application, the first interface may be any one of the following: a conversation interface of an instant messaging application or a message interface of a text messaging application, etc. The specific interface may be determined according to actual use requirements and is not limited in the embodiment of the present application.

[0058] In the embodiment of the present application, the above-mentioned operation link is used to perform the first operation.

[0059] Optionally, in an embodiment of the present application, the electronic device may display an operation link in the message display area of ​​the first interface.

[0060] For example, in combination with the above scenario 1, Figure 2 As shown, taking a mobile phone as an example, the mobile phone can display a conversation interface 10 with Xiaohong. When the mobile phone receives a first message "Fill in your personal information in your resume", the mobile phone can automatically generate an operation link 11 corresponding to the first message based on the message content of the first message. Figure 2 The operation link 11 is represented as “DeepLink:com.1.2.3.4.5” and is displayed in the message display area 12 in the conversation interface 10 .

[0061] As another example, in combination with the above scenario 2, the mobile phone can display a conversation interface with Xiaohong. When the mobile phone receives the first message "Send your employment experience", the mobile phone can automatically generate an operation link "DeepLink:com.6.7.8" corresponding to the first message based on the message content of the first message, and display the operation link in the message display area in the conversation interface.

[0062] Optionally, in an embodiment of the present application, the electronic device may display an operation control corresponding to the operation link in the message display area of ​​the first interface, and the operation control is used to perform the first operation.

[0063] Optionally, in an embodiment of the present application, the electronic device can detect whether the accessibility function is enabled. If the accessibility function is enabled, the electronic device can generate an operation link based on the message content of the first message. If the accessibility function is not enabled, the electronic device can directly display the first message on the first interface without any processing.

[0064] For example, the electronic device can determine whether the accessibility function is enabled based on the flag corresponding to the accessibility function. For example, if the flag corresponding to the accessibility function is 1, it is determined that the electronic device has enabled the accessibility function; if the flag corresponding to the accessibility function is 0, it is determined that the electronic device has not enabled the accessibility function.

[0065] Optionally, in the embodiment of the present application, Figure 1 ,like Figure 3 As shown, the application operation method provided in the embodiment of the present application also includes the following step 301.

[0066] Step 301: Upon receiving a first message, the electronic device broadcasts the content of the first message.

[0067] Optionally, in an embodiment of the present application, when the first message is a text message, the electronic device can convert the message content of the first message into a first audio through a speech conversion model or a neural network model, and broadcast the first audio through a microphone in the electronic device.

[0068] For example, the mobile phone can input the content of the first message into a speech conversion model, such as a text-to-speech (TTS) model. The TTS model can remove non-text information such as special symbols and spaces from the message content. The mobile phone then detects the language of the message content, such as Chinese or English, and splits the message content into multiple words. The mobile phone then searches a speech library for multiple audio files that match the multiple words. Finally, the multiple audio files are concatenated in the order of the multiple words to obtain the first audio file.

[0069] Optionally, in the embodiment of the present application, the microphone may be any one of the following: a dynamic microphone, a condenser microphone, a piezoelectric microphone, or an electromagnetic microphone, etc. The specific type may be determined according to actual use requirements and is not limited in the embodiment of the present application.

[0070] Optionally, in an embodiment of the present application, the electronic device may determine whether to automatically broadcast the message content of the first message based on the user's input to the automatic voice playback control or the manual voice playback control in the accessibility setting interface.

[0071] For example, in combination with the above scenario 1, Figure 4A As shown, the mobile phone can display an accessibility setting interface 13, which includes an automatic voice broadcast control 14 and a manual voice broadcast control 15; if the user selects the automatic voice broadcast control 14, the mobile phone can automatically broadcast the first message "Fill in your personal information in your resume" through the microphone of the mobile phone after receiving the first message "Fill in your personal information in your resume". Figure 4B As shown, if the user selects the manual voice broadcast control 15, after the mobile phone receives the first message "Fill in your personal information in the resume", the user needs to click and input the first message to trigger the mobile phone to broadcast the first message "Fill in your personal information in the resume" through the microphone of the mobile phone.

[0072] As another example, in combination with the above scenarios 2 and 3, the mobile phone can display an accessibility settings interface, which includes an automatic voice broadcast control and a manual voice broadcast control. If the user selects the automatic voice broadcast control, after receiving the first message "Send your employment experience", the mobile phone can automatically broadcast the first message "Send your employment experience" through the microphone of the mobile phone. If the user selects the manual voice broadcast control, after receiving the first message "Send your employment experience", the mobile phone needs to click and input the first message to trigger the mobile phone to broadcast the first message "Send your employment experience" through the microphone of the mobile phone.

[0073] It should be noted that the execution time of the above step 301 can be after step 201 or before step 201, which can be determined according to actual use requirements and is not limited in the embodiment of the present application. Figure 3 As shown, the execution timing of the above step 301 can be after step 201.

[0074] In the embodiment of the present application, the electronic device can enable the user to know the message content of the first message through voice by broadcasting the first message, thereby avoiding the phenomenon that the user cannot view the message content due to unclear or invisible vision.

[0075] Optionally, in the embodiment of the present application, the first interface includes a first message, combined with Figure 1 ,like Figure 5 As shown, after the above step 201, the application operation method provided in the embodiment of the present application also includes the following step 401.

[0076] Step 401: The electronic device replaces the first message in the first interface with an operation link.

[0077] In an embodiment of the present application, after receiving the first message, the electronic device may display the first message in the message display area of ​​the first interface, and after a preset time, replace the first message in the first interface with an operation link.

[0078] Optionally, in the embodiment of the present application, the preset duration may be preset by the electronic device; or, the preset duration may be user-defined. Specific settings may be determined based on actual usage requirements and are not limited in the embodiment of the present application.

[0079] For example, the preset duration may be 1 minute, 2 minutes, or 5 minutes, etc. The specific duration may be determined according to actual use requirements and is not limited in the present embodiment.

[0080] Optionally, in an embodiment of the present application, the electronic device may directly overlay the operation link in the display area corresponding to the first message to replace the first message with the operation link; or, the electronic device may remove the first message from the display area corresponding to the first message, and add the operation link to the display area corresponding to the first message to replace the first message with the operation link.

[0081] For example, in combination with the above scenario 1, Figure 6 As shown, after receiving the first message, the mobile phone can display the first message "Fill in your personal information in the resume" in the message display area 12 of the first interface 10 and automatically broadcast the message content of the first message, such as Figure 2As shown, the electronic device can automatically cover the operation link 11 in the message display area 12 after 2 minutes, thereby replacing the first message displayed in the message display area 12 with the operation link "DeepLink:com.1.2.3.4.5".

[0082] As another example, in combination with the above-mentioned scenarios 2 and 3, after receiving the first message, the mobile phone can display the first message "Send your employment experience" in the message display area of ​​the first interface, and automatically broadcast the message content of the first message, and after 2 minutes, automatically cover the operation link in the message display area, thereby replacing the first message displayed in the message display area with the operation link "DeepLink:com.6.7.8".

[0083] Optionally, in an embodiment of the present application, the electronic device may display both the first message and the operation link in the first interface.

[0084] In one example, the electronic device may display both the first message and the operation link in a message display area in the first interface.

[0085] In another example, the electronic device may display the first message in a message display area in the first interface, and display the operation link in the first interface through a pop-up window.

[0086] In an embodiment of the present application, the electronic device can reduce the display content of the message display area in the first interface by replacing the first message in the first interface with an operation link, and avoids the problem of accidental touch caused by the user due to unclear vision when both the first message and the operation link are displayed in the first interface.

[0087] Step 202: The electronic device receives a first input.

[0088] In the embodiment of the present application, the first input is used to select an operation link from the first interface.

[0089] Optionally, in an embodiment of the present application, the first input may be a click input of the user on the operation link; or the first input may be a voice input of the user.

[0090] Optionally, in the embodiment of the present application, the click input may be a click input of the user on the operation link through a touch device such as a finger or a stylus.

[0091] Exemplarily, the click input may be a single click input, a double click input, or any number of click inputs, and may also be a long press input or a short press input.

[0092] For example, the first input may be a single click input of the user on the operation link.

[0093] Optionally, in an embodiment of the present application, after the electronic device displays the operation link in the first interface, the voice recognition function can be automatically turned on, so that the electronic device can directly receive the user's voice input.

[0094] For example, in combination with the above scenario 3, for a blind user, the electronic device can automatically turn on the voice recognition function without the user having to click on the voice recognition control, thereby improving the convenience of the electronic device in recognizing the user's voice.

[0095] Optionally, in an embodiment of the present application, after receiving the user's voice input, the electronic device may perform semantic recognition on the voice to determine the user's intention.

[0096] Exemplarily, the electronic device may input the above-mentioned speech into the first model to perform semantic recognition on the speech output by the user through the first model, thereby determining the user's user intention.

[0097] Exemplarily, the first model may be an artificial intelligence (AI) model. For example, the AI ​​model may include any one of the following: a neural network model, a large language model, etc. The specific model may be determined based on actual usage requirements and is not limited in the present embodiment.

[0098] It should be noted that since the user may be a visually impaired user who cannot see clearly or cannot see, the electronic device can enable the user to choose the input that is convenient for him or her by providing the user with click input or voice input, so that the user can operate the electronic device conveniently.

[0099] Optionally, in an embodiment of the present application, after the electronic device displays the operation link in the first interface, the electronic device may broadcast a third prompt message and display the third prompt message in the first interface to prompt the user to input the operation link.

[0100] Optionally, in the embodiment of the present application, Figure 1 ,like Figure 7 As shown, after the above step 201, the application operation method provided in the embodiment of the present application also includes the following step 501.

[0101] Step 501: The electronic device broadcasts a second prompt message.

[0102] In the embodiment of the present application, the second prompt information is used to prompt the user that an operation link has been generated.

[0103] Optionally, in the embodiment of the present application, the second prompt information may be preset by the electronic device; or the second prompt information may be user-defined. The specific information may be determined based on actual use requirements and is not limited in the embodiment of the present application.

[0104] For example, the second prompt message may be: "An operation link that can be automatically executed has been generated. You can click it or enter the "execute" voice message without worry."

[0105] Optionally, in an embodiment of the present application, the electronic device may display the second prompt information in the first interface through a pop-up window; or, the electronic device may display the second prompt information in the message display area in the first interface.

[0106] For example, combining the above scenario 1 and Figure 2 ,like Figure 8 As shown, after the mobile phone displays the operation link "DeepLink:com.1.2.3.4.5" in the message display area 12 of the first interface 10, a second prompt message "An operation link that can be automatically executed has been generated. You can click or enter the "execute" voice" with confidence" can be displayed on the first interface 10 through a pop-up window 16, and the second prompt message is automatically broadcast. The user can then click or enter the operation link "DeepLink:com.1.2.3.4.5" or enter the voice input.

[0107] As another example, in combination with the above scenarios 2 and 3, after the mobile phone displays the operation link "DeepLink:com.6.7.8" in the message display area of ​​the first interface, a second prompt message "An operation link that can be automatically executed has been generated. You can click or enter the "execute" voice" with confidence" can be displayed in the first interface through a pop-up window, and the second prompt message is automatically broadcast. The user can then click and enter the operation link "DeepLink:com.6.7.8" or make a voice input.

[0108] Optionally, in an embodiment of the present application, the second prompt information may be automatically generated according to the message content of the first message.

[0109] Exemplarily, after obtaining the message content of the first message, the electronic device can extract at least one of the subject, predicate and object of the message content, and then splice at least one of the subject, predicate and object with the first preset prompt information template to obtain the second prompt information.

[0110] For example, combined with the above scenario 1, the first message is "Fill in your personal information in the resume". The mobile phone can extract the predicate and object in the first message, that is, "Fill in the resume", and then splice the "Fill in the resume" with the first preset prompt information template "The operation link has been generated, you can click or enter the "execute" voice with confidence" to obtain the second prompt information "The fill in resume operation link has been generated, you can click or enter the "execute" voice with confidence".

[0111] Optionally, in an embodiment of the present application, after generating the second prompt information, the electronic device may perform voice conversion processing on the second prompt information to obtain a second audio, and broadcast the second audio.

[0112] Exemplarily, the electronic device may broadcast the second audio through a microphone.

[0113] It should be noted that the process of the electronic device performing voice conversion processing on the second prompt information can be found in the above embodiment in detail, and will not be described again here to avoid repetition.

[0114] Step 203: The electronic device executes a first operation corresponding to the operation link in response to the first input.

[0115] Optionally, in an embodiment of the present application, the above-mentioned operation link may include at least one instruction identifier.

[0116] Optionally, in an embodiment of the present application, the above-mentioned first operation corresponds to at least one atomic instruction.

[0117] Exemplarily, each instruction identifier in the at least one instruction identifier may correspond one-to-one to each atomic instruction in the at least one atomic instruction.

[0118] In the embodiment of the present application, the at least one atomic instruction is an atomic instruction corresponding to the first operation.

[0119] It should be noted that atomic instructions refer to instructions that are inseparable during the execution of the operation and will not be interrupted by the operating system or other hardware.

[0120] Optionally, in the embodiment of the present application, the first operation may include but is not limited to any of the following: data submission operation, data acquisition operation or data editing operation, etc. The specific operation can be determined according to actual use requirements and is not limited in the embodiment of the present application.

[0121] In the embodiment of the present application, the electronic device can automatically perform the first operation according to the operation link without user intervention.

[0122] It should be noted that the first operation corresponding to the operation link executed by the electronic device can be found in detail in the following embodiments, and will not be described again here to avoid repetition.

[0123] Step 204: The electronic device replies to the first message based on the operation result corresponding to the first operation.

[0124] In an embodiment of the present application, the electronic device can directly send the operation result corresponding to the first operation to the electronic device of another user to reply to the first message.

[0125] For example, in combination with the above scenario 1, after the mobile phone successfully executes the data submission operation corresponding to the first message "Fill in your personal information in the resume", the mobile phone can directly send the operation result corresponding to the data submission operation "to complete the submission" to Xiaohong's corresponding mobile phone to reply to the first message.

[0126] For example, combining the above Figure 2 ,like Figure 9 As shown, after obtaining the operation result "Submission Completed" corresponding to the first operation, the mobile phone can call the input method function in the first interface 10 and automatically fill the operation result "Submission Completed" in the input box of the input method function. Then, the send control in the input method function is automatically called to send the operation result "Submission Completed" to Xiaohong's electronic device. Finally, the operation result "Submission Completed" is displayed in the message display area 12 of the first interface 10.

[0127] As another example, combining the above scenarios 2 and 3, after the mobile phone successfully executes the data acquisition operation corresponding to the first message "Send your employment experience", the mobile phone can directly send Xiao Wang's employment experience "Worked in Company A for 3 years, the main job is B" to Xiao Hong's corresponding mobile phone to reply to the first message.

[0128] For example, combining the above Figure 2 ,like Figure 10 As shown, after obtaining the operation result corresponding to the first operation, "Worked at Company A for 3 years, main job B," the mobile phone can call the input method function in the first interface 10 and automatically fill the operation result "Worked at Company A for 3 years, main job B" in the input box of the input method function. Then, the send control in the input method function is automatically called to send the operation result "Worked at Company A for 3 years, main job B" to Xiaohong's electronic device. Finally, the operation result "Worked at Company A for 3 years, main job B" is displayed in the message display area 12 of the first interface 10.

[0129] Optionally, in the embodiment of the present application, Figure 1 ,like Figure 11 As shown, the above step 204 can be specifically implemented through the following step 204a.

[0130] Step 204a: The electronic device adds the operation result to the preset reply statement to obtain a first reply message, and displays the first reply message in the first interface.

[0131] Optionally, in an embodiment of the present application, the above-mentioned preset reply statement may be preset by the electronic device; or, the above-mentioned preset reply statement may be customized by the user.

[0132] For example, the preset reply statement may be: "Already done"; or the preset reply statement may be: "Thank you for checking it out."

[0133] Optionally, in an embodiment of the present application, the electronic device may determine different preset reply statements based on the message content of the first message.

[0134] For example, in combination with the above scenario 1, the above first message is: "Fill in your personal information in the resume". The mobile phone determines through semantic analysis that it only needs to perform the first operation and does not need to obtain information based on the first operation. The mobile phone can determine the preset reply statement as: "Operation has been performed".

[0135] As another example, combining the above scenarios 2 and 3, the above first message is: "Send your employment experience", and the mobile phone determines through semantic analysis that the mobile phone not only needs to perform the first operation, but also needs to reply based on the information obtained by the first operation. The mobile phone can determine the preset reply sentence as: "Thank you for checking it out."

[0136] Optionally, in an embodiment of the present application, the electronic device may add the operation result to a preset reply statement according to the grammatical structure to obtain a first reply message.

[0137] Optionally, in an embodiment of the present application, the above grammatical structure may include: subject, predicate, object, attributive, adverbial and complement.

[0138] For example, in Scenario 1 above, if the first message is "Fill in your personal information in your resume," the phone determines through semantic analysis that only the first operation needs to be performed, and does not need to obtain information based on the first operation. Therefore, the phone can determine the preset reply statement as "operation." Then, after the phone successfully performs the first operation, it obtains the operation result of the first operation, namely, "Submission completed." Finally, according to the grammatical rules corresponding to the subject, predicate, and object of the grammatical structure, the phone can concatenate the operation result "Upload completed" with the preset reply statement "operation," resulting in the first reply message "Submission completed."

[0139] As another example, combining Scenarios 2 and 3 above, if the first message is: "Send me your employment experience," the phone not only needs to perform the first operation but also respond based on the information obtained from the first operation. The phone can determine the default response sentence as: "Thank you for your review." Then, after the phone successfully performs the first operation, it obtains the result of the first operation: "Worked at Company A for 3 years, primarily in B." Finally, the result of the operation ("Worked at Company A for 3 years, primarily in B") is concatenated with the default response sentence "Thank you for your review," resulting in the first response message: "Worked at Company A for 3 years, primarily in B. Thank you for your review."

[0140] In an embodiment of the present application, after receiving the above-mentioned first reply message, the electronic device can send the first reply message to the electronic devices of other users and display the first reply message in the message display area of ​​the first interface.

[0141] For example, in combination with the above scenario 1, Figure 2 ,like Figure 12 As shown, after receiving the first reply message "To complete the submission operation", the mobile phone can call the input method function in the first interface 10 and automatically fill the first reply message "To complete the submission operation" in the input box of the input method function. Then, the send control in the input method function is automatically called to send the first reply message "To complete the submission operation" to Xiaohong's electronic device. Finally, the first reply message "To complete the submission operation" is displayed in the message display area 12 of the first interface 10.

[0142] For example, in combination with the above scenario 2 and scenario 3, in combination with 2, as shown in FIG. Figure 13 As shown, after receiving the first reply message "Worked at Company A for 3 years, main job B, thank you for your hard work," the mobile phone can call the input method function in the first interface 10 and automatically fill the first reply message "Worked at Company A for 3 years, main job B, thank you for your hard work" in the input box of the input method function. Then, the send control in the input method function is automatically called to send the first reply message "Worked at Company A for 3 years, main job B, thank you for your hard work" to Xiaohong's electronic device. Finally, the first reply message "Worked at Company A for 3 years, main job B, thank you for your hard work" is displayed in the message display area 12 of the first interface 10.

[0143] In the application operation method provided in an embodiment of the present application, an electronic device may display a first interface, the first interface including an operation link corresponding to a first message, the operation link being generated based on the message content of the first message; then, receiving a first input; then, in response to the first input, performing a first operation corresponding to the operation link; and finally, replying to the first message based on the operation result corresponding to the first operation. In this solution, since the electronic device can automatically generate an operation link based on the message content of the first message, a visually impaired user only needs one input to trigger the electronic device to immediately perform the first operation corresponding to the first message and automatically reply to the first message based on the operation result of the first operation. This simplifies the process of visually impaired users performing complex operations in electronic devices and the process of replying to messages, thereby allowing visually impaired users to conveniently perform operations through electronic devices.

[0144] Optionally, in the embodiment of the present application, Figure 1 ,like Figure 14 As shown, the application operation method provided in the embodiment of the present application also includes the following steps 601 to 603.

[0145] Step 601: Upon receiving a first message, the electronic device performs semantic analysis on the message content of the first message to obtain operation description information and operation condition information.

[0146] In an embodiment of the present application, the above-mentioned operation description information includes: user intention information, application name of the target application, interface name of the target application, control name in the interface corresponding to the interface name, and information imported into the target application.

[0147] Optionally, in an embodiment of the present application, the application name of the above-mentioned target application can be: the application package name of the target application, such as "com.A"; or, the application name of the target application can be: the application name of the target application displayed on the desktop of the electronic device, such as "A application".

[0148] Exemplarily, the above interface name may be an interface identifier of an independent interface in the target application.

[0149] For example, the interface identifier may be any one of the following: a text identifier, a numeric identifier, or a special symbol identifier.

[0150] It is understood that the information imported into the target application may be determined based on the content of the first message. For example, if the first message is "Put your personal information in your resume," the information imported into the target application may include: name, address, ID number, personal photo, and phone number.

[0151] In an embodiment of the present application, the electronic device may perform semantic analysis on the first message through the first model, thereby obtaining operation description information and operation condition information.

[0152] Exemplarily, the above-mentioned operation condition information may be operation sequence information.

[0153] For example, after obtaining the content of the first message, the electronic device may input the content of the first message into the first model, and the first model may pre-process the content of the message to obtain processed content. Then, the processed content of the message may be convolved to obtain the operation description information and the operation condition information.

[0154] Optionally, in the embodiment of the present application, the above preprocessing may include at least one of the following: word segmentation, part-of-speech tagging, and entity recognition. The specific details can be determined according to actual use requirements and are not limited by the embodiment of the present application.

[0155] For example, combining scenario 1, the first message is "Fill in your personal information in your resume". The mobile phone inputs the first message "Fill in your personal information in your resume" into the AI ​​model. The AI ​​model can split the message content "Fill in your personal information in your resume" into 5 words "will", "your", "personal information", "fill in", and "resume". Then, convolution is performed on these 5 words separately to obtain the user intention of "fill in personal information in the resume", the operation description information: "Get the name and phone number from the settings application, run application A, open the resume interface in application A, fill in the name and phone number in the form in the resume interface, and click the submit control", and the operation sequence information: "First get the name and phone number from the settings application, then run application A, then open the resume interface in application A, then fill in the name and phone number in the form in the resume interface, and finally click the submit control."

[0156] For example, combining scenario 2 and scenario 3, with the first message being "Send your employment experience", the mobile phone inputs the first message "Send your employment experience" into the AI ​​model. The AI ​​model can split the message content "Send your employment experience" into four words: "will", "your", "employment experience", and "send". The four words are then convolved separately to obtain the user intention of "sending personal experience", the operation description information: "Search for employment experience in the document application, copy the employment experience, and send the copied employment experience to the application corresponding to the first interface", and the operation sequence information: "First search for the employment experience "Worked in Company A for 3 years, with the main job being B" in the document application", then copy the employment experience, and finally send the employment experience to the application corresponding to the first interface".

[0157] Step 602: The electronic device performs instruction generation processing on the operation description information to obtain at least one atomic instruction and at least one instruction identifier corresponding to the at least one atomic instruction.

[0158] In the embodiment of the present application, one instruction identifier among the above-mentioned at least one instruction identifier corresponds to an atomic instruction.

[0159] Optionally, in an embodiment of the present application, the above-mentioned instruction identifier can be any one of the following: a text identifier, a digital identifier, or a special symbol identifier, etc., which can be determined based on actual usage and is not limited by the embodiment of the present application.

[0160] Optionally, in an embodiment of the present application, the electronic device can identify the operation type of the operation description information to determine the operation type corresponding to the operation description information, and then input the operation description information into the instruction template corresponding to the operation type of the operation description information for instruction mapping to obtain at least one atomic instruction.

[0161] Exemplarily, the electronic device may perform feature extraction processing on the operation description information to obtain multiple feature information corresponding to the operation description information, and then input the multiple feature information into a classification model to obtain the operation type corresponding to the operation description information.

[0162] For example, the above classification model may include any of the following: a random forest model, a support vector machine (SVM) model, a text convolutional neural network (TextCNN) model, a long short-term memory network (LSTM) model, or a Transformer classifier, etc. The specific model can be determined according to actual use requirements and is not limited in the embodiments of this application.

[0163] Optionally, in an embodiment of the present application, the above-mentioned operation type may include at least one of the following: file operation type, system control type, data query type, automation process type, network operation type and smart device control, etc.

[0164] For example, taking the operation description information of "get name and phone number from the settings application, run application A, open the resume interface in application A, fill in name and phone number in the form in the resume interface, and click the submit control," the mobile phone performs feature extraction on the operation description information to obtain multiple feature information, and then inputs the multiple feature information into the Transformer classifier to predict that the operation types of the operation description information include: system control type and data query type. The mobile phone can then input "get name and phone number from the settings application" into the instruction template corresponding to the data query type for instruction mapping, obtaining the atomic instruction corresponding to "get name and phone number," and input "run application A, open the resume interface in application A, fill in name and phone number in the form in the resume interface, and click the submit control" into the instruction template corresponding to the system control type for instruction mapping, obtaining the atomic instruction corresponding to "run application A, open the resume interface in application A, fill in name and phone number in the form in the resume interface, and click the submit control."

[0165] For example, taking the operation description information of "search for employment experience from the document application, copy the employment experience, and send the copied employment experience to the application corresponding to the first interface" as an example, the mobile phone performs feature extraction on the operation description information to obtain multiple feature information, and then inputs the multiple feature information into the Transformer classifier to predict that the operation types of the operation description information include: system control type and data query type. Then, the mobile phone can input "search for employment experience from the document application" into the instruction template corresponding to the data query type for instruction mapping, and obtain the atomic instruction corresponding to "search for employment experience", and input "copy employment experience, send the copied employment experience to the application corresponding to the first interface" into the instruction template corresponding to the system control type for instruction mapping, and obtain the atomic instruction corresponding to "copy employment experience, send the copied employment experience to the application corresponding to the first interface".

[0166] Optionally, in an embodiment of the present application, after obtaining at least one atomic instruction, the electronic device can generate at least one instruction identifier corresponding to the at least one atomic instruction, and store each instruction identifier in the at least one instruction identifier in a one-to-one association with each atomic instruction in the at least one atomic instruction.

[0167] In one example, an electronic device can perform a hash operation on each atomic instruction in at least one atomic instruction to obtain a hash value corresponding to each atomic instruction in at least one atomic instruction, and then use the hash value corresponding to each atomic instruction in the at least one atomic instruction as the instruction identifier of each atomic instruction in at least one atomic instruction.

[0168] In another example, the electronic device can generate a non-repeating random number corresponding to each atomic instruction based on the instruction content of each atomic instruction in at least one atomic instruction, and then use the non-repeating random number corresponding to each atomic instruction as the instruction identifier of each atomic instruction.

[0169] For example, combined with the above scenario 1, taking the operation description information as "Get the name and phone number from the settings application, run application A, open the resume interface in application A, fill in the name and phone number in the form in the resume interface, and click the submit control", the operation description information corresponds to 5 atomic instructions, and the 5 atomic instructions are: "Get the name and phone number from the settings application", "Run application A", "Open the resume interface in application A", "Fill in the name and phone number in the form in the resume interface" and "Click the submit control". The 5 atomic instructions are processed with non-repetitive random number generation, and the instruction identifier corresponding to the atomic instruction of "Get the name and phone number from the settings application" is 1, the instruction identifier corresponding to the atomic instruction of "Run application A" is 2, the instruction identifier corresponding to the atomic instruction of "Open the resume interface in application A" is 3, the instruction identifier corresponding to the atomic instruction of "Fill in the name and phone number in the form in the resume interface" is 4, and the instruction identifier corresponding to the atomic instruction of "Click the submit control" is 5.

[0170] For example, combining the above scenarios 2 and 3, taking the operation description information of "search for employment experience from the document application, copy the employment experience, and send the copied employment experience to the application corresponding to the first interface" as an example, the operation description information corresponds to 3 atomic instructions, and the 3 atomic instructions are: "search for employment experience from the document application", "copy employment experience" and "send the copied employment experience to the application corresponding to the first interface". The 3 atomic instructions are processed by non-repetitive random number generation, and the instruction identifier corresponding to the atomic instruction of "search for employment experience from the document application" is 6, the instruction identifier corresponding to the atomic instruction of "copy employment experience" is 7, and the instruction identifier corresponding to the atomic instruction of "send the copied employment experience to the application corresponding to the first interface" is 8.

[0171] Step 603: The electronic device combines at least one instruction identifier according to the operation condition information and the link template to obtain an operation link.

[0172] Optionally, in the embodiment of the present application, the link template may be preset by the electronic device; or the link template may be user-defined. The specific configuration may be determined based on actual usage requirements and is not limited by the embodiment of the present application.

[0173] Exemplarily, the link template may be: “DeepLink:com.”.

[0174] In an embodiment of the present application, the electronic device may sort at least one instruction identifier according to the operation sequence description information, and then concatenate the sorted at least one instruction identifier with a link template to obtain an operation link.

[0175] For example, in conjunction with Scenario 1 above, taking the operation description information of "get name and phone number from the settings app, run App A, open the resume interface in App A, enter name and phone number in the form in the resume interface, and click Submit," the five instruction identifiers corresponding to the first message are: "1, 4, 2, 3, 5." The mobile phone can rearrange the five instruction identifiers according to the operation sequence description information of "first get name and phone number from the settings app, then run App A, then open the resume interface in App A, then enter name and phone number in the form in the resume interface, and finally click Submit," resulting in the rearranged five instruction identifiers of "1, 2, 3, 4, 5." Finally, the rearranged five instruction identifiers are spliced ​​with the link template to obtain "DeepLink:com.1.2.3.4.5."

[0176] For example, combining the above scenarios 2 and 3, taking the operation description information of "search for employment experience in the document application, copy the employment experience, and send the copied employment experience to the application corresponding to the first interface" as an example, the three instruction identifiers corresponding to the first message are: "8, 6, 7". The mobile phone can rearrange the three instruction identifiers according to the operation sequence description information "first search for employment experience "Worked at Company A for 3 years, main job B" in the document application, then copy the employment experience, and finally send the employment experience to the application corresponding to the first interface", and obtain the rearranged three instruction identifiers as "6, 7, 8". Finally, the rearranged three instruction identifiers are spliced ​​with the link template to obtain "DeepLink:com.6.7.8".

[0177] It should be noted that the execution timing of the above steps 601 to 603 can be before the above step 201, or after the above step 201, which can be determined according to actual use requirements and is not limited in the embodiment of the present application. Figure 14 As shown, the execution timing of the above steps 601 to 603 can be before step 201.

[0178] In an embodiment of the present application, the electronic device performs semantic parsing on the message content of the first message to obtain operation description information and operation sequence description information, then performs instruction generation processing on the operation description information to obtain at least one atomic instruction and at least one instruction identifier corresponding to the at least one atomic instruction, and finally, splices at least one instruction identifier according to the operation sequence description information and the link template to obtain an operation link. It can be understood that the electronic device's semantic parsing of the message content of the first message can enable the electronic device to accurately determine the user's intention, avoid the inability to complete the first operation by generating at least one atomic instruction, and reduce the storage size of the operation link by obtaining the operation link through at least one instruction identifier.

[0179] Optionally, in the embodiment of the present application, Figure 1 ,like Figure 15 As shown, the above step 203 can be specifically implemented through the following step 203a.

[0180] Step 203a: In response to the first input, the electronic device displays a second interface corresponding to the interface name of the target application, and sequentially executes at least one atomic instruction in the second interface.

[0181] In an embodiment of the present application, the electronic device can run the target application in the foreground and jump from the first interface to the second interface corresponding to the interface name of the target application, and finally execute at least one atomic instruction in the second interface in sequence according to the operation execution order.

[0182] For example, in combination with the above scenario 1, Figure 16 As shown, the mobile phone can display the resume interface 17 in application A, that is, the second interface mentioned above, which includes a name input box 18, a phone number input box 19 and a submit control 20. The mobile phone then automatically fills in the name input box 18 with the user's name: A, and automatically fills in the phone number: 12345612345 in the phone number input box 19, and automatically clicks the submit control 20 to submit the user's personal information.

[0183] As another example, in combination with the above-mentioned scenarios 2 and 3, the mobile phone can display the resume interface in the B application, that is, the above-mentioned second interface, which includes the user's personal resume, and the personal resume includes employment experience. The electronic device can traverse the resume content of the personal resume, and when the title of "Employment Experience" is identified, copy all the text corresponding to the title of "Employment Experience", that is, the above-mentioned "Worked in Company A for 3 years, the main job is B", and transfer all the text to the first interface.

[0184] In an embodiment of the present application, the electronic device automatically executes the first operation corresponding to the first message, which simplifies the process of the user performing complex operations in the electronic device, so that the user can conveniently perform the operations required by the user in the electronic device.

[0185] Optionally, in the embodiment of the present application, Figure 1 ,like Figure 17 As shown, after the above step 203, the operating method provided in the embodiment of the present application further includes the following steps 701 to 703.

[0186] Step 701: The electronic device broadcasts a first prompt message.

[0187] In the embodiment of the present application, the above-mentioned first prompt information is used to prompt the user to complete the first operation.

[0188] Optionally, in the embodiment of the present application, the first prompt information may be preset by the electronic device; or the first prompt information may be customized by the user. The specific information may be determined according to actual use requirements and is not limited by the embodiment of the present application.

[0189] Optionally, in an embodiment of the present application, the first prompt information may be generated by the electronic device according to the message content of the first message.

[0190] Exemplarily, the electronic device identifies the user intention from the message content of the first message, and combines the user intention with the second preset prompt information to obtain the above-mentioned first prompt information.

[0191] For example, combined with the above scenario 1, the first message is "Fill in your personal information in the resume", and the mobile phone recognizes the user's intention of "submitting personal information" from the message content of the first message, and then splices the user's intention of "submitting personal information" with the second preset prompt information "The system has completed the operation steps, whether to submit personal information", and obtains the first prompt information "The system has completed the operation steps, whether to submit personal information".

[0192] Optionally, in an embodiment of the present application, the electronic device may perform voice conversion processing on the first prompt information to obtain a third audio, and broadcast the third audio.

[0193] Exemplarily, the electronic device may broadcast the third audio through a microphone.

[0194] Optionally, in an embodiment of the present application, the electronic device may display the first prompt information in the second interface and broadcast the first prompt information; or, the electronic device may display the first prompt information in the second interface.

[0195] Exemplarily, the electronic device may display the first prompt information in the second interface through a pop-up window.

[0196] Optionally, in an embodiment of the present application, the first prompt information may include a confirmation control and a cancel control.

[0197] It can be understood that the above-mentioned confirmation control is used to trigger the electronic device to perform the first operation, and the above-mentioned cancellation control is used to trigger the electronic device not to perform the first operation.

[0198] Step 702: The electronic device receives a second input.

[0199] In the embodiment of the present application, the second input is used to trigger the electronic device to complete the first operation.

[0200] Optionally, in the embodiment of the present application, the second input may be a click input of the user on the confirmation control; or the second input may be a voice input of the user. The specific input may be determined according to actual use requirements and is not limited in the embodiment of the present application.

[0201] Exemplarily, the second input may be a single click input of the confirmation control by the user; or, the second input may be the user speaking an “OK” voice.

[0202] Step 703: The electronic device completes the first operation in response to the second input.

[0203] It should be noted that the specific process of the electronic device sequentially executing at least one atomic instruction in the second interface can be found in the above embodiment, and will not be described again here to avoid repetition.

[0204] In one example, combined with scenario 1, such as Figure 16 As shown, Figure 18 As shown, the mobile phone can display a first prompt message "The system has completed the operation steps, are you sure to submit" through a pop-up window 21 in the resume interface 17, and display a confirmation control 22 and a cancel control 23 in the pop-up window 21. The user can click the confirmation control 22 to input, so that the mobile phone can perform the submission operation, thereby completing the first operation.

[0205] In another example, combined with scenario 1, such as Figure 16 As shown, Figure 19 As shown, the mobile phone can prompt the first prompt information "The system has completed the operation steps, are you sure to submit" through voice in the resume interface 17. The user can say "OK" voice to enable the mobile phone to perform the submission operation, so that the mobile phone can complete the first operation.

[0206] In an embodiment of the present application, the electronic device can provide a prompt message to the user before the operation is completed, so that the user can determine whether the first operation is completed, thereby improving the user experience.

[0207] For example, the following takes a visually impaired user as an example to explain the operation method provided in the embodiment of the present application in detail, which may include the following steps 30 to 36.

[0208] Step 30: The electronic device displays a conversation interface.

[0209] Step 31: When the electronic device receives the first message, the first message is displayed in the conversation interface, and the message content of the first message is voice broadcasted.

[0210] Step 32: The electronic device generates an operation link based on the message content of the first message.

[0211] Step 33: The electronic device replaces the first message displayed in the reply interface with an operation link and voice broadcasts the second prompt information.

[0212] Step 34: The electronic device receives the user's voice input.

[0213] Step 35: The electronic device automatically executes the first operation corresponding to the operation link in response to the voice input.

[0214] Step 36: The electronic device replies to the first message according to the operation result of the first operation.

[0215] In the embodiment of the present application, since an operation link can be automatically generated through the message content of the first message, the user only needs one input to trigger the electronic device to automatically execute the first operation corresponding to the first message, and automatically reply to the first message through the operation result of the first operation; this simplifies the process of visually impaired users performing complex operations in electronic devices and the process of replying to messages, so that visually impaired users can conveniently perform the operations they need through electronic devices.

[0216] The above-mentioned method embodiments, or various possible implementation methods in each method embodiment, can be executed separately, or, under the premise that there is no contradiction, can also be executed in combination with each other. The specific implementation can be determined according to actual usage requirements, and the embodiments of the present application do not limit this.

[0217] It should be noted that the application operation method provided in the embodiment of the present application can be executed by an operating device. In the embodiment of the present application, the application operation device provided in the embodiment of the present application is described by taking the application operation device executing the operation method as an example.

[0218] Figure 20 A possible structural diagram of the application operation device involved in the embodiment of the present application is shown. Figure 20 As shown, the application operating device 70 may include: a display module 71, a receiving module 72 and a processing module.

[0219] The display module is configured to display a first interface, which includes an operation link corresponding to a first message, the operation link being generated based on the content of the first message, wherein the first message is a conversation message. The receiving module 72 is configured to receive a first input. The processing module 73 is configured to, in response to the first input received by the receiving module 72, execute a first operation corresponding to the operation link, and reply to the first message based on the result of the first operation.

[0220] In one possible implementation, the above-mentioned processing module 73 is also used to, when receiving the first message, perform semantic analysis on the message content of the first message to obtain operation description information and operation condition information; and perform instruction generation processing on the operation description information to obtain at least one atomic instruction and at least one instruction identifier corresponding to at least one atomic instruction, one instruction identifier corresponding to one atomic instruction; and splice at least one instruction identifier according to the operation condition information and the link template to obtain an operation link; wherein the operation description information includes: user intention information, the application name of the target application, the interface name of the target application, the control name in the interface corresponding to the interface name, and information imported into the target application.

[0221] In a possible implementation, the display module 71 is specifically configured to display the second interface corresponding to the interface name of the target application. The processing module 73 is specifically configured to sequentially execute at least one atomic instruction in the second interface.

[0222] In one possible implementation, the processing module 73 is further configured to, after the display module 71 displays the second interface corresponding to the interface name of the target application, broadcast a first prompt message, where the first prompt message prompts the user to complete the first operation. The receiving module 72 is further configured to receive a second input. The processing module 73 is specifically configured to complete the first operation in response to the second input received by the receiving module 72.

[0223] In a possible implementation, the first interface includes a first message, and the processing module 73 is further configured to replace the first message in the first interface with an operation link after the first interface is displayed.

[0224] In a possible implementation, the processing module 73 is further configured to broadcast a second prompt message after the display module displays the first interface, where the second prompt message is used to prompt the user that the operation link has been generated.

[0225] An embodiment of the present application provides an application operating device. Since the operating device can automatically generate an operation link based on the message content of a first message, a visually impaired user only needs one input to trigger the application operating device to immediately execute a first operation corresponding to the first message, and automatically reply to the first message with the operation result of the first operation; this simplifies the process of visually impaired users performing complex operations in the operating device and the process of replying to messages, thereby enabling the visually impaired to conveniently perform operations through the application operating device.

[0226] The application operating device in the embodiment of the present application can be an electronic device or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or a device other than a terminal. For example, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc. It can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., and the embodiment of the present application does not specifically limit it.

[0227] The application operating device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0228] The application operation device provided in the embodiment of the present application can implement each process implemented in the above embodiment. To avoid repetition, it will not be described here.

[0229] Alternatively, as Figure 21 As shown, an embodiment of the present application also provides an electronic device 90, including a processor 91 and a memory 92, wherein the memory 92 stores a program or instruction that can be run on the processor 91, and when the program or instruction is executed by the processor 91, the various steps of the above-mentioned operation method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0230] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0231] Figure 22 A schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application.

[0232] The electronic device 100 includes but is not limited to components such as a radio frequency unit 101 , a network module 102 , an audio output unit 103 , an input unit 104 , a sensor 105 , a display unit 106 , a user input unit 107 , an interface unit 108 , a memory 109 , and a processor 110 .

[0233] Those skilled in the art will understand that the electronic device 100 may also include a power source (such as a battery) to power each component, and the power source may be logically connected to the processor 110 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system. Figure 22 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be repeated here.

[0234] The display unit 106 is configured to display a first interface including an operation link corresponding to a first message, the operation link being generated based on the content of the first message, wherein the first message is a conversation message. The user input unit 107 is configured to receive a first input. The processor 110 is configured to, in response to the first input, execute a first operation corresponding to the operation link; and reply to the first message based on the result of the first operation.

[0235] Optionally, in an embodiment of the present application, the above-mentioned processor 110 is further used to, upon receiving the first message, perform semantic analysis on the message content of the first message to obtain operation description information and operation condition information; and perform instruction generation processing on the operation description information to obtain at least one atomic instruction and at least one instruction identifier corresponding to at least one atomic instruction, one instruction identifier corresponding to one atomic instruction; and splice at least one instruction identifier according to the operation condition information and the link template to obtain an operation link; wherein the operation description information includes: user intention information, the application name of the target application, the interface name of the target application, the control name in the interface corresponding to the interface name, and information imported into the target application.

[0236] Optionally, in an embodiment of the present application, the processor 110 is specifically configured to display a second interface corresponding to the interface name of the target application, and to sequentially execute at least one atomic instruction in the second interface.

[0237] Optionally, in an embodiment of the present application, the processor 110 is further configured to, after displaying the second interface corresponding to the interface name of the target application, broadcast a first prompt message, where the first prompt message is used to prompt the user to complete the first operation. The user input unit 107 is further configured to receive a second input. The processor 110 is specifically configured to complete the first operation in response to the second input.

[0238] Optionally, in an embodiment of the present application, the first interface includes a first message, and the processor 110 is further configured to replace the first message in the first interface with an operation link after displaying the first interface.

[0239] Optionally, in an embodiment of the present application, the processor 110 is further configured to broadcast a second prompt message after displaying the first interface, where the second prompt message is configured to prompt the user that an operation link has been generated.

[0240] An embodiment of the present application provides an electronic device. Since an operation link can be automatically generated according to the message content of a first message, a visually impaired user only needs one input to trigger the electronic device to immediately execute a first operation corresponding to the first message, and automatically reply to the first message according to the operation result of the first operation; this simplifies the process of visually impaired users performing complex operations in the electronic device and the process of replying to messages, thereby allowing visually impaired users to conveniently perform operations through the electronic device.

[0241] The electronic device provided in the embodiment of the present application can implement each process implemented in the above method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0242] The beneficial effects of various implementations in this embodiment can be specifically referred to the beneficial effects of the corresponding implementations in the above method embodiment. To avoid repetition, they will not be described here.

[0243] It should be understood that in an embodiment of the present application, the input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes a touch panel 1071 and at least one of other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and an operating stick, which will not be repeated here.

[0244] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM), and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0245] Processor 110 may include one or more processing units. Optionally, processor 110 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 110.

[0246] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0247] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0248] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment and achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0249] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0250] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned operation method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0251] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0252] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0253] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. An application operation method, characterized in that: The method comprises: Displaying a first interface, the first interface including an operation link corresponding to the first message, the operation link being generated based on the message content of the first message, wherein the first message is a conversation message; receiving a first input; In response to the first input, executing a first operation corresponding to the operation link; Reply to the first message based on the operation result corresponding to the first operation.

2. The method according to claim 1, characterized in that The method further comprises: When the first message is received, semantically parsing the message content of the first message to obtain operation description information and operation condition information; Performing instruction generation processing on the operation description information to obtain at least one atomic instruction and at least one instruction identifier corresponding to the at least one atomic instruction, where one instruction identifier corresponds to one atomic instruction; splicing the at least one instruction identifier according to the operation condition information and the link template to obtain the operation link; The operation description information includes: user intention information, the application name of the target application, the interface name of the target application, the control name in the interface corresponding to the interface name, and information for importing the target application.

3. The method according to claim 1 or 2, characterized in that The executing the first operation corresponding to the operation link includes: A second interface corresponding to the interface name of the target application is displayed, and the at least one atomic instruction is executed sequentially in the second interface.

4. The method according to claim 1, wherein After executing the first operation corresponding to the operation link, the method further includes: broadcasting a first prompt message, where the first prompt message is used to prompt the user to complete the first operation; receiving a second input; In response to the second input, the first operation is completed.

5. The method according to claim 1, wherein The first interface includes the first message. After displaying the first interface, the method further includes: The first message in the first interface is replaced with the operation link.

6. The method according to claim 1, characterized in that After displaying the first interface, the method further includes: A second prompt message is broadcasted, where the second prompt message is used to prompt the user that the operation link has been generated.

7. An application operating device, characterized in that: The operating device includes: a display module, a receiving module and a processing module; The display module is configured to display a first interface, the first interface including an operation link corresponding to a first message, the operation link being generated based on the message content of the first message, wherein the first message is a conversation message; The receiving module is configured to receive a first input; The processing module is configured to execute a first operation corresponding to the operation link in response to the first input received by the receiving module; and reply to the first message based on an operation result corresponding to the first operation.

8. The device according to claim 7, characterized in that The processing module is further configured to, upon receiving the first message, perform semantic parsing on the message content of the first message to obtain operation description information and operation condition information; and perform instruction generation processing on the operation description information to obtain at least one atomic instruction and at least one instruction identifier corresponding to the at least one atomic instruction, where one instruction identifier corresponds to one atomic instruction; and combining the at least one instruction identifier according to the operation condition information and the link template to obtain the operation link; The operation description information includes: user intention information, the application name of the target application, the interface name of the target application, the control name in the interface corresponding to the interface name, and information for importing the target application.

9. The device according to claim 7 or 8, characterized in that The display module is specifically configured to display a second interface corresponding to the interface name of the target application; The processing module is specifically configured to execute the at least one atomic instruction in sequence in the second interface.

10. The device according to claim 9, characterized in that The processing module is further configured to broadcast a first prompt message after the display module displays the second interface corresponding to the interface name of the target application, where the first prompt message is used to prompt the user to complete the first operation; The receiving module is further configured to receive a second input; The processing module is specifically configured to complete the first operation in response to the second input received by the receiving module.

11. The device according to claim 7, characterized in that The first interface includes the first message, and the processing module is further configured to replace the first message in the first interface with the operation link after the first interface is displayed.

12. The device according to claim 7, characterized in that The processing module is further configured to broadcast a second prompt message after the display module displays the first interface, where the second prompt message is configured to prompt the user that the operation link has been generated.

Images