Task processing method and electronic device

Abstract

The application provides a task processing method and an electronic device, and relates to the technical field of terminals. Through the method, one or more tasks can be automatically executed in response to a user instruction, and a global interaction window and a document interaction window are provided as an entrance of different task scenarios for the user to input the instruction. After the electronic device receives an instruction input by the user in the document interaction window, the electronic device can identify the document instruction and respond, and can also identify a complex instruction irrelevant to the document and respond. The document task is automatically executed in the document interaction window, and for a non-document task requested to be processed in the document interaction window, the task closed loop can be completed in a specific manner, the compatibility of task processing is improved, the success rate of automatic execution of the task is improved, and then the user experience and operation efficiency are improved.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a task processing method and an electronic device. Background Technology

[0002] In daily smartphone use, users often need to manually perform a series of user interface (UI) operations to complete specific tasks, such as processing documents, ordering takeout, turning off auto-renewal services, sending files, or other possible tasks. However, manually performing each step of these tasks is not only inconvenient but also prone to errors. Some tasks even have overly complex operation paths, making the user experience cumbersome and time-consuming, thus negatively impacting the user experience. Summary of the Invention

[0003] This application provides a task processing method and an electronic device that can automatically execute document tasks in a document interaction window, and can complete the task loop in a specific way for non-document tasks requested for processing in the document interaction window, thereby improving the compatibility of task processing and the success rate of automatic task execution, and thus improving user experience and operational efficiency.

[0004] In a first aspect, embodiments of this application provide a task processing method, the method comprising: when a user inputs a first instruction in a global interactive window, and the first instruction indicates the execution of a first task, the electronic device automatically executes the first task and displays the execution result of the first task in the global interactive window; the first task is a non-document task;

[0005] When a user enters a second command in the document interaction window, and the second command instructs the execution of a document task, the electronic device automatically executes the document task and displays the execution result of the document task in the document interaction window;

[0006] When a user enters a third instruction in the document interaction window, and the third instruction instructs the execution of the document task and the first task, the electronic device automatically executes the document task and displays the execution result of the document task in the document interaction window. The electronic device also displays a first jump prompt message, which indicates that the first task will be automatically executed after jumping to the global interaction window.

[0007] The task processing method provided in this application can automatically execute one or more tasks in response to user instructions. It provides a global interaction window and a document interaction window as entry points for different task scenarios, allowing users to input instructions. After receiving instructions from the user in the document interaction window, the electronic device can recognize and respond to both document instructions and complex instructions unrelated to the document. Document tasks are automatically executed in the document interaction window, and for non-document tasks requested in the window, a task loop can be completed in a specific way, improving the compatibility of task processing and the success rate of automatic task execution, thereby enhancing user experience and operational efficiency.

[0008] The global interactive window is an interactive window or interface that enables the automatic execution of various tasks, such as through system assistant applications. Users can input commands such as ordering takeout, checking bill renewals, and sending files through the global interactive window to trigger electronic devices to automatically perform tasks such as ordering takeout, checking bill renewals, and sending files.

[0009] The document interaction window is a dedicated interactive window or interface for handling document tasks, and can be provided through document intelligence applications, for example. Users can input document-related commands through the document interaction window to trigger electronic devices to automatically execute document tasks.

[0010] The solution proposed in this application enables an electronic device to recognize and respond to document-related instructions, as well as non-document-related instructions, and complex instructions that include both document-related and document-unrelated information. It can identify document and non-document tasks based on user instructions and supports task decomposition, employing different processing flows to complete document and non-document tasks separately, thus improving the compatibility of task processing.

[0011] In some possible implementations, the method further includes: when a user enters the first instruction in the document interaction window, the electronic device displays the first jump prompt information in the document interaction window.

[0012] With the above solution, after receiving instructions from the user in the document interaction window, the electronic device can recognize and respond to document instructions, non-document instructions, and complex instructions that include both document-related and document-unrelated information. This improves task processing compatibility, user experience, and operational efficiency.

[0013] In some possible implementations, the method further includes: when a user inputs the third instruction in the document interaction window, the electronic device invokes a document proxy model to perform inference based on the third instruction to obtain a multi-task operation sequence in DSL format; wherein the multi-task operation sequence includes a document task operation sequence and a non-document task operation sequence.

[0014] The document proxy model performs inference based on the third instruction, including vertical domain detection.

[0015] Among them, document tasks can be identified through in-vertical detection, while non-document tasks can be identified through out-of-vertical detection.

[0016] Document tasks are also known as executable tasks, while non-document tasks are also known as out-of-domain tasks.

[0017] The document proxy model proposed in this application supports both in-domain and out-domain document detection, thereby enabling the identification of executable document tasks and non-document tasks that are not supported for execution based on user instructions, thus improving the compatibility of task processing.

[0018] In some possible implementations, after obtaining the DSL-formatted multitasking operation sequence, the method further includes: splitting the DSL-formatted multitasking operation sequence into tasks to obtain the document task operation sequence and the non-document task operation sequence; wherein the non-document task operation sequence includes a first parameter description, the first parameter description indicating that the first task does not support execution.

[0019] For example, the first task is to buy a plane ticket, and the first parameter is described as not_supported(query = buy a plane ticket).

[0020] The document proxy model proposed in this application supports both in-domain and out-domain document detection. This allows it to identify executable document tasks and non-document tasks that are not supported for execution based on user instructions. Furthermore, it supports task decomposition, employing different processing flows to complete document tasks and non-document tasks separately. This improves the compatibility of task processing and increases the success rate of automated task execution.

[0021] The automatic execution of document tasks includes: automatically executing the document tasks based on a document task operation sequence.

[0022] The display of the first jump prompt information includes: displaying the first jump prompt information based on the non-document task operation sequence.

[0023] The above solution can identify document tasks and non-document tasks based on user instructions, and supports task decomposition, using different processing flows to complete document tasks and non-document tasks respectively, thereby improving the compatibility of task processing and thus improving user experience and operational efficiency.

[0024] In some possible implementations, the document task operation sequence includes a first task identifier, a first tool name, first tool parameters, and first configuration information. The first configuration information is used to configure whether the execution result of the document task is displayed to the user; each task parameter corresponds to a slot, and the slot is used to fill in a slot value.

[0025] The DSL data with a specific format provided in this application embodiment includes the following four elements: 1) taskID: representing the task ID, one task corresponds to one ID; for example, 1, 2, 3, etc. can be used to represent the execution order of multiple tasks; 2) tool: representing the tool name; 3) params: representing the tool parameters; 4) show: indicating whether to show to the user, the optional value is yes or no, or the optional value is True or False, True means to show, False means not to show).

[0026] For example, if the user instruction is "Please help me translate this document", the document agent model will perform inference and obtain the document task arrangement result in DSL format as: ['1', 'doc_translate(part=None, target_language=None)', Yes]. Here, the task ID is 1, the tool is the document translation tool doc_translate, the parameter params includes the part and the target language target_language, and show is "Yes".

[0027] For example, the task orchestration result in DSL format is [taskID,tool(params1=XXX,params2=XXX),show].

[0028] In some embodiments, the four elements described above can be used to arrange tasks in a list format.

[0029] For example, the document task is: Help me translate the main idea of ​​the document. The document task operation sequence can be represented as:

[0030] [['1','doc_summary(part=None,limit=None)',False],

[0031] ['2','doc_translate(part=result(ID1).summary_result,target_language=None)',True]].

[0032] In some possible implementations, the document task operation sequence may further include third configuration information for multi-document task scenarios, wherein the third configuration information is used to indicate that the slot value of one task depends on the result of another task.

[0033] For example, multi-document task scenarios include translation tasks and document summary tasks.

[0034] In this embodiment of the application, for complex tasks, DSL data with a specific form also includes slot dependency elements, represented as result(ID=XXX), which represents the task result with the parameter dependency identifier XXX of the slot.

[0035] For example, if the user instruction is "Please help me translate the main idea of ​​this document", the document agent model will perform inference, and the resulting DSL-formatted "document" task orchestration (i.e., the document task operation sequence) can be represented as follows:

[0036] [['1','doc_summary(part=None,limit=None)'],

[0037] ['2','doc_translate(part=result(ID1),summary_result,target_language=None)']].

[0038] In this case, the content to be translated in the translation task in Identifier 2 depends on the results of the document summary task in Identifier 1.

[0039] The above solutions can handle complex tasks and improve the user experience.

[0040] In some possible implementations, the method further includes: in response to a user's first operation, the electronic device launches a document agent application and displays the document interaction window; and detects whether the user has entered a command in the document interaction window through the document agent application; wherein the document agent application supports invoking a document proxy model.

[0041] In some possible implementations, the non-document task operation sequence includes a second task identifier, a second tool name, a second tool parameter, and second configuration information; wherein, the slot of the second tool parameter is set to the first task, the second tool name is used to indicate that the first task does not support execution, and the second configuration information is used to configure that the execution result of the first task is not displayed to the user.

[0042] For example, a sequence of non-document task operations can be represented as ['1', 'not_supported(query=buy a plane ticket)', False].

[0043] In some embodiments, for document tasks that are supported for execution, show=True can be set. For non-document tasks that are not supported for execution, show=False can be set to hide them from the user.

[0044] The following example illustrates the multi-task operation sequence corresponding to a complex task. For instance, if the user instruction is "Please help me translate the main idea of ​​this document and buy a plane ticket," the multi-task operation sequence can be obtained through reasoning using a document proxy model:

[0045] [['1','doc_summary(part=None,limit=None)'],

[0046] ['2','doc_translate(part=result(ID1),summary_result,target_language=None)'],

[0047] ['3', 'not_supported(query=buy a plane ticket)']].

[0048] By splitting tasks, we can obtain document task operation sequences and non-document task operation sequences.

[0049] The document task operation sequence includes: ['1', 'doc_summary(part=None,limit=None)'], ['2', 'doc_translate(part=result(ID1),summary_result,target_language=None)'].

[0050] The non-document task operation sequence includes: ['3', 'not_supported(query=buy a plane ticket)'].

[0051] In some possible implementations, after the electronic device displays the first jump prompt information, the method further includes: jumping from the document interaction window to the global interaction window; generating a fourth instruction based on a non-document task operation sequence, the fourth instruction instructing the execution of the first task; displaying the fourth instruction in the global interaction window; automatically executing the first task in response to the fourth instruction; and displaying the execution result of the first task in the global interaction window.

[0052] For example, based on the non-document task operation sequence ['3', 'not_supported(query=buy a plane ticket)'], the fourth instruction "buy a plane ticket" can be generated, corresponding to the first task being to buy a plane ticket.

[0053] With the solution proposed in this application, for non-document tasks, one can jump to a global interactive window and then process the non-document tasks. For example, first, intent recognition is performed based on the DSL data of the non-document task, then the vertical domain is distributed, the task agent model corresponding to the non-document task is determined, and inference is performed based on the task processing model to obtain the task orchestration result in DSL format; then, the non-document task can be automatically executed based on the task orchestration result in DSL format.

[0054] In some possible implementations, the automatic execution of the first task in response to the fourth instruction includes: waking up the system assistant application of the electronic device in response to the fourth instruction; the system assistant application performing intent recognition based on the fourth instruction to determine the first task; invoking a first task proxy model to perform reasoning to obtain a first task operation sequence in DSL format; and automatically executing the first task based on the first task operation sequence.

[0055] For example, the first task is to buy a plane ticket, and the sequence of operations for the first task can be represented as follows:

[0056] [['1','open_app(app=xxx)'],['2','buy_ticket(from=xxx,to=xxx)']].

[0057] In some possible implementations, the step of invoking the first task proxy model to perform inference and obtain the first task operation sequence in DSL format includes: integrating the LoRa data corresponding to the first task with a general proxy model to obtain the first task proxy model; and performing inference based on the first task proxy model, according to the prompt cache information corresponding to the first task and the fourth instruction, to obtain the first task operation sequence in DSL format.

[0058] In some possible implementations, the step of reasoning based on the prompt cache information corresponding to the first task and the fourth instruction to obtain the first task operation sequence in DSL format based on the first task proxy model includes: obtaining the prompt cache information corresponding to the first task, wherein the prompt cache information is in DSL format and includes a task identifier, a tool name, and multiple tool parameters, each tool parameter corresponding to a slot; inputting the prompt cache information and the fourth instruction into the first task proxy model; and filling the slots in the prompt cache information according to the fourth instruction to generate the first task operation sequence in DSL format.

[0059] The task processing method provided by the embodiments of this application can not only automatically execute a series of preset task operations, but also handle complex operation sequences, realize full-process page jump of the task, and accurately identify the target control in each page after jump, and simulate click events on the target control to realize automatic task execution.

[0060] In some possible implementations, the first task includes any of the following: querying automatic renewal, sending a file, disabling application permissions, optimizing overall device performance, disabling application notifications, and ordering takeout.

[0061] In some possible implementations, in response to the user's second operation, the system assistant application of the electronic device is woken up and a global interaction window is displayed. The system assistant application detects whether the user has entered a command in the global interaction window. When the system assistant application detects the first command, it performs intent recognition based on the first command to determine the first task. The first task proxy model is called to obtain the first task operation sequence in DSL format. Based on the first task operation sequence, the first task is automatically executed.

[0062] The above solution enables the automatic execution of one or more tasks in response to user commands. It provides a global interaction window and a document interaction window as entry points for different task scenarios, allowing users to input commands. After receiving a command from the user in the document interaction window, the electronic device can recognize and respond to both document commands and complex commands unrelated to the document. Document tasks are automatically executed within the document interaction window, and for non-document tasks requested in the window, a specific task loop can be completed, improving task processing compatibility and the success rate of automatic task execution, thereby enhancing user experience and operational efficiency.

[0063] In some possible implementations, the jump from the document interaction window to the global interaction window includes: automatically jumping from the document interaction window to the global interaction window; or, jumping from the document interaction window to the global interaction window in response to a user's confirmation operation.

[0064] This application's embodiments can accurately identify whether a task is a document task, a non-document task, or a complex task based on user instructions. Complex tasks include one or more document tasks and one or more non-document tasks. For various possible scenarios where user instructions are entered in the document interaction window, this application provides the following three implementation methods:

[0065] Scenario 1: Document Task. If the electronic device recognizes a user instruction indicating a document task, it can directly process that task.

[0066] Scenario 2: Complex Tasks. If the electronic device recognizes that the user's command includes both document and non-document tasks, it can directly process the document task and indicate that the current task includes a non-document task. After the document task is completed, it automatically jumps to the global interaction window and executes the non-document task. Alternatively, if the electronic device recognizes that the user's command includes both document and non-document tasks, it can directly process the document task and indicate that the current task includes a non-document task, suggesting that the non-document task can be executed after jumping to the global interaction window. After receiving user confirmation, the electronic device jumps from the document interaction window to the global interaction window and automatically executes the non-document task.

[0067] Scenario 3: Non-Document Tasks. If the electronic device recognizes a user command indicating a non-document task, it can indicate that the current task is a non-document task, then navigate to the global interaction window and automatically execute the non-document task. Alternatively, if the electronic device recognizes a user command indicating a non-document task, it can indicate that the current task is a non-document task and suggest that the non-document task can be executed after navigating to the global interaction window; after receiving user confirmation, the electronic device will navigate from the document interaction window to the global interaction window and automatically execute the non-document task.

[0068] This application's embodiments improve upon both the task processing method and the document proxy model. Before the improvement, for a standalone document interaction window, only user-inputted document commands could be recognized and document tasks processed; however, it could not automatically execute tasks when receiving complex user input commands. After the improvement, using the task processing method provided in this application's embodiments, for a standalone document interaction window, when receiving complex user input commands, complex tasks can be identified and broken down to obtain DSL data for document tasks and DSL data for non-document tasks. Then, document tasks and non-document tasks are executed according to different task processing methods.

[0069] The task processing method provided in this application, for a separately established document interaction window, can identify and decompose complex tasks upon receiving complex user input instructions. Furthermore, it can distribute the decomposed tasks to their respective task proxy models, each employing a different task processing flow to complete the task. This improves the compatibility of task processing and increases the success rate of automated task execution, thereby enhancing user experience and operational efficiency.

[0070] Secondly, this application provides a task processing apparatus, which includes units for performing the method described in the first aspect above. This apparatus is capable of performing the method described in the first aspect above, and a description of the units within this apparatus is provided in the description of the first aspect above; for brevity, it will not be repeated here.

[0071] The method described in the first aspect above can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions. For example, a processing module or unit, a display module or unit, etc.

[0072] Thirdly, this application provides an electronic device, which includes a processor, a computer program or instructions stored in a memory, wherein the processor is used to execute the computer program or instructions to cause the method in the first aspect to be performed.

[0073] Fourthly, this application provides a computer-readable storage medium having a computer program (also referred to as instructions or code) stored thereon for implementing the method of the first aspect. For example, when the computer program is executed by a computer, it enables the computer to perform the method of the first aspect.

[0074] Fifthly, this application provides a chip including a processor. The processor is used to read and execute a computer program stored in a memory to perform the methods in the first aspect and any possible implementation thereof. Optionally, the chip further includes a memory connected to the processor via a circuit or wire.

[0075] Sixthly, this application provides a chip system including a processor. The processor is used to read and execute a computer program stored in a memory to perform the methods in the first aspect and any possible implementation thereof. Optionally, the chip system further includes a memory connected to the processor via a circuit or wire.

[0076] In a seventh aspect, this application provides a computer program product comprising a computer program (also referred to as instructions or code), which, when executed by an electronic device, causes the electronic device to implement the method in the first aspect.

[0077] It is understood that the beneficial effects of the second to seventh aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0078] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0079] Figure 2 A schematic diagram of the software architecture of an electronic device provided in an embodiment of this application;

[0080] Figure 3 A flowchart illustrating the task processing method provided in the first embodiment of this application;

[0081] Figure 4 A schematic diagram of the interactive interface of the task processing method provided in the first embodiment of this application;

[0082] Figure 5 A flowchart illustrating the task processing method provided in the first embodiment of this application;

[0083] Figure 6A and Figure 6B A schematic diagram of the interface for applying the task processing method provided in the first embodiment of this application to a file sending scenario;

[0084] Figure 7 A schematic diagram of the interface for applying the task processing method provided in the first embodiment of this application to a food delivery scenario;

[0085] Figure 8A A flowchart illustrating the task processing method provided in the first embodiment of this application;

[0086] Figure 8B A flowchart illustrating the task processing method provided in the first embodiment of this application;

[0087] Figure 9 A flowchart illustrating the task processing method provided in the second embodiment of this application;

[0088] Figure 10 A schematic diagram of the interactive interface of the task processing method provided in the second embodiment of this application;

[0089] Figure 11 A schematic diagram of the interactive interface of the task processing method provided in the second embodiment of this application;

[0090] Figure 12 A flowchart illustrating the task processing method provided in the second embodiment of this application;

[0091] Figure 13 A schematic diagram illustrating an improvement to the task processing method provided in the third embodiment of this application;

[0092] Figure 14 A schematic diagram illustrating the task identification based on user instructions in the task processing method provided in the third embodiment of this application;

[0093] Figure 15 This is a schematic diagram of the overall flow of the task processing method provided in the third embodiment of this application;

[0094] Figure 16 A flowchart illustrating the application of the task processing method provided in the third embodiment of this application to document processing tasks;

[0095] Figure 17 A flowchart illustrating the task processing method provided in the third embodiment of this application applied to processing non-document tasks;

[0096] Figure 18 A flowchart illustrating the application of the task processing method provided in the third embodiment of this application to handle complex tasks;

[0097] Figures 19 to 21 A schematic diagram of the user interface for the task processing method provided in the third embodiment of this application. Detailed Implementation

[0098] The embodiments of this application will be further described in detail below with reference to specific examples and the accompanying drawings.

[0099] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0100] In the following embodiments of this application, the term UI refers to the medium interface through which an application or operating system interacts and exchanges information with the user. It realizes the conversion between the internal form of information and the form that the user can accept. The user interface is source code written in a specific computer language such as Java or Extensible Markup Language (XML). The interface source code is parsed and rendered on the electronic device, ultimately presenting content that the user can recognize. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be visible interface elements such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets displayed on the screen of an electronic device.

[0101] In daily smartphone use, users often need to frequently perform a series of operations to complete specific tasks, such as processing documents, ordering takeout, turning off auto-renewal services, sending files, or other possible tasks. However, manually performing each step of these tasks is not only inconvenient but also prone to errors. Some tasks even have overly complex operation paths that are difficult for users to find, making it impossible to complete the operation smoothly and affecting the user experience.

[0102] To address the aforementioned issues, this application provides a task processing method and an electronic device capable of automatically executing one or more tasks in response to user instructions. It provides a global interaction window and a document interaction window as entry points for different task scenarios, allowing users to input instructions. After receiving instructions from the user in the document interaction window, the electronic device can recognize and respond to both document-related instructions and complex instructions unrelated to documents. Document tasks are automatically executed in the document interaction window, and for non-document tasks requested in the document interaction window, a task loop can be completed in a specific manner, improving the compatibility of task processing and the success rate of automatic task execution, thereby enhancing user experience and operational efficiency.

[0103] The global interactive window is an interactive window or interface that enables the automatic execution of various tasks, such as through system assistant applications. Users can input commands such as ordering takeout, checking bill renewals, and sending files through the global interactive window to trigger electronic devices to automatically perform tasks such as ordering takeout, checking bill renewals, and sending files.

[0104] The document interaction window is a dedicated interactive window or interface for handling document tasks, and can be provided through document intelligence applications, for example. Users can input document-related commands through the document interaction window to trigger electronic devices to automatically execute document tasks.

[0105] According to the solution in this application, after receiving instructions input by the user in the document interaction window, the electronic device can recognize and respond to both document instructions and non-document instructions, as well as complex instructions containing both document-related and document-unrelated information. The document proxy model supports both in-document and out-of-document detection, enabling the identification of document and non-document tasks based on user instructions. Furthermore, it supports task decomposition, employing different processing flows to complete document and non-document tasks separately, thus improving the compatibility of task processing.

[0106] In practical applications, many business operations provided by electronic devices involve numerous steps, complex operation paths, or a large amount of specific information that requires repeated manual input. The task processing method provided in this application can improve these problems and provide great convenience to users.

[0107] The following description, in conjunction with the accompanying drawings, describes the electronic device to which the task processing method provided in this application is applied. For example, the electronic device in this application may be a mobile phone, tablet computer, ultra-mobile personal computer (UMPC), netbook, cellular phone, personal digital assistant (PDA), wearable device (such as a smartwatch, smart bracelet), or other device with voice wake-up functionality. This application does not impose any special limitations on the specific form of the electronic device.

[0108] In some examples, users can input commands via their mobile phones to trigger the phones to automatically perform tasks.

[0109] In other examples, users can quickly issue commands through wearable devices and forward them to their mobile phones, which then automatically execute the tasks. The mobile phones then return the task completion results to the wearable devices, which display the results.

[0110] Figure 1 This illustration shows a hardware structure diagram of an electronic device according to an embodiment of this application. For example, a mobile phone is used as an example. Figure 1As shown, the electronic device may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver (i.e., earpiece) 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0111] The aforementioned sensor modules may include sensors such as pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, and bone conduction sensors.

[0112] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0113] A controller can be the nerve center and command center of an electronic device. Based on the instruction opcode and timing signals, the controller generates operation control signals to control the fetching and execution of instructions.

[0114] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0115] It is understood that the interface connection relationships between the modules illustrated in this embodiment are merely illustrative and do not constitute a structural limitation on the electronic device. In other embodiments, the electronic device may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0116] In this embodiment of the application, the electronic device can implement the method provided in this embodiment of the application through the processor 110.

[0117] The above is a detailed description of the embodiments of this application using electronic device 100 as an example. It should be understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on electronic device 100. Electronic device 100 may have more or fewer components than shown in the figures, may combine two or more components, or may have different component configurations. The various components shown in the figures can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0118] In addition, an operating system runs on top of the aforementioned hardware. This operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux, Unix, Android, iOS, or Windows. Applications can be installed and run on this operating system.

[0119] The operating system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0120] Figure 2 This is a software structure block diagram of the electronic device 100 according to an embodiment of this application.

[0121] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, Android... TM The system is divided into five layers, from top to bottom: the application layer, the application framework layer, the Android runtime (Android...). TM The runtime layer, system library layer, hardware abstraction layer (HAL), and kernel layer.

[0122] The application layer can include a series of application packages. For example, the application layer includes, but is not limited to, game apps, map apps, short video apps, social apps, shopping apps, etc.

[0123] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions. In this embodiment, the application framework layer may include a window manager service (WMS), an input manager service (IMS), virtual event handlers, and a view system, etc., and this embodiment does not impose any limitations on these aspects.

[0124] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views.

[0125] For example, in embodiments of this application, the view system can be used to display dynamic light effect tooltips in the auto-execution window to indicate that the current page is a window for auto-execution tasks. The view system can also be used to display prompt information in the auto-execution window to indicate the specific actions or steps of the UI task being performed on the current page. The view system can also be used to display a minimize control (also called a minimize to small window button) in the auto-execution window to trigger a switch from full-screen mode to small window mode. The view system can also be used to display a stop control in the auto-execution window to trigger a stop to the execution of the UI task.

[0126] The Input Management Service (IMS) is used to manage input events. For example, in this embodiment, input events may include virtual events such as virtual click events, virtual swipe events, and virtual input events. Input events may also include user operation events such as user click events, user swipe events, and user input events.

[0127] Virtual event handlers are used to process virtual events.

[0128] For example, in this embodiment of the application, during the automatic execution of UI tasks, the virtual event handler can implement the following virtual events:

[0129] Virtual click events refer to the system automatically clicking on target controls related to UI tasks within an auto-execution window.

[0130] Virtual swipe events are events in which the system automatically triggers a swipe operation within a specific area related to a UI task in an auto-execution window.

[0131] Virtual input events allow the system to automatically input specific text or symbols related to UI tasks into input boxes within automated execution windows. The notification manager enables applications to display notifications in the status bar; these can be used to convey informational messages and can disappear automatically after a short pause without user interaction.

[0132] The Window Management Service (WMS) is used to manage window programs. For example, the window manager can control whether an autorunning window is displayed in full-screen mode, in a small window mode, or switch between full-screen (or full-screen window) mode and small window mode, or vice versa.

[0133] The Android runtime is responsible for scheduling and managing the Android system. The system library layer, also known as the native layer, native service layer, or native framework layer, is a native code platform library that can provide native services and link libraries for hardware operations to the upper layers.

[0134] The HAL layer is a wrapper around Linux kernel drivers, providing interfaces to the upper layers and shielding them from the implementation details of the lower-level hardware.

[0135] The kernel layer is the layer between hardware and software. The kernel layer contains at least display drivers, audio drivers, and sensor drivers.

[0136] For clarity, the diagram also illustrates the hardware layer that interacts with the aforementioned software structure. For example, the hardware layer may include hardware such as displays and sensors.

[0137] It should be noted that although the embodiments of this application are described using the Android system as an example, the basic principles are also applicable to electronic devices based on operating systems such as iOS or Windows.

[0138] The execution subject of the task processing method provided in this application embodiment can be the aforementioned electronic device, or a functional module and / or functional entity within the electronic device capable of implementing the task processing method. Furthermore, the solution of this application can be implemented through hardware and / or software, the specific implementation of which can be determined according to actual usage requirements; this application embodiment does not impose any limitations. The following description uses an electronic device as an example, in conjunction with the accompanying drawings, to exemplarily illustrate the task processing method provided in this application embodiment.

[0139] The embodiments of this application will be described below with reference to the accompanying drawings and through several exemplary embodiments. The methods in the following embodiments can all be implemented in an electronic device having the above-described hardware structure and software architecture. The hardware structure diagram of the electronic device can be as follows: Figure 1 As shown, the software structure block diagram of an electronic device can be as follows: Figure 2As shown, the embodiments of this application are not limited to this. For ease of explanation, the electronic device used in the embodiments of this application is a mobile phone.

[0140] The task processing method provided in this application is described below with reference to specific embodiments.

[0141] First, it should be noted that various tasks exist in actual use, such as processing documents, ordering takeout, checking and renewing bills, sending files, or other possible tasks. This application provides a task processing method applicable to a global scenario, capable of handling tasks in different vertical domains. Furthermore, considering that users frequently handle document tasks in their daily lives and work, this application provides a document task processing method specifically for the document vertical domain, capable of supporting document task processing and providing users with powerful document-related services.

[0142] The following three embodiments illustrate the possible implementations of the task processing method provided in this application.

[0143] First Implementation: Global Task Processing Method

[0144] This application provides a task processing method applicable to a global scenario. Using a global interactive window as the user interaction entry point, when the electronic device receives a task instruction input by the user in the global interactive window, the electronic device automatically executes the task according to the user's input. The global interactive window serves as the interaction entry point for various tasks.

[0145] Figure 3 A flowchart illustrating a task processing method applied to a global scenario, as provided in an embodiment of this application, is shown. Figure 3 As shown, the system assistant of the electronic device is awakened after receiving user input. The system assistant displays a global interaction window, through which the user can interact with the system assistant. When the system assistant receives a user instruction, it automatically executes the task according to the user instruction and then replies to the user instruction, such as displaying the automatic execution result.

[0146] Figure 4 This diagram illustrates the global interactive window of the task processing method provided in an embodiment of this application. Figure 4 As shown in (a), after the system assistant is activated, it displays a global dialog interface. The system assistant receives the user's command, "Cancel automatic renewal for all applications." In response to the user's voice command, the electronic device activates the system assistant and displays the message, "Cancel automatic deductions for all applications." Figure 4 As shown in (b), the system assistant responds to the user's command, displaying the message "Okay, will automatically check and cancel all application auto-renewal," and then the electronic device automatically checks the auto-renewal services, such as... Figure 4 As shown in (c), the electronic device displays the results of the automatic execution to the user.

[0147] Figure 5 The diagram shows a flowchart of a task processing method applied to a global scene according to the first embodiment of this application.

[0148] S201. Receive user instructions, perform intent recognition on user instructions, and determine the first task.

[0149] In this embodiment, when a user wishes to use an electronic device to inquire about automatic renewal, send files, order takeout, etc., the user can input a command into the electronic device. After receiving the user's command, the electronic device can perform intent recognition based on the user's command to obtain user intent information. One piece of user intent information corresponds to one or more tasks. The aforementioned first task includes one or more tasks.

[0150] In this embodiment, the electronic device receives a user instruction, performs semantic recognition processing on the user instruction, obtains keywords, and then determines the user intent information based on the keywords.

[0151] For example, in some scenarios, the user's instruction may be "Please help me check which application is automatically deducting fees", or "Please help me check if application / service XX is automatically deducting fees", or "Please help me turn off all automatic deduction services", or "Please help me turn off the automatic deduction service of application / service XX". The electronic device can determine the user's intent information as "check automatic deduction" by performing semantic recognition based on the user's instruction.

[0152] For example, in some scenarios, user instructions might be "Send me a file to someone" or "Send me a WeChat message." TM When users send a photo of a received plane ticket to someone, or ask them to send a photo to someone, electronic devices can determine the user's intent as "to send a file to someone" by semantically recognizing the user's instructions.

[0153] For example, in some scenarios, the user's instruction is "order me a large iced Americano" or "order me an iced coffee and a hot milk tea". Through semantic recognition, the user's intent information is obtained as "order takeout".

[0154] User commands can be voice input and / or text input.

[0155] User commands can activate the smart assistance applications on electronic devices. These smart assistance applications can be system assistant applications or other auxiliary applications. System assistant applications are system applications within electronic devices.

[0156] Electronic devices offer a variety of wake-up methods, such as voice wake-up, power button wake-up, shortcut key wake-up, swipe wake-up, gesture wake-up, search wake-up, and breath wake-up.

[0157] Voice wake-up is the most common method, allowing users to activate the system assistant by saying "Hello, YOYO" to their phone. Power button wake-up allows users to press and hold the power button until the system assistant is activated. Shortcut key wake-up involves adding a shortcut to the system assistant on the desktop, allowing users to quickly wake it up with a single tap. Swipe wake-up and gesture wake-up offer more personalized operation methods, allowing users to activate the system assistant by setting specific swipe actions or gestures. Search wake-up involves searching for "voice assistant" on the screen to invoke the system assistant. Breath wake-up is a convenient way to activate the system assistant using breath; users can bring the bottom of the phone close to their mouth and speak a voice command without using a specific wake-up word. This method is more natural and convenient. Users can wake up the system assistant using any of the above methods. For ease of explanation, the following examples use voice wake-up as an example.

[0158] In some embodiments, when the electronic device displays a desktop or application page, the electronic device can receive user input instructions, wake up the system assistant, and determine user intent information based on the user instructions.

[0159] In this embodiment of the application, the electronic device can identify keywords in the user instructions according to the user instructions, and determine the user intent information corresponding to the keywords according to the preset correspondence between the keywords and the user intent information.

[0160] In this embodiment, the electronic device has multiple preset user intent information, each corresponding to one or more keywords. Table 1 schematically illustrates the correspondence between preset user intent information and keywords.

[0161] Table 1

[0162] sequence User intent information Keywords Vertical mission 1 Check automatic renewal Automatic, renewal, deduction Query renewal task 2 Send a file to someone Send, File Send file task 3 Ordering takeout Orders, coffee, milk tea, takeout Ordering takeout

[0163] It should be noted that the user intent information in Table 1 is illustrative and can be used to illustrate the concepts. In actual implementation, other user intent information can be preset, such as opening App1 to send a red envelope to someone. This application embodiment does not limit the specific content of the preset user intent information. Similarly, the keywords in Table 1 are illustrative and can be set according to actual needs in actual implementation. This application embodiment does not limit the specific content of the preset user intent information.

[0164] S202. Input the user command into the first task agent model to obtain the task operation sequence.

[0165] The task operation sequence includes multiple steps, each step including a page, the target control on the page, the UI task, and whether the user takes over.

[0166] It should be noted that the system assistant can respond to user commands and trigger electronic devices to automatically execute tasks. This is achieved using RPA (robotic process automation) technology, which automates clicks and page redirects to achieve the goal of automatic task execution. For example, scenarios for automatically executing tasks may include checking payment renewals, sending files, ordering takeout, etc.

[0167] This application employs RPA technology to automate task execution. RPA technology can simulate user operations on electronic devices, such as keyboard input and mouse clicks, using pre-defined programs. It can automatically execute repetitive, regular, and predictable business processes or tasks, thereby automating business processes.

[0168] The automatic execution information for a single step includes the page, the target control on that page, the UI task, and whether the user has taken over. For ease of explanation, this is denoted as step i (page, target control, UI task, user takeover). Step i represents the i-th step. i is a positive integer.

[0169] In some embodiments, the target control on the page is at least one of the following: an icon, a button (e.g., a circular button), a toggle switch, or text. The corresponding UI tasks include, but are not limited to, operations supported on the device such as simulating actions, automatically clicking, swiping, long-pressing the target control, dragging, multi-finger pressing, and deep linking.

[0170] For example, the user intent information is "optimize overall system performance", and the corresponding task operation sequence includes the following two steps. Step 1 (System Manager page, one-click optimization option, click the one-click optimization option, no user intervention required); Step 2 (One-click optimization page, completion option, click the completion option, no user intervention required).

[0171] In some embodiments, user intervention is reflected in the steps when it is required. If a step does not require user intervention, it may not be reflected in the steps.

[0172] For another example, if the user intent is "turn off notifications for App1", the corresponding task operation sequence includes the following three steps: Step 1 (Settings main page, notification options, click the notification option); Step 2 (Notification settings page, notification management options, click the notification management option); Step 3 (Notification management page, App1's on / off option, slide the App1's on / off option).

[0173] For another example, if the user intent is "disable the browser's microphone permission", the corresponding task operation sequence includes the following 5 steps: Step 1 (Settings main page, Application options, click Application options); Step 2 (Application settings page, Permission management options, click Permission management options); Step 3 (Permission management page, Microphone options, click Microphone options); Step 4 (Microphone permission settings page, Browser options, click Browser options); Step 5 (Browser microphone permission settings page, Disable option, click Disable option).

[0174] It should be noted that the above task operation sequence is an illustrative example, and the task operation sequence in this application embodiment is not limited to the above situation. In actual implementation, the number of steps in the task operation sequence can be set according to actual usage requirements, and the page information, target control, UI task, and / or whether user intervention is required in each step can be determined according to actual usage requirements.

[0175] For example, if the user intent is "disable the browser's microphone permission", the corresponding task sequence may include the following three steps: Step 1 (Permission management page, microphone option, click the microphone option); Step 2 (Microphone permission settings page, browser option, click the browser option); Step 3 (Browser microphone permission settings page, disable option, click the disable option).

[0176] For another example, if the user intent is "to disable the browser's microphone permission", the corresponding task sequence could include the following one step: Step 1 (Browser's microphone permission settings page, disable option, click disable option).

[0177] In other words, after initiating the automatic execution process, the electronic device directly redirects to the "Browser Microphone Permission Settings Page." On this page, the target control, the "Disable Option," is identified, its coordinates are determined, and then a simulated click operation is used to select the Disable Option. This completes the task of disabling the browser's microphone permission.

[0178] In other embodiments, the target control on the page is a search box, and the corresponding UI task includes automatically entering information into the search box through simulated operation.

[0179] For example, to retrieve user intent information such as sending a file to someone, disabling a permission for an application, or disabling notifications for an application, you need to enter information such as the application name, person's name, file name, or permission name in the search box to find specific objects such as applications, specific people, specific files, or specific permissions.

[0180] Table 2 shows the task operation sequence, including the search box and the case where information is entered into the search box.

[0181] Table 2

[0182]

[0183]

[0184] For example, if the user intent is "turn off notifications for App1", the corresponding task sequence includes the following three steps: Step 1 (Settings main page, notification options, click the notification options); Step 2 (Notification settings page, search box, automatically enter App1 in the search); Step 3 (Notification management page, App1's on / off option, identify the on / off status and slide the on / off option for App1).

[0185] It should be noted that the above task operation sequences are all illustrative. It can be understood that in actual implementation, the task operation sequence corresponding to each user intent information can be determined according to actual usage needs, and this application embodiment does not limit it.

[0186] S203. Based on the task operation sequence, begin the automatic execution process of the first task.

[0187] The automated execution process may include identifying the target control on each page and determining the position of the target control, simulating clicking the target control or entering information in the search box, and redirecting to the next page after the click operation or the information entered in the search box.

[0188] In this embodiment of the application, the electronic device can automatically execute tasks by simulating click operations, swipe operations, or input operations according to the task operation sequence.

[0189] For example, an electronic device can automatically jump to a page of a UI task based on the sequence of task operations. According to the UI task corresponding to that page, it can identify the target control on the page and determine the position of the target control. Then, it can complete the click on the target control by simulating a click operation, and then automatically jump to the next page.

[0190] For example, an electronic device can automatically jump to a page of a UI task based on the sequence of task operations. According to the UI task corresponding to that page, it can identify the search box on the page and determine its position. Then, by simulating an input operation, it can enter the slot information of the UI task into the search box and then automatically jump to the next page.

[0191] In some embodiments, when an electronic device simulates a click, input, or swipe operation on a page during an automated execution process, a cursor can be displayed at the click, input, or swipe location on the page. This makes each execution step of the automated execution process very intuitive and can improve the user experience.

[0192] In this application, the display size, display format, display color, etc. of the cursor are not limited, and can be determined according to actual usage requirements.

[0193] S204. After the automatic execution process is completed, display the automatic execution result.

[0194] The task processing method provided in this application can be applied to various user scenarios. For example, a user can request to query the application's automatic renewal service, or a user can request to send a file to someone, or a user can request to disable application permissions, or a user can request to optimize the overall performance of the device, or a user can request to disable application notifications, or a user can request to order takeout (such as coffee or milk tea) through a shopping application.

[0195] The following, in conjunction with the interface diagrams, uses the processing flow of sending files and ordering takeout as examples to exemplify the task processing method provided in the first embodiment of this application.

[0196] Example 1: Processing flow of a file sending task

[0197] The task processing method provided in this application embodiment allows an electronic device to automatically send a file to user B in response to a user A's request. The device navigates through various pages according to a path, first searching for the file and then navigating to the file interface, followed by a sharing page to send the file to user B. Table 3 illustrates various scenarios for sending a file to someone and exemplary user input commands.

[0198] For example, the user enters the instruction "Help me send the xx file to someone".

[0199] For example, a user might enter the instruction "Help me send the photo of the airline ticket received by App1 to someone".

[0200] For example, a user might enter the command "Send me a photo to someone".

[0201] Electronic devices can respond to user commands, automatically search for files on the device by file name, quickly display the search results to the user, and automatically send the files to a specified user.

[0202] Table 3 schematically illustrates the task operation sequence in the automated file sending process, where the user input command is "Send document 1 to Zhang San". The task operation sequence includes page navigation information, target controls on each page, and the corresponding UI tasks for each page, indicating whether user interaction is required.

[0203] Table 3

[0204] step page Target control UI Tasks User takeover 1 File Management Main Page search box Simulate input of "Document 1" / 2 Search results page Document 1 / Button Simulate clicking the target control / 3 Application icon page App1 Simulate clicking the target control / 4 App1 Contacts Page search box Simulate inputting "Zhang San" / 5 Search results page Zhang San / Select Button Simulate clicking the target control / 6 Share confirmation page share Simulate clicking the target control / 7 Share completed page return Simulate clicking the target control /

[0205] The target control can be text, an icon, a button, and / or a search box, or it can be other forms.

[0206] The following diagram illustrates the interface implementation of the automatic file sending process.

[0207] Figure 6A and Figure 6B The diagram illustrates an interface of the task processing method provided in this application embodiment applied to a file sending scenario.

[0208] like Figure 6A As shown in (a) to (f), in the global interaction window, the electronic device receives the user's instruction "Send document 1 to Zhang San," waking up the system assistant. Responding to the user's input, the system assistant replies "Okay, now jump to file management to find document 1 and send it to Zhang San." The electronic device can recognize that the user's request is to send a file based on the user's input, and determine that the file sending scenario is searching for a file by name. Based on this, it determines the task operation sequence and begins the automatic execution process.

[0209] After initiating the automated process, the electronic device automatically navigates to the file management page, identifies the target control "search box" on the page, determines its coordinates, and selects the search box at those coordinates using a simulated click. The user then enters "Document1" into the search box. The electronic device displays that a file has been found, along with its name "Document1" and a circle control. The device then identifies the target control "circle control," determines its coordinates, and selects it at those coordinates using a simulated click.

[0210] Once the automatic execution process begins, a light effect notification icon appears on the full screen, and the current status "Searching for you in file management" is displayed at the bottom of the page, along with a stop button and a shrink window button.

[0211] After automatically selecting a file, the electronic device automatically triggers file sharing and displays the application icon page. The electronic device identifies the target control "App1" on the application icon page and determines the position coordinates of the target control "App1". At the determined position coordinates, it selects the target control "App1" by simulating a click operation.

[0212] Then, as Figure 6B As shown in (g) to (k), the file management page automatically redirects to the App1 contacts page. After redirecting to the App1 page, the status information changes to "Executing automatically" or "Sending to you by App1".

[0213] Then, the electronic device identifies the target control "search box" on the contacts page and determines its location coordinates. At the determined coordinates, it simulates a click to select the search box and enters "Zhang San" into it. The electronic device displays that "Zhang San" has been found. If no control is available for the user to select, the text "Zhang San" is the default target control. The electronic device identifies the target control "Zhang San" and determines its location coordinates, then simulates a click to select the text "Zhang San" at the determined coordinates.

[0214] After selecting the text "Zhang San", a sharing confirmation window pops up. The electronic device recognizes the target control "Share" in the sharing confirmation window and determines the position coordinates of the target control. At the determined position coordinates, the target control "Share" is selected by simulating a click operation.

[0215] After selecting the text "Zhang San", a sharing completion window pops up. The electronic device recognizes the target control "Back" in the sharing completion window and determines the position coordinates of the target control. At the determined position coordinates, the target control "Back" is selected by simulating a click operation.

[0216] After selecting the text "Return", the automatic execution result is displayed, such as "Document 1 has been sent to Zhang San".

[0217] Example 2: Processing flow of food delivery tasks

[0218] The task processing method provided in this application embodiment can respond to a user's order for takeout, determine an automatic execution path / step, and automatically redirect the user to the beverage ordering page based on the automatic execution path / step, placing the order for the user according to the user's operation. For example, the takeout ordering scenario can include ordering beverages such as coffee / milk tea, as well as other food ordering scenarios. For ease of explanation, the following description uses ordering beverages as an example.

[0219] Through the task processing method provided in this application embodiment, the electronic device can respond to user instructions (with detailed information), automatically search for the food delivery page of a certain application on the electronic device, quickly display the search results to the user, and place an order for the user based on the user's operation.

[0220] For example, the user instruction is "Order me a large iced Americano from store C". Table 4 schematically illustrates the task operation sequence in the automated ordering process. The task operation sequence includes page navigation information, target controls on each page, and the corresponding UI tasks for each page, as well as whether the user takes over.

[0221] Table 4

[0222]

[0223] Figure 7 This illustration shows a schematic diagram of the interface when the task processing method provided in this application is applied to a food delivery scenario. Figure 7 As shown in (a) to (h), in the global interactive window, the user's command is "Order me a large iced Americano from store C". The electronic device responds to the user's command and initiates the automatic execution process corresponding to ordering takeout. After the automatic execution process is started, the electronic device displays each page of the automatic execution process in a full-screen window, displays dynamic light effects around the window, and displays the status information "Automatically executing" on each page.

[0224] The electronic device automatically redirects to the food delivery page, identifies the target control "search box" on the page, determines the coordinates of the search box, and then simulates an input operation by entering the text "C store" into the search box to achieve the purpose of automatic search.

[0225] Then, the electronic device automatically redirects to the C-store page, identifies the target control "search box" on the C-store page, determines the position coordinates of the search box, and then simulates an input operation by entering the text "Americano coffee" into the search box to achieve the purpose of automatic search.

[0226] Then, the electronic device automatically jumps to the product page, identifies multiple specifications such as cup type: large and temperature: ice, determines the position coordinates of target controls such as "large" and "ice", and then simulates a click operation to select target controls such as "large" and "ice".

[0227] After automatically selecting the specifications, the system identifies the target control "Add to Cart" on the product page, determines the coordinates of "Add to Cart", and then simulates a click operation to select "Add to Cart".

[0228] Then, the electronic device automatically redirects to the shopping cart page, and the automated process ends.

[0229] After the automatic execution process is completed, the electronic device will stop displaying the dynamic light effect pattern and update the status information to "Added to cart, please checkout," indicating that the user is in control. The user can click the checkout button on the shopping cart page according to their actual needs, and then click the "Pay Now" button on the order payment page to complete the task of ordering drinks.

[0230] This application also provides a task proxy model applied to the above-described task processing method. This task processing model can be applied to tasks in different vertical domains. When an electronic device implements the above-described task processing method applied to a global scenario, the electronic device can use this task processing model to process various tasks (such as ordering takeout, querying renewals, and sending files).

[0231] In this embodiment, LoRa data from different tasks can be integrated based on a general proxy model to obtain proxy models for different tasks. The following describes the implementation of the general proxy model, LoRa data for various tasks, and how to integrate the general proxy model and LoRa data for various tasks to obtain task proxy models.

[0232] General Proxy Model

[0233] In this embodiment, a general proxy model can be provided for tasks in different vertical domains. This general proxy model is a large model trained based on various business data, and has wide applicability.

[0234] A general-purpose proxy model is applicable to tasks across different vertical domains. For example, a general-purpose proxy model can handle tasks such as ordering takeout, querying renewals, and sending files.

[0235] Configure LoRa data for different vertical domain tasks

[0236] In this embodiment of the application, LoRa data can be configured separately for tasks in different vertical domains. For example, LoRa data for ordering takeout can be configured for the takeout ordering task; LoRa data for querying renewal can be configured for the renewal query task; and LoRa data for sending files can be configured for the file sending task.

[0237] For example, corresponding to an independent card vertical domain (or independent vertical domain), the LoRa model for ordering takeout can be configured on a single GPU card. Corresponding to a shared card vertical domain, the LoRa model for querying renewals and the LoRa model for sending files can be configured on the same GPU card.

[0238] Using different vertical domains Task Integrate the configured LoRa data with the general proxy model

[0239] In the embodiments of this application, the electronic device can integrate LoRa data configured for different vertical domain tasks with a general proxy model to obtain proxy models for different vertical domain tasks.

[0240] For example, for a takeout ordering task, integrating the LoRa data configured for the takeout ordering task with a general proxy model can yield a proxy model specific to the takeout ordering task. This proxy model is simply referred to as the takeout ordering proxy model.

[0241] For example, for a query renewal task, integrating the LoRa data configured for the query renewal task with a general proxy model can yield a proxy model specific to the query renewal task. This proxy model is simply referred to as the query renewal proxy model.

[0242] For example, for a file sending task, integrating the LoRa data configured for the file sending task with a general proxy model can yield a proxy model specific to the file sending task. This proxy model is simply referred to as the file sending proxy model.

[0243] Therefore, by integrating the LoRa data configured for each vertical task with the general proxy model, a proxy model specific to each vertical task can be obtained. In comparison, the general proxy model has the advantage of wide applicability, while the proxy model for a specific vertical task obtained through integration has the advantage of stronger task processing performance.

[0244] It should be noted that in practical use, when the user command includes a first vertical domain task, the LoRa data configured for the first vertical domain task can be integrated with the general proxy model to obtain the proxy model for the first vertical domain task. Then, by performing inference through the proxy model of the first vertical domain task, the task orchestration result in DSL format can be obtained.

[0245] For example, after an electronic device receives a user's "send file" task command in the global interaction window, the device can identify the user's intent as "send file" and determine the "send file" vertical task. Then, the LoRa data for the "send file" vertical task is integrated with a general agent model to obtain a file-sending agent model. This agent model processes the "send file" task command and outputs a task orchestration result in DSL format. Based on the DSL-formatted task orchestration result, the electronic device automatically executes the file-sending task.

[0246] In an embodiment of the present application, task decomposition can be performed based on a vertical domain to obtain task agent models suitable for different tasks. The task agent model has the ability of task planning and can convert a user instruction into a task orchestration result in DSL format. For example, after the electronic device receives a user instruction, the electronic device can identify the user intention according to the user instruction, determine the first task, then obtain the task orchestration result in DSL format according to the task agent model corresponding to the first task, and then automatically execute the task according to the task orchestration result.

[0247] Exemplarily, referring to Figure 8A As shown, the electronic device can input the user instruction into a pre-trained task agent model, perform model inference through the task agent model, output a task orchestration result (referred to as DSL data for short) in the format of a domain-specific language (DSL), and then parse the DSL data and execute the task. Among them, the DSL description language is a programming language designed specifically for a certain specific domain.

[0248] For example, taking the task of ordering takeout (ordering drinks) as an example, the user instruction is "Help me order a large iced latte from store C". After the electronic device receives the user instruction, it can input the user instruction into the task agent model, perform model inference through the task agent model, and output DSL data. Exemplarily, the DSL data can be:

[0249] [['1', 'order(app = None, shop = store C, product = latte, no = 1, size = large, temperature = iced, sweetness = None, desLocation = None)']].

[0250] Alternatively, the DSL data can also be:

[0251] [[1, order(shop = store C, product = latte, no = 1, size = large, temperature = iced)]]

[0252] After the electronic device obtains the DSL data, it can further parse the DSL data and execute the takeout ordering task. In this way, the user only needs to input an instruction, and the electronic device can help the user complete a series of operations for ordering takeout through the App, improving the user experience.

[0253] For another example, consider creating a reminder task. The user's instruction is "Submit a meeting reminder for 2 PM today." After receiving the user's instruction, the electronic device inputs the instruction into the task agent model. The task agent model performs model inference and outputs DSL data corresponding to the created reminder task (create_reminder). For example, the DSL data could be: [[1, create_reminder(content = meeting reminder, time = 2 PM)]].

[0254] After acquiring the DSL data corresponding to the reminder creation task, the electronic device can then parse the DSL data and execute the "reminder creation task." In this way, the user only needs to input commands, and the electronic device can help the user complete a series of operations to create a reminder, improving the user experience.

[0255] For another example, consider a ride-hailing task where the user's instruction is "take a ride to xx building". After receiving the user's instruction, the electronic device inputs the instruction into the task agent model, performs model inference through the task agent model, and outputs DSL data corresponding to the ride-hailing task (take_taxi). For example, the DSL data can be: [[1, take_taxi(deplocation=None, deslocation=xx building, app=None)]].

[0256] After acquiring the DSL data corresponding to a ride-hailing task (take_taxi), the electronic device can then parse the DSL data and execute the task. In this way, the user only needs to input commands, and the electronic device can help the user complete a series of operations to hail a ride through the app, improving the user experience.

[0257] For another example, consider a navigation task. The user command is "navigate to xx building by bike". After receiving the user command, the electronic device inputs the user command into the task proxy model, performs model inference through the task proxy model, and outputs DSL data corresponding to the navigation task (navigate). For example, the DSL data can be: [[1, navigation(deplocation=None, deslocation=xx building, pathPoint=None, trafficType=cycling, app=None)]].

[0258] After acquiring the DSL data corresponding to the navigation task, the electronic device can parse the DSL data and execute the task. In this way, the user only needs to input commands, and the electronic device can help the user complete a series of operations to set up navigation through the app, improving the user experience.

[0259] The reasoning and training processes of the above task proxy model are explained in detail below.

[0260] Reasoning process of the task agent model

[0261] This application provides a task proxy model applicable to different tasks. It can decompose tasks based on vertical domains, integrate LoRa data configured in different vertical domains with a general proxy model, and then perform inference through the integrated task proxy model. The general proxy model is a large model trained based on various business data.

[0262] The following sections describe the overall reasoning process of the task proxy model, the use of a large language model (LLM) combined with a hint caching strategy for reasoning, the optimization of the DSL design, and the optimization of the model output.

[0263] Overall reasoning process

[0264] In this embodiment, the task proxy model needs to have intent recognition capabilities, and determine the LoRa data corresponding to the vertical domain based on the recognized intent information. This LoRa data is then integrated with the general proxy model. Different vertical domains provide LoRa models for different tasks. By integrating these LoRa models with the general proxy model and using the integrated task proxy model for model inference, prediction performance is improved.

[0265] This application provides two different types of vertical domains: independent vertical domains and shared vertical domains. Independent vertical domains pre-set LoRa data corresponding to a single task, while shared vertical domains pre-set LoRa data corresponding to multiple tasks.

[0266] Independent vertical domain configuration of LoRa data refers to configuring the LoRa data of a certain service on a GPU card.

[0267] In this context, "multiple LoRa data configured on a single GPU card" means that the LoRa data corresponding to multiple services are configured together on a single GPU card.

[0268] In this embodiment of the application, after the user's instruction is identified, tasks such as ordering takeout, sending files, or querying renewal can be identified. For different tasks, different vertical domain configurations of LoRa data can be integrated with a general proxy model. Then, the integrated task proxy model is used for inference to obtain DSL data for different tasks.

[0269] For services with fewer slots, latency pressure is lower. Therefore, LOA data related to these services can be configured in the shared SIM card vertical domain. For example, services with fewer slots could include renewal queries and file transfers. In the shared SIM card vertical domain, LOA data related to file transfers (referred to as file transfer LOA) and LOA data related to renewal queries (referred to as renewal query LOA) can be configured.

[0270] For services with a large number of slots, latency pressure is significant. Therefore, LoRa data related to these services with many slots can be configured separately in an independent vertical domain. For example, services with many slots could be food delivery services. LoRa data related to food delivery (referred to as food delivery LoRa) can be configured in the shared vertical domain.

[0271] It should be noted that the above examples of services with fewer or more slots are illustrative, as are the examples of configuring which LoRa data in independent vertical domains or shared SIM vertical domains. In actual implementation, the specific LoRa data to be configured in independent vertical domains or shared SIM vertical domains can be determined according to actual needs.

[0272] This application's embodiments construct a task agent model framework based on vertical domain distribution, and design a strategy for inference by integrating LoRa based on a shared vertical domain with a general agent model, as well as a strategy for inference by integrating LoRa based on an independent vertical domain with a general agent model.

[0273] Figure 8B A schematic diagram illustrating the internal implementation flow of the task proxy model provided in an embodiment of this application is shown.

[0274] refer to Figure 8B As shown, after the user's instructions are input into the task agent model, the intent is first identified based on the user's instructions to obtain intent information (such as ordering takeout, sending files, or checking renewals).

[0275] Then, the data is distributed to a vertical domain based on the intent information. For example, if the intent information is to order takeout, it is distributed to a separate vertical domain that has pre-defined takeout LoRa data. If the intent information is to query renewal, it is distributed to a shared card vertical domain that has pre-defined multiple LoRa data, such as file sending LoRa data and renewal query LoRa data.

[0276] After the vertical domains are distributed, LoRa data from either independent or shared vertical domains is integrated with the general agent model. The LoRa data from both independent and shared vertical domains uses the same task agent model foundation.

[0277] For example, if the intent information is to order takeout, and it is distributed to an independent vertical domain, then the takeout order LoRa data preset by the independent vertical domain is integrated with the general proxy model.

[0278] For example, if the intent is to query renewal, the pre-set renewal LoRa data from the shared card's vertical domain is integrated with the general proxy model. Similarly, if the intent is to send a file, the pre-set file sending LoRa data from the shared card's vertical domain is integrated with the general proxy model.

[0279] In some cases, after the data is distributed to the shared card vertical domain according to the intent information, if the currently configured LoRa data of the shared card vertical domain is inconsistent with the intent information, the currently configured LoRa data of the shared card vertical domain will be switched to the LoRa data corresponding to the intent information.

[0280] For example, if the intent is to query renewal, but the currently configured LoRa data for the shared SIM card vertical domain is for sending files, then it can be determined that the currently configured LoRa data for the shared SIM card vertical domain is inconsistent with the intent. Accordingly, the currently configured LoRa data for the shared SIM card vertical domain can be switched to the query renewal LoRa data corresponding to the intent. After switching to the query renewal LoRa data, it is integrated with the general proxy model.

[0281] Then, model inference is performed using the integrated task agent model, and the inferred DSL data is output.

[0282] In this embodiment, LoRa data for different tasks are selected based on the characteristics of the vertical domain and integrated with a general proxy model. Model inference is then performed using the integrated task proxy model, which maximizes the saving of latency and resources.

[0283] The LoRa data is generated by first using a large model to produce a batch of corpora based on business understanding, and then manually annotated. The LoRa data for each business can be a list, where each element is a sample (input; output).

[0284] The following is an example of LoRa data for a food delivery business.

[0285] [{"input":"<|im_start|>You are an agent with task planning capabilities, able to transform user input into a DSL for tool orchestration.\n1. The DSL requires a list of tasks. Example format: [['1','Tool Name 1 (Parameter 1 = Parameter Value 1, Parameter 2 = Parameter Value 2,...)'],['2','Tool Name 2 (Parameter 1 = Parameter Value 1,...)']]. The first item of each task is the task ID, sequentially encoded starting from 1; the second item is the tool name, tool parameters, and corresponding values; Parameter 1 and Parameter 2 are the parameters required by the tool. If a parameter does not have a specific value, that parameter and its value are not output; 2. Task orchestration must be based solely on the given candidate tools.\nNote: Only the DSL is output. Formatted results, no need to output analysis process or other content. Candidate Tool: 1. Ordering takeout. Tool Input Parameters: app (application name, e.g., Meituan, Mini Program, optional), shop (merchant brand name, e.g., Luckin Coffee, optional), product (product, e.g., Americano, Latte, optional), no (quantity, must be mapped to a number, e.g., one cup is 1, optional), size (portion, e.g., extra-large, optional), temperature (temperature, optional), sweetness (sweetness, optional), location (destination, optional). Output Parameters: Empty. User Input: Order me a Starbucks latte, extra-large with ice. DSL: <|im_end|>",

[0286] "output":"[['1','order(shop=Starbucks,product=Latte,no=1,size=Kreta,temperature=Ice)']]"},

[0287] {"input":"<|im_start|>You are an agent with task planning capabilities, able to transform user input into a DSL for tool orchestration.\n1. The DSL requires a list of tasks. Example format: [['1','Tool Name 1 (Parameter 1 = Parameter Value 1, Parameter 2 = Parameter Value 2,...)'],['2','Tool Name 2 (Parameter 1 = Parameter Value 1,...)']]. The first item of each task is the task ID, sequentially encoded starting from 1; the second item is the tool name, tool parameters, and corresponding values; Parameter 1 and Parameter 2 are the parameters required by the tool. If a parameter does not have a specific value, that parameter and its value will not be output. 2. Task orchestration must be based solely on the given candidate tools.\nNote: Only the DSL format is output.} The result does not require outputting the analysis process or other content. Candidate Tools: 1. Ordering Takeout Tool Input Parameters: app (application name, e.g., Meituan, Mini Program, optional), shop (merchant brand name, e.g., Luckin Coffee, optional), product (product, e.g., Americano, Latte, optional), no (quantity, must be mapped to a number, e.g., one cup is 1, optional), size (portion, e.g., extra-large, optional), temperature (temperature, optional), sweetness (sweetness, optional), location (destination, optional) Output Parameters: Empty User Input: Order a medium-sized red bean milk tea and a large-sized red bean milk tea on Meituan. DSL: <|im_end|>",

[0288] "output":"[['1','order(app=Meituan,product=Red Bean Milk Tea,no=1,size=Medium)'],['2','order(app=Meituan,product=Red Bean Milk Tea,no=1,size=Large)']]"}......].

[0289] Inference is performed using a large language model (LLM) combined with a hint caching strategy.

[0290] In the model inference stage using the integrated task agent model, this application employs a large language model (LLM) combined with a prompt cache strategy. The LLM is a deep learning model trained on massive amounts of text data, capable not only of generating natural language text but also of deeply understanding text meaning and handling various natural language tasks such as text summarization, question answering, and translation.

[0291] In this embodiment of the application, a hint caching strategy is designed based on the task proxy model framework. Based on the hint caching strategy, hints can be added to the input of the LLM model to help the model better understand and generate text.

[0292] The following details the process of model inference using the hint caching strategy in this application.

[0293] In this embodiment, the prompt vector may include a system prompt module (system_prompt), a tool prompt module (tool_prompt), and a user command module (user_query). The system prompt module (system_prompt) provides a fixed task orchestration format for task orchestration. The tool prompt module (tool_prompt) provides a candidate toolset and input / output parameters, which can be modified or replaced according to specific tools. The user command module (user_query) contains the user's dialogue.

[0294] The system prompt module `system_prompt` is a proxy module with task planning capabilities, which can transform user input into a DSL (Specific Language List) result for tool orchestration. The DSL requires a list of tasks. An example of the overall format is: `[['1','Tool Name 1 (Parameter 1 = Parameter Value 1, Parameter 2 = Parameter Value 2,...)'],['2','Tool Name 2 (Parameter 1 = Parameter Value 1,...)']]`. The first item of each task is the task ID, sequentially encoded starting from 1; the second item is the tool name, tool parameters, and their corresponding values; `Parameter 1` and `Parameter 2` are the parameters required by the tool. If a parameter does not have a specific value, that parameter and its value are not output; task orchestration is based only on the given candidate tools. Note: Only the DSL format result needs to be output; the analysis process and other content do not need to be output.

[0295] The `tool_prompt` module allows modification of the tool description and its input / output parameter descriptions. For example, a candidate toolset includes ordering food delivery. Tool input parameter: `app` (application name, e.g., Meituan). TM Eleme TM Mini-program (optional), shop (merchant or brand name, such as Luckin Coffee) TM ,Starbucks TM (Optional), product (e.g., Americano, Latte, Iced Tea, optional), no (quantity, needs to be mapped to Arabic numerals, e.g., one cup is 1, optional), size (e.g., extra-large, large, optional), temperature (e.g., iced, hot, optional), sweetness (e.g., standard sweet, less sweet, optional), location (delivery destination, e.g., home, office, optional). Output parameter: empty.

[0296] The user instruction module `user_query` can include various user input examples; for example, a user input example could be "Order me a Starbucks". TM "A latte, extra voluminous with ice."

[0297] The aforementioned cue vectors can be used in the model inference stage, enabling model inference based on intent information and cue vectors.

[0298] In this embodiment of the application, in actual use, after receiving a user instruction, the electronic device can provide the user instruction as an input parameter to the task agent model. The task agent model performs intent recognition on the user instruction to obtain intent information. Based on the intent information, it determines the vertical domain corresponding to the intent information and integrates the LoRa data configured in the vertical domain with the general agent model. The integrated task agent model performs inference. During the inference stage, the aforementioned prompt vector is used to fill the slots in the DSL task list based on the intent information to obtain the task arrangement result in DSL format.

[0299] DSL design optimization

[0300] This application presents an embodiment of a low-latency DSL language that meets execution requirements. The following describes the requirements for the DSL data format in the task agent model.

[0301] In some embodiments, DSL data can be a task list. This task list may include one or more tasks. The task list has a preset task description syntax format.

[0302] For example, the task list includes two parts: task ID and task details. The first item in the task list is the task identifier (taskID), which is sequentially encoded starting from 1. The second item in the list is the task details, which includes the tool name, tool parameters, and parameter values.

[0303] For example, the task description syntax format for DSL data can be:

[0304] [[′1′,′toolname1(parameter1=parametervalue1,parameter2=parametervalue2,…)′],

[0305] [′2′,′toolname2(parameter1=parametervalue1, parameter2=parametervalue2, ...)′],

[0306] […],].

[0307] The DSL data above exemplifies two task identifiers: taskID=1 and taskID=2. This indicates that there are two tasks. In actual implementation, the task list may include one task or more than one task.

[0308] In the aforementioned DSL data, the tool name refers to the tool to be invoked to complete the task corresponding to the intent information. For example, if the tool name is "order," the "order" tool will be invoked when the task of ordering takeout is completed. Similarly, if the tool name is "create_reminder," the "create_reminder" tool will be invoked when the task of creating a reminder is completed. Likewise, if the tool name is "navigate," the "navigate" tool will be invoked when the task of navigating is completed. Finally, if the tool name is "take_taxi," the "take_taxi" tool will be invoked when the task of hailing a taxi is completed.

[0309] In the DSL data described above, parameters 1 and 2 are the parameters that the tool needs to invoke. Each parameter corresponds to a slot, which is used to fill the parameter value. The parameter value 1 can be filled into the slot for parameter 1 based on user instructions. Similarly, the parameter value 2 can be filled into the slot for parameter 2 based on user instructions.

[0310] It should be noted that in DSL data, if specific information for a parameter cannot be found based on the user command, then that parameter and its value will not be output. For example, xx tool (parameter1 = , parameter2 = ). If the user command includes information for parameter1 but not for parameter2, then the DSL data can be represented as: xx tool (parameter1 = parameter value1).

[0311] For example, for a takeout order, the parameters of the order tool can include the app that supports takeout services, the shop, the product, the quantity, the cup size, the temperature, the sweetness, and the address.

[0312] DSL data can be [[1, order(shop=C store, product=latte, no=1, size=large, temperature=ice)]].

[0313] For example, when creating a reminder task, the parameters for the create_reminder tool can include content and time.

[0314] DSL data can be [[1, create_reminder(content=Meeting Reminder, time=14 o'clock)]].

[0315] For example, for a ride-hailing task, the parameters of the take_taxi tool can include the origin deplocation, the destination deslocation, and the app that supports ride-hailing services.

[0316] DSL data can be [[1, take_taxi(deslocation=xx building)]].

[0317] For example, for navigation tasks, the parameters of the navigate tool can include the origin deplocation, destination deslocation, pathpoint, traffic type, and the app that supports navigation services.

[0318] DSL data can be [[1, navigate(deslocation = xx building, trafficType = cycling)]].

[0319] In some embodiments, when there are more than two tasks in the task list, the output of the preceding task can also be used as the tool parameter value for the subsequent task.

[0320] For example, phoneNumber = result(taskID = 1).number.

[0321] In this embodiment of the application, the electronic device can perform task orchestration to obtain task orchestration results in the form of DSL data.

[0322] It should be noted that when orchestrating tasks, they must be based on the given candidate tools and cannot exceed the scope of the candidate toolset. For example, the candidate toolset may include checking renewal fees, canceling fee collection, hailing a ride, revoking app permissions, disabling app notifications, optimizing phone performance, and ordering coffee, etc.

[0323] It should be noted that the task proxy model only needs to output DSL data, and does not need to output the analysis process or other content.

[0324] The following explanation uses the check_renewal method for renewal as an example.

[0325] The tool is named `check_renewal`, and its input parameters include `server`, `project`, and `app`. The output parameters are empty. `server` is the name of the service provider, such as the name of a shopping app or video app offering renewal services. `project` is the renewal item; besides automatic renewal / automatic deduction, specific renewal items need to be defined, such as VIP membership / super membership / regular membership, etc. `app` is an app platform with payment functionality, capable of processing payments or deductions for the renewal services provided by the service provider.

[0326] After inputting the user instruction "Help me cancel the VIP membership of xx video" into the task agent model, the task agent model can output DSL data:

[0327] [['1', 'check_renewal(server=First Video App, project=VIP membership, app=App1 and App2)']].

[0328] Optimization of model output

[0329] This application can output DSL in a fixed format.

[0330] In some embodiments, through the task agent model, the following three basic task elements can be output: taskID (task identifier, such as 1, 2, 3 arranged in sequence), tool (tool name), params (tool parameters).

[0331] In some other embodiments, for complex tasks, the task agent model can also output the following two key elements: result (ID = XXX) and condition field. result (ID = XXX) indicates that the parameter of the slot depends on the value of a certain XXX task. The condition field indicates that the task will be executed only when a certain condition is met.

[0332] Exemplarily, the user input user_query can be "Help me order a large iced latte from C store and a large hot milk tea from H store". After receiving this user input, the task agent model can output, through model inference:

[0333] [['1', 'order(app=None, shop=C store, product=latte, no=1, size=large, temperature=iced, sweetness=None, desLocation=None)'],

[0334] [['2', 'order(app=None, shop=H store, product=milk tea, no=1, size=large, temperature=hot, sweetness=None, desLocation=None)']].

[0335] In some embodiments, the task agent model can only output the parameters with slot values, which can reduce the latency. For example, after receiving the above user input, the task agent model can output, through model inference:

[0336] [['1', 'order(shop = C store, product = latte, no = 1, size = large cup, temperature = iced)'],

[0337] [['2', 'order(shop = H store, product = milk tea, no = 1, size = large cup, temperature = hot)']].

[0338] The task agent model only outputs parameters with slot values, which can reduce latency.

[0339] Second embodiment: Document-specific task processing method

[0340] The embodiments of the present application provide a document - specific task processing method, taking the document interaction window as the user interaction entry to provide users with document services with more functions. When the electronic device receives a task instruction input by the user in the document interaction window, the electronic device implements the document - specific task processing method according to the document task instruction input by the user.

[0341] Figure 9 The flow diagram of the document - specific task processing method provided by the embodiments of the present application is shown. As Figure 9 shown, in response to the user operation, the system assistant calls out the document interaction window. The system assistant can interact with the user through the document interaction window. The user can input document - related questions in the document interaction window. After the system assistant receives the user instruction, when the user instruction meets the conditions, the system assistant automatically executes the task and returns the automatic execution result. When the user instruction does not meet the conditions, the system assistant will return that the task is not supported for processing.

[0342] Figure 10 The interface diagram of the document - specific task processing method provided by the embodiments of the present application is shown. As Figure 10 shown in (a) to (f) in the figure, after the system assistant of the electronic device is awakened, the electronic device responds to the user operation and displays the options of the assistant services most recently used by the user, such as the life service assistant, the sports and health assistant, the news and information assistant, the document agent, etc. Among them, the document agent can provide various services related to documents. After the user selects the "document agent" option, the electronic device displays the document interaction window.

[0343] The "upload document" option is included in the document interaction window. Exemplarily, in response to the user's operation of uploading a document, the electronic device displays the content of Document 1.

[0344] The document processing options are also included in the document interaction window. Exemplarily, the document processing options include intelligent summary, opinion summary, intelligent polishing, intelligent translation, work summary, etc.

[0345] The document interaction window also includes a command input box. For example, when the electronic device receives a document command input by the user, it automatically executes the document processing task according to the command. Furthermore, the electronic device displays the user command in the document interaction window and responds to the user command through a system assistant.

[0346] For example, if the user's instruction is "Help me translate document 1", the system assistant will reply "Okay, the document has been translated for you, and the translation result is as follows: xxx".

[0347] Figure 11 This is another schematic diagram of the document-specific task processing method provided in the embodiments of this application. For example... Figure 11 As shown in (a) to (f), the electronic device receives a user's click on the document application icon, launches the document application, and displays a new document page. The new document page includes an instruction input box. When the electronic device receives a document instruction from the user, it can generate the corresponding document based on that instruction. For example, after the user inputs the instruction "Artificial Intelligence Development Report," the electronic device generates and displays the "Artificial Intelligence Development Report" document.

[0348] After the electronic device displays the document content, it activates the document intelligence function and displays a document interaction window. This window includes a command input box and document processing options. For example, these options include intelligent summarization, intelligent polishing, and intelligent translation. Users can enter document commands in the input box or select a command from the processing options to trigger the electronic device to process the currently displayed document accordingly. The electronic device can automatically execute document processing tasks based on the user's input or selection of commands. Furthermore, the device displays the user's commands in the interaction window and responds to them via a system assistant. For example, if the user command is "intelligent polishing," the system assistant might reply, "Okay, intelligent polishing of the document, result as follows: xxx."

[0349] In addition to the document-specific task processing method described above, this application also provides a document task proxy model. When an electronic device implements the document-specific task processing method described above, the electronic device can use the document task proxy model to process document tasks.

[0350] Among them, the document task proxy model is a proxy model used to process document tasks. It can perform inference on the task proxy model to obtain task orchestration results in DSL format.

[0351] Figure 12 This is a schematic diagram illustrating the application process of a document task proxy model provided in an embodiment of this application. For example... Figure 12As shown, when an electronic device displays a document interaction window, if it receives a user instruction about a document task, the electronic device inputs the user instruction into the document agent model, performs reasoning through the document agent model, and obtains a "document" task arrangement result in DSL format. Then, the DSL execution engine of the electronic device can generate a task tree (task operation sequence) and execute the document task based on the "document" task arrangement result in DSL format.

[0352] For example, if the user instruction is "Please help me translate this document", the document proxy model will be used to infer the DSL format "document" task arrangement result, which can be represented as follows:

[0353] [['1','doc_translate(part=None,target_language=None)']].

[0354] It should be noted that the document interaction window in the second embodiment is an interaction entry point specific to the document task, supporting interaction regarding the document task. (See again...) Figure 12 After the user inputs a command into the document task proxy model, the document task proxy model will perform a document vertical domain detection, that is, identify whether the task indicated by the user command is a document task.

[0355] On the one hand, if the task indicated by the user instruction is a document task, that is, if the condition is met, then the task orchestration result in DSL format is output through the document task proxy model, and then the document task is executed based on the task orchestration result in DSL format.

[0356] On the other hand, if the task indicated by the user instruction is not a document task, i.e. the condition is not met, then the response will not support processing the task, or will not respond to the user instruction, or will process the task but will not execute the task.

[0357] In practical use, possible scenarios where users input commands in the document interaction window include:

[0358] Scenario 1: When an electronic device displays a document interaction window, the user enters a document command to trigger the electronic device to execute a document task. For example, "Please help me translate this document" is a document command, and the task it indicates is to translate the document.

[0359] In this case, the document task agent model supports processing document tasks and responding with the results of document task processing.

[0360] Scenario 2: When an electronic device displays a document interaction window, the user enters a non-document command to trigger the electronic device to execute a non-document task. For example, "Please send the document to Zhang San" is a non-document command, and the task it indicates is sending a file.

[0361] In this situation, the document task proxy model does not support handling non-document tasks, and the response does not support handling non-document tasks.

[0362] Scenario 3: When an electronic device displays a document interaction window, the user's input includes both document-related and non-document-related instructions to trigger the electronic device to execute multiple tasks. For example, the user's instruction is "Please help me translate this document and then send it to Zhang San." This instruction includes the document instruction "translate the document" and the non-document instruction "send the document to Zhang San," indicating tasks including translating the document and sending the file. The document translation task is within the document's vertical domain, while the file sending task is not.

[0363] In this situation, the document task agent model cannot recognize such complex user instructions, so it will classify them as non-document tasks and reply that it does not support processing non-document tasks.

[0364] Third embodiment: A document-specific task processing method that supports out-of-domain detection

[0365] Compared to the second embodiment's solution for supporting document processing tasks, the third embodiment of this application makes corresponding improvements. The third embodiment of this application provides a document-specific task processing method. After the electronic device receives instructions input by the user in the document interaction window, the electronic device can recognize and respond to document instructions, non-document instructions, and complex instructions that include both document-related and document-unrelated information, and respond accordingly.

[0366] In the third embodiment, the document-specific task processing method not only supports detection within the document vertical domain but also detection outside the document vertical domain. This allows for the identification of document tasks and non-document tasks based on user instructions. Furthermore, it supports task decomposition, employing different processing flows to complete document tasks and non-document tasks separately, thereby improving the compatibility of task processing and increasing the success rate of automatic task execution.

[0367] Figure 13 This is a schematic diagram of the interface improved from the second embodiment in the third embodiment. Figure 13 (a) and (b) in the figure are schematic diagrams of the interface corresponding to the second embodiment. Figure 13 (c) in the figure is a schematic diagram of the interface corresponding to the third embodiment.

[0368] like Figure 13 As shown in (a), this situation presents the following problem: for instructions containing tasks outside the vertical domain, the device replies that all tasks are executable, but in reality, the tasks are not executed. The electronic device does not respond to user instructions, which affects the user experience.

[0369] For example, a user enters the command "Send this document to Zhang San" in the document interaction window; here, the user command refers to the "Send File" task, which is an external task for the document vertical domain. The device replies "Okay, sending it to you."

[0370] For example, a user enters the command "Translate this document and send it to Zhang San" in the document interaction window. Here, the user command specifies two tasks: "Translate document" and "Send file." "Translate document" is a document task, while "Send file" is a non-document task. However, within the document vertical domain, "Send file" is an external task. The device replies, "Okay, sending it to you."

[0371] like Figure 13 As shown in (b), this situation presents the following problem: for instructions containing tasks outside the vertical domain, the device responds that it cannot execute any of them, affecting the user experience.

[0372] For example, when a user enters the command "Send this document to Zhang San" in the document interaction window, the device replies "Sorry, unable to complete".

[0373] For example, when a user enters the command "Help me translate this document and then send it to Zhang San" in the document interaction window, the device replies "Sorry, unable to complete".

[0374] like Figure 13 As shown in (c), in the improved scheme, the electronic device can respond to and complete the task with fine precision for instructions containing tasks outside the vertical domain.

[0375] For example, when a user enters the command "Send me the document to Zhang San" in the document interaction window, the device replies "Sorry, the document cannot be sent directly. You can jump to the global interaction window to operate" and displays a "Jump" control.

[0376] For example, when a user enters the command "Translate this document and send it to Zhang San" in the document interaction window, the device replies "Okay, the document has been translated for you. The translation result is as follows: xxx. At the same time, it will automatically jump to the global interaction window to complete the document sending."

[0377] The third embodiment of this application has improvements in both the task processing method and the document proxy model.

[0378] First, let me explain the improvement in the task processing method. Before the improvement, for a separately set up document interaction window, it could only recognize the document commands entered by the user and process document tasks; however, it could not automatically execute tasks when it received complex commands entered by the user.

[0379] With the improvements, the task processing method provided in this application embodiment can identify and decompose complex tasks when a user input complex instructions are received in a separately established document interaction window, obtaining DSL data for document tasks and DSL data for non-document tasks. Then, the document tasks and non-document tasks are executed according to different task processing methods.

[0380] For document tasks, the document task can be executed based on the document task's DSL data.

[0381] For non-document tasks, you can jump to the global interactive window and then process the non-document tasks based on the task processing method provided in the first embodiment. For example, first, perform intent recognition based on the DSL data of the non-document task, then distribute the vertical domain, determine the task agent model corresponding to the non-document task, and perform inference based on the task processing model to obtain the task orchestration result in DSL format; then, the non-document task can be automatically executed based on the task orchestration result in DSL format.

[0382] Document tasks are also known as executable tasks, while non-document tasks are also known as out-of-domain tasks.

[0383] The embodiments of this application can accurately identify whether a task is a document task, a non-document task, or a complex task based on user instructions. Complex tasks include one or more document tasks and one or more non-document tasks.

[0384] Figure 14 This is a schematic diagram illustrating task identification for various user commands.

[0385] like Figure 14 As shown in (a), the user instruction is "Help me summarize the document," and through task recognition, one document task, "Summarize the document," can be identified. The user instruction is "Help me summarize the document and translate it," and through task recognition, two document tasks, "Summarize the document and translate it," can be identified.

[0386] like Figure 14 As shown in (b), the user instruction is "Send the document to Zhang San," and through task identification, a non-document task "Send document" can be identified. The user instruction is "Buy me a plane ticket," and through task identification, a non-document task "Buy plane ticket" can be identified.

[0387] like Figure 14 As shown in (c), the user instruction is "Send the document to Zhang San for me." Through task identification, one document task "Translate the document" and one non-document task "Send the document" can be identified. The user instruction is "Buy me a plane ticket." Through task identification, two document tasks "Summarize the document and translate the document" and one non-document task "Send the document" can be identified.

[0388] The task processing method provided in this application embodiment can identify and decompose complex tasks when a user inputs complex instructions in a separately established document interaction window. It can also distribute the decomposed tasks to their respective task proxy models and complete the tasks using different task processing flows.

[0389] The following section describes the improvements to the document proxy model. Before the improvements, the document proxy model supported in-of-distribution (IOD) detection. It could generate task orchestration results in DSL format, but was limited to orchestration results for "document" tasks in DSL format.

[0390] The improved document proxy model supports both in-of-distribution (IOD) and out-of-distribution (OOD) detection. Based on user instructions, the model can perform both IOD and OOD detection, generating DSL data in a specific format. This DSL data can include DSL data for document tasks, DSL data for non-document tasks, or both.

[0391] The DSL data provided in this application embodiment includes the following four elements in a specific form:

[0392] 1) taskID: Represents the task ID, one ID corresponds to one task; for example, 1, 2, 3... etc. can be used to represent the execution order of multiple tasks;

[0393] 2) tool: Indicates the name of the tool;

[0394] 3) params: Represents tool parameters;

[0395] 4) show: Indicates whether to show to the user. The possible values ​​are yes or no, or True or False. True means show, and False means do not show.

[0396] In some embodiments, the four elements described above can be used to arrange tasks in a list format. For example, the task arrangement result in DSL format is [taskID,tool(params1=XXX,params2=XXX),show]. Another example is: [[1,tool(params1=XXX,params2=XXX),show],[2,tool(params1=XXX,params2=XXX),show],…].

[0397] Here, `params1` and `params2` are the parameters required by the tool. `XXX` represents the corresponding value of the parameter. If a parameter does not have a specific value, then the parameter and its value will not be output.

[0398] In some embodiments, for document tasks that are supported for execution, show = Yes can be set. For non-document tasks that are not supported for execution, they can be kept hidden from the user, so show = No can be set.

[0399] For example, if the user instruction is "Please help me translate this document", the document agent model will perform inference and obtain the document task arrangement result in DSL format as: ['1', 'doc_translate(part=None, target_language=None)', Yes]. Here, the task ID is 1, the tool is the document translation tool doc_translate, the parameter params includes the part and the target language target_language, and show is "Yes".

[0400] In other embodiments, the show field can be set based on semantic understanding of whether the user wants to see a certain arrangement result.

[0401] It should be noted that the necessity of the show field lies in the fact that, for the orchestration of complex tasks, some intermediate processes are orchestrated to complete the final task, and are not the results that the user actually wants. Therefore, it is necessary to distinguish between the two.

[0402] In this application embodiment, for complex tasks, DSL data in a specific form also includes the following two key elements:

[0403] 1) result(ID=XXX) is a slot dependency element, representing the task result whose parameter dependency identifier is XXX.

[0404] For example, if the user instruction is "Please help me translate the main idea of ​​this document", the document proxy model will perform inference, and the resulting DSL-formatted "document" task orchestration can be represented as follows:

[0405] [['1','doc_summary(part=None,limit=None)'],

[0406] ['2','doc_translate(part=result(ID1),summary_result,target_language=None)']].

[0407] In this case, the content to be translated in the translation task in Identifier 2 depends on the results of the document summary task in Identifier 1.

[0408] 2) `not_supported(query=XXX)` indicates an unsupported element. The tool uses `not_supported` to represent this, and `query` only displays the specific command that is not supported for execution. For example, the user command "buy a plane ticket" is an unsupported command and can be represented as: `not_supported(query=buy a plane ticket)`.

[0409] Based on the DSL data design principles provided in the embodiments of this application, some task examples are given below.

[0410] Task Example 1: Help me translate the main idea of ​​the document and buy a plane ticket.

[0411] The corresponding DSL data can be represented as:

[0412] [['1','doc_summary(part=None,limit=None)',False],

[0413] ['2','doc_translate(part=result(ID1).summary_result,target_language=None)',True],

[0414] ['3','not_supported(query=buy a plane ticket)',False]]

[0415] The doc_summary task is an executable intermediate task. Setting it to "False" will prevent the task results from being displayed to the user.

[0416] The doc_translate task is an executable critical task, and setting it to "True" will display the task results to the user.

[0417] Among them, "Buy a plane ticket" is a task that does not support not_supported. You can set it to "False" so that the task result is not displayed to the user.

[0418] Example Task 2: What is the main idea of ​​the article? Please translate the main idea for me, and then buy a plane ticket.

[0419] The corresponding DSL data can be represented as:

[0420] [['1','doc_summary(part=None,limit=None)',True],

[0421] ['2','doc_translate(part=result(ID1).summary_result,target_language=None)',True],

[0422] ['3','not_supported(query=buy a plane ticket)',False]]

[0423] The tasks of summarizing the document doc_summary and translating the document doc_translate are both executable critical tasks. Setting "True" will allow the results of both tasks to be displayed to the user.

[0424] Among them, "Buy a plane ticket" is a task that does not support not_supported. You can set it to "False" so that the task result is not displayed to the user.

[0425] Task Example 3: Help me summarize the third paragraph of the document, then buy a plane ticket, and oh right, translate the summary results.

[0426] The corresponding DSL data can be represented as:

[0427] [['1','doc_summary(part=Third paragraph,limit=None)',True],

[0428] ['2','not_supported(query=buy a plane ticket)',False],

[0429] ['3','doc_translate(part=result(ID1).summary_result,target_language=None)',True]]

[0430] It should be noted that the order in which each task appears in the DSL data can be displayed according to user instructions.

[0431] The following is a flowchart illustrating the task processing method provided in the third embodiment of this application, with reference to the accompanying drawings.

[0432] like Figure 15 As shown, when a user command is received in the document interaction window, the user command is input into the document proxy model. For non-document tasks, in-domain and out-of-domain detection is performed, and the task orchestration results in DSL format are output.

[0433] Scenario 1: Document Task

[0434] If the document agent model identifies a document task based on user instructions, then the document agent model generates a “document” task orchestration result in DSL format.

[0435] For example, if the user instruction is "Please help me translate this document", the document proxy model will be used to infer the DSL format "document" task arrangement result, which can be represented as follows:

[0436] [['1','doc_translate(part=None,target_language=None)']].

[0437] For example, if the user instruction is "Please help me translate the main idea of ​​the document", the document proxy model will perform inference to obtain the DSL format "document" task orchestration result:

[0438] [['1','doc_summary(part=None,limit=None)'],

[0439] ['2','doc_translate(part=result(ID1),summary_result,target_language=None)']].

[0440] In Scenario 1, after the document proxy model outputs the DSL data corresponding to the document task, the task tree can be generated and the document task can be executed through the DSL execution engine.

[0441] Scenario 2: Non-document tasks

[0442] If the document proxy model identifies a non-document task (outside the vertical domain) based on user instructions, then the document proxy model generates DSL data corresponding to the non-document task.

[0443] For example, if the user instruction is "Please help me buy a plane ticket", the inference process through the document proxy model will yield non-document data in DSL format: [['1', 'not_supported(query=buy a plane ticket)']].

[0444] In Scenario 2, after the document proxy model outputs the DSL data corresponding to the non-document task, it jumps to the global interactive window and then automatically executes the non-document task according to the task processing method provided in the first embodiment above. For example, it identifies the intent, distributes the vertical domain, the proxy model infers and generates DSL data, and parses the DSL and executes the task.

[0445] Scenario 3: Complex Tasks

[0446] If the document proxy model identifies a complex task (including document tasks and non-document tasks) based on user instructions, it generates a task orchestration result in DSL format. This DSL format task orchestration result includes DSL data corresponding to document tasks and DSL data corresponding to non-document tasks.

[0447] For example, if the user instruction is "Please help me translate the main idea of ​​this document and buy me a plane ticket," the task orchestration result in DSL format obtained through inference using the document agent model can be represented as follows:

[0448] [['1','doc_summary(part=None,limit=None)'],

[0449] ['2','doc_translate(part=result(ID1),summary_result,target_language=None)'],

[0450] ['3', 'not_supported(query=buy a plane ticket)']].

[0451] The DSL data corresponding to the document task includes: ['1', 'doc_summary(part=None,limit=None)'], ['2', 'doc_translate(part=result(ID1),summary_result,target_language=None)'].

[0452] The DSL data for non-document tasks includes: ['3', 'not_supported(query=buy a plane ticket)'].

[0453] In Scenario 3, the document proxy model outputs DSL data corresponding to document tasks and DSL data corresponding to non-document tasks. On one hand, for document tasks, the DSL execution engine parses the DSL data corresponding to the document task, generates a task tree, and executes the document task. On the other hand, for non-document tasks, it jumps to the global interactive window and then automatically executes the non-document task according to the task processing method provided in the first embodiment above. For example, based on the DSL data corresponding to non-document tasks, it identifies the intent, distributes the vertical domain, infers and generates DSL data using the proxy model, and parses the DSL and executes the task.

[0454] The following flowcharts illustrate the possible implementation methods for the three scenarios mentioned above.

[0455] Figure 16 This is a flowchart illustrating the application of the task processing method provided in this embodiment of the application to document tasks. Figure 16 As shown, in the document interaction window, the electronic device receives the user's instruction "Help me translate the document," inputs the user's instruction into the document proxy model, and the document proxy model infers the "document" task arrangement result in DSL format: [['1', 'doc_translate(part=None,target_language=None)']]. Then, the electronic device can generate a task tree and execute the document task through the DSL execution engine.

[0456] Figure 17 This is a flowchart illustrating the task processing method provided in this application for non-document tasks. Figure 17 As shown, in the document interaction window, the electronic device receives the user instruction "Buy me a plane ticket", inputs the user instruction into the document proxy model, and the document proxy model infers the DSL data: [['1', 'not_supported(query=Buy a plane ticket)']].

[0457] In this scenario, the electronic device can automatically jump from the document interaction window to the global interaction window. Based on [['1', 'not_supported(query=buy a plane ticket)']], the intent is identified as "buy a plane ticket". Then, the vertical domain is distributed, identified as the "flight vertical domain". Next, the agent model is identified, identified as the flight agent model. The DSL data for the ticket purchase task is input through the flight agent model: [['1', 'open_app(app=xxx)'], ['2', 'buy_ticket(from=xxx,to=xxx)']]. The DSL data for the ticket purchase task is then parsed, without vertical domain detection. Finally, the DSL execution engine generates a task tree based on the parsing results and executes the ticket purchase task.

[0458] Figure 18 This is a flowchart illustrating the application of the task processing method provided in this embodiment to a complex task. Figure 18 As shown, in the document interaction window, the electronic device receives the user instruction "Please help me translate the main idea of ​​the document and buy a plane ticket". The user instruction is input into the document proxy model, and the document proxy model infers the DSL data: [['1', 'doc_summary(part=None,limit=None)'], ['2', 'doc_translate(part=result(ID1),summary_result,target_language=None)'], ['3', 'not_supported(query=buy a plane ticket)']].

[0459] Then, the electronic device decomposes the DSL data based on the DSL data to obtain DSL data for non-document tasks and DSL data for document tasks.

[0460] The DSL data for the document task is [['1', 'doc_summary(part=None,limit=None)'], ['2', 'doc_translate(part=result(ID1),summary_result,target_language=None)']].

[0461] The DSL data for non-document tasks is [['3', 'not_supported(query=buy a plane ticket)']].

[0462] For document tasks, electronic devices can generate a task tree and execute document tasks through the DSL execution engine.

[0463] For non-document tasks, the electronic device can automatically jump from the document interaction window to the global interaction window. Based on [['1', 'not_supported(query=buy a plane ticket)']], the intent is identified as "buy a plane ticket". Then, a vertical domain is distributed, identified as the "flight vertical domain". Next, the agent model is identified, identified as the flight agent model. The DSL data for the ticket purchase task is input through the flight agent model: [['1', 'open_app(app=xxx)'], ['2', 'buy_ticket(from=xxx,to=xxx)']]. The DSL data for the ticket purchase task is then parsed, without vertical domain detection. Finally, the DSL execution engine generates a task tree based on the parsing results and executes the ticket purchase task.

[0464] For user instructions in various possible scenarios, this application provides the following three possible implementation methods:

[0465] Scenario 1: Identify tasks within a document's vertical domain.

[0466] If an electronic device recognizes a user instruction indicating a document task, the electronic device can directly process that document task.

[0467] Scenario 2: Identify document tasks and non-document tasks (i.e., tasks outside the document vertical domain).

[0468] Method 1: If the electronic device recognizes that the task indicated by the user command includes both document tasks and non-document tasks, the electronic device can directly process the document task and prompt that the current task includes non-document tasks. After the document task is completed, it will automatically jump to the global interactive window and automatically execute the non-document task.

[0469] Method 2: If the electronic device recognizes that the task indicated by the user's command includes both document tasks and non-document tasks, the electronic device can directly process the document task and prompt that the current task includes a non-document task, and prompt that the non-document task can be executed after jumping to the global interaction window; after the electronic device receives the user's confirmation operation, the electronic device jumps from the document interaction window to the global interaction window and automatically executes the non-document task.

[0470] Scenario 3: Identify tasks within a document's vertical domain.

[0471] Method 1: If the electronic device recognizes that the user command indicates a non-document task, the electronic device can prompt that the current task is a non-document task, and then jump to the global interactive window to automatically execute the non-document task.

[0472] Method 2: If the electronic device recognizes that the user command indicates a non-document task, the electronic device can prompt that the current task is a non-document task and suggest that the non-document task can be executed after jumping to the global interaction window; after the electronic device receives the user's confirmation operation, the electronic device jumps from the document interaction window to the global interaction window and automatically executes the non-document task.

[0473] Figure 19 A schematic diagram of the interactive interface of the task processing method provided in the embodiments of this application. Figure 1 .like Figure 19 As shown, the electronic device displays a document interaction window and receives the user's instruction "Translate document 1 for me". The electronic device replies "Okay, the document has been translated for you. The translation result is as follows: xxx".

[0474] Figure 20 A schematic diagram of the interactive interface of the task processing method provided in the embodiments of this application. Figure 2 .like Figure 20 As shown in (a), the electronic device displays a document interaction window and receives the user's instruction "Buy me a plane ticket." The electronic device replies, "Okay, I have automatically redirected to the global interaction window to purchase the ticket for you." Figure 20 As shown in (b), the electronic device jumps from the document interaction window to the global interaction window, and the electronic device replies "Okay, I will redirect you to the flight service page to purchase your ticket."

[0475] Figure 21 A schematic diagram of the interactive interface of the task processing method provided in the embodiments of this application. Figure 3 .like Figure 21 As shown in (a), the electronic device displays a document interaction window and receives the user's instruction, "Translate document 1 for me and then send it to Zhang San." The electronic device replies, "Okay, the document has been translated for you. The translation result is as follows: xxx," along with "Please complete the file sharing in the global interaction window" and a "Confirm" control. After the user clicks the "Confirm" control, as shown... Figure 21 As shown in (b), the electronic device jumps from the document interaction window to the global interaction window, and displays the user command "Send the translated document to Zhang San" and replies "Okay, will redirect to the file management page to send you the document".

[0476] With the solution proposed in this application, when a user enters a first instruction in the document interaction window and the first instruction contains a document task, the electronic device can recognize the user's intent based on the user's instruction, determine the document task, and automatically execute the document task, helping the user save time and effort.

[0477] With the solution proposed in this application, when a user inputs a user command from a document-specific entry point that contains a task unrelated to the document, the electronic device can recognize the user's intent and break down the task to obtain DSL data for the document task and DSL data for the non-document task. Then, it can automatically execute the document task based on the DSL data for the document task and automatically execute the non-document task based on the DSL data for the non-document task, helping users save time and effort.

[0478] This solution enables the automatic execution of a series of preset tasks based on user intent. Furthermore, it can handle complex operation sequences, ensuring the automated execution process is completed smoothly according to user needs, thus saving users significant time and effort.

[0479] It should be noted that in the embodiments of this application, "greater than" can be replaced with "greater than or equal to", "less than or equal to" can be replaced with "less than", or "greater than or equal to" can be replaced with "greater than", and "less than" can be replaced with "less than or equal to".

[0480] The various embodiments described herein can be independent solutions or combinations thereof based on their inherent logic, and all such solutions fall within the protection scope of this application.

[0481] The foregoing mainly describes the solutions provided by the embodiments of this application from the perspective of method steps. It is understood that, in order to achieve the above functions, the electronic device implementing this method includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of protection of this application.

[0482] This application embodiment can divide an electronic device into functional modules based on the above method example. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, other feasible division methods may exist. The following description uses the division of functional modules according to each function as an example.

[0483] This application also provides a chip coupled to a memory, which is used to read and execute computer programs or instructions stored in the memory to perform the methods in the above embodiments.

[0484] This application also provides an electronic device including a chip for reading and executing computer programs or instructions stored in a memory, causing the methods in the various embodiments to be performed.

[0485] This embodiment also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the task processing method described in the above embodiment.

[0486] This embodiment also provides a computer program product. The computer-readable storage medium stores program code. When the computer program product is run on a computer, it causes the computer to perform the above-mentioned related steps to implement the task processing method in the above embodiment.

[0487] In this embodiment, the electronic device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0488] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0489] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0490] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. In the description of embodiments in this application, unless otherwise stated, "multiple" means two or more; for example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0491] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A task processing method, characterized in that, The method is applied to an electronic device, and the method includes: When a user enters a first command in the global interaction window, and the first command instructs the execution of a first task, the electronic device automatically executes the first task and displays the execution result of the first task in the global interaction window; the first task is a non-document task. When a user enters a second command in the document interaction window, and the second command instructs the execution of a document task, the electronic device automatically executes the document task and displays the execution result of the document task in the document interaction window; When a user enters a third instruction in the document interaction window, and the third instruction instructs the execution of the document task and the first task, the electronic device automatically executes the document task and displays the execution result of the document task in the document interaction window. The electronic device also displays a first jump prompt message, which indicates that the first task will be automatically executed after jumping to the global interaction window.

2. The method according to claim 1, characterized in that, The method further includes: When the user enters the first instruction in the document interaction window, the electronic device displays the first jump prompt information in the document interaction window.

3. The method according to claim 1 or 2, characterized in that, The method further includes: When the user inputs the third instruction in the document interaction window, the electronic device calls the document agent model to perform inference based on the third instruction and obtains a multi-task operation sequence in DSL format; wherein, the multi-task operation sequence includes a document task operation sequence and a non-document task operation sequence. The document proxy model performs inference based on the third instruction, including in-vertical detection and out-of-vertical detection.

4. The method according to claim 3, characterized in that, After obtaining the multi-task operation sequence in DSL format, the method further includes: The multi-task operation sequence in the DSL format is split into task sequences to obtain the document task operation sequence and the non-document task operation sequence. The non-document task operation sequence includes a first parameter description, which indicates that the first task does not support execution.

5. The method according to claim 3 or 4, characterized in that, The automatic execution of the document task includes: automatically executing the document task based on the document task operation sequence; The display of the first jump prompt information includes: displaying the first jump prompt information based on the non-document task operation sequence.

6. The method according to any one of claims 3 to 5, characterized in that, The document task operation sequence includes a first task identifier, a first tool name, a first tool parameter, and first configuration information; The first configuration information is used to configure whether the execution result of the document task is displayed to the user. Each task parameter corresponds to a slot, and the slot is used to fill the slot value.

7. The method according to claim 6, characterized in that, The document task operation sequence also includes third configuration information for multi-document task scenarios, which indicates that the slot value of one task depends on the result of another task.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: In response to the user's first operation, the electronic device launches the document intelligence application and displays the document interaction window; and detects whether the user has entered a command in the document interaction window through the document intelligence application. The document agent application supports calling the document proxy model.

9. The method according to any one of claims 3 to 8, characterized in that, The non-document task operation sequence includes a second task identifier, a second tool name, second tool parameters, and second configuration information; The second tool parameter slot is set to the first task, the second tool name is used to indicate that the first task does not support execution, and the second configuration information is used to configure that the execution result of the first task is not displayed to the user.

10. The method according to any one of claims 1 to 9, characterized in that, After the electronic device displays the first redirect prompt message, the method further includes: Jump from the document interaction window to the global interaction window; A fourth instruction is generated based on a non-document task operation sequence, the fourth instruction instructing the execution of the first task; The fourth instruction is displayed in the global interactive window; In response to the fourth instruction, the first task is executed automatically; The execution result of the first task is displayed in the global interactive window.

11. The method according to claim 10, characterized in that, The automatic execution of the first task in response to the fourth instruction includes: In response to the fourth instruction, the system assistant application of the electronic device is activated; The system assistant application performs intent recognition based on the fourth instruction to determine the first task; The first task agent model is invoked for inference to obtain the first task operation sequence in DSL format. Based on the first task operation sequence, the first task is executed automatically.

12. The method according to claim 11, characterized in that, The step of invoking the first task agent model for inference to obtain the first task operation sequence in DSL format includes: The first task proxy model is obtained by integrating the LoRa data corresponding to the first task with the general proxy model. Based on the first task agent model, the first task operation sequence in the DSL format is obtained by reasoning based on the prompt cache information corresponding to the first task and the fourth instruction.

13. The method according to claim 12, characterized in that, The step of reasoning based on the first task agent model, according to the prompt cache information corresponding to the first task and the fourth instruction, to obtain the first task operation sequence in DSL format includes: Obtain the prompt cache information corresponding to the first task. The prompt cache information is in DSL format and includes a task identifier, a tool name, and multiple tool parameters, with each tool parameter corresponding to a slot. Input the cached information and the fourth instruction into the first task agent model; The first task agent model fills the slots in the prompt cache information according to the fourth instruction, and generates the first task operation sequence in the DSL format.

14. The method according to claim 10, characterized in that, The step of jumping from the document interaction window to the global interaction window includes: Automatically jump from the document interaction window to the global interaction window; or, In response to the user's confirmation, the user is redirected from the document interaction window to the global interaction window.

15. The method according to any one of claims 1 to 14, characterized in that, The first task includes any of the following: querying automatic renewal, sending files, disabling application permissions, optimizing overall device performance, disabling application notifications, and ordering takeout.

16. An electronic device, characterized in that, The electronic device includes: one or more processors, and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 15.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 15.

18. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by an electronic device, causes the electronic device to perform the method as described in any one of claims 1 to 15.

Images