Task orchestration execution method and apparatus for human-machine interaction engine, and device

Through the human-computer interaction engine, the artificial intelligence model and engine module are used, combined with user requests and system status information, the operating system's insufficient support for AI application tasks is solved, and intelligent task orchestration and execution are realized.

WO2025156901A1PCT designated stage expired Publication Date: 2025-07-31MATTER INNOVATION PTE LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2024/141534
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2024-01-23
Filing Date
2024-12-23
Publication Date
2025-07-31

AI Technical Summary

Technical Problem

The existing operating systems lack global perception of system status information and analysis of users' common operation behaviors, resulting in the inability to effectively support task execution of AI applications.

Method used

Through the human-computer interaction engine, artificial intelligence big models and engine modules are used to combine user requests, system status information and user operation behaviors to actively sense user intentions and perform tasks orchestration and execution.

Benefits of technology

It realizes intelligent automated task arrangement and execution of AI applications, and improves the intelligence and efficiency of the operating system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024141534_31072025_PF_FP_ABST
    Figure CN2024141534_31072025_PF_FP_ABST
Patent Text Reader

Abstract

A task orchestration execution method and apparatus for a human-machine interaction engine, and a device, relating to the technical field of computer task orchestration. The task orchestration execution method for a human-machine interaction engine comprises: receiving a request of a user (S110); acquiring state information of a system (S120); performing analysis processing on previous operation behaviors of the user, so as to obtain a common operation behavior of the user (S130); according to the request of the user, the state information of the system and the common operation behavior of the user, determining an intention of the user (S140); according to the intention of the user, performing an orchestration processing on at least one task required to be executed, so as to obtain the order of execution of each task (S150); and, according to the order of execution, successively executing the corresponding tasks (S170).
Need to check novelty before this filing date? Find Prior Art

Description

Task scheduling execution method, device and equipment for human-computer interaction engine CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application refers to Chinese patent application No. 202410097725.0, entitled “Task orchestration execution method, device and equipment for human-computer interaction engine”, filed on January 23, 2024, which is incorporated into this application in its entirety by reference. Technical Field

[0002] The present disclosure relates to the technical field of computer task scheduling, and in particular to a task scheduling execution method, apparatus, device, computer-readable storage medium, and computer program product for a human-computer interaction engine. Background Art

[0003] With the increasing use of AI (artificial intelligence) applications such as large models, current traditional touch screen operations on mobile phones / tablets can no longer serve the future AI automation task processing well.

[0004] Therefore, how to more intelligently integrate the automated task orchestration and execution of AI applications is one of the core issues that need to be addressed in the design and development of future AI operating systems (OS). Summary of the Invention

[0005] The present disclosure provides a task scheduling and execution method for a human-computer interaction engine.

[0006] According to one aspect of the present disclosure, a method for scheduling and executing tasks in a human-computer interaction engine is provided, comprising: receiving a user request; obtaining system status information; analyzing and processing the user's previous operating behaviors to obtain the user's commonly used operating behaviors; determining the user's intention based on the user's request, the system status information, and the user's commonly used operating behaviors; scheduling and processing at least one task to be executed based on the user's intention to obtain an execution order for each task; and executing the corresponding tasks in sequence according to the execution order.

[0007] According to another aspect of the present disclosure, a task scheduling and execution device of a human-computer interaction engine is provided, comprising: a first receiving module for receiving a user's request; an acquisition module for acquiring system status information; an analysis module for analyzing and processing the user's previous operation behaviors to obtain the user's commonly used operation behaviors; an artificial intelligence large model module for determining the user's intention based on the user's request, the system's status information and the user's commonly used operation behaviors; an artificial intelligence engine module for scheduling and processing at least one task to be executed according to the user's intention to obtain the execution order of each task; and an execution module for executing corresponding tasks in sequence according to the execution order.

[0008] According to yet another aspect of the present disclosure, a task orchestration execution device of a human-computer interaction engine is provided. The task orchestration execution device is connected to at least one processor and at least one memory, and a computer program is stored on the at least one memory. When the computer program is executed by the at least one processor, the at least one processor executes a task orchestration execution method of the human-computer interaction engine.

[0009] According to yet another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the processor executes a task scheduling execution method of a human-computer interaction engine.

[0010] According to yet another aspect of the present disclosure, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the processor executes a task scheduling execution method of a human-computer interaction engine.

[0011] According to one or more embodiments of the present disclosure, it is possible to actively perceive the system's status information and the user's common operating behaviors, and by analyzing the user's intentions, orchestrate and execute at least one task to be performed, thereby accelerating AI automated calling of the software interface (UI).

[0012] These and other aspects of the disclosure will be apparent from and elucidated with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar components or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to the present disclosure and should not be regarded as limiting the scope of the present disclosure.

[0014] FIG1 is a flowchart illustrating a task orchestration execution method of a human-computer interaction engine according to an exemplary embodiment;

[0015] FIG2 is a flowchart illustrating a portion of an exemplary process in the method of FIG1 according to an exemplary embodiment;

[0016] FIG3 is a schematic block diagram illustrating a task scheduling execution device of a human-computer interaction engine according to an exemplary embodiment; and

[0017] FIG. 4 is a block diagram illustrating an exemplary computer device that can be used with the exemplary embodiments. DETAILED DESCRIPTION

[0018] In this disclosure, unless otherwise specified, the use of terms such as "first" and "second" to describe various elements is not intended to limit the positional relationship, temporal relationship, or importance relationship of these elements. Such terms are only used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of the element, while in some cases, based on the context of the description, they may also refer to different instances.

[0019] The terms used in the description of the various examples described in this disclosure are for the purpose of describing specific examples only and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. As used herein, the term "plurality" means two or more, and the term "based on" should be interpreted as "based at least in part on". In addition, the terms "and / or" and "at least one of..." cover any one of the listed items and all possible combinations.

[0020] It should be understood that the terms "system," "device," "unit," and / or "module" used herein are a method for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, if other terms can achieve the same purpose, the terms may be replaced by other expressions.

[0021] Flowcharts are used in this disclosure to illustrate the operations performed by systems according to embodiments of the present disclosure. It should be understood that the preceding or following operations do not necessarily need to be performed in exact order. Instead, the steps may be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0022] When the current operating system executes complex AI tasks, the local human-computer interaction engine, after receiving the processing results from the cloud, will use the corresponding software interface operation tools of each platform. For example, the Android operating system can generally use UI automator to complete the operation of the application software interface, thereby realizing the execution of complex AI tasks.

[0023] However, the APP (application) control and system control in existing operating systems lack the ability to globally perceive the system's status information and analyze users' common operating behaviors. In addition, the control capabilities for traditional applications are single and are not specifically designed to support the execution of AI application tasks.

[0024] In order to enable more intelligent access to automated task orchestration and execution of AI applications, the present disclosure provides a task orchestration and execution method for a human-computer interaction engine.

[0025] Exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0026] FIG. 1 is a flowchart illustrating a task orchestration execution method 100 of a human-computer interaction engine according to an exemplary embodiment.

[0027] The task orchestration execution method 100 of the human-computer interaction engine provided in this embodiment can be executed by a task orchestration execution device of the human-computer interaction engine. The task orchestration execution device of the human-computer interaction engine can be implemented in software and / or hardware. The task orchestration execution device of the human-computer interaction engine can be composed of two or more physical entities, or a single physical entity. Generally speaking, the task orchestration execution device of the human-computer interaction engine can be a terminal device, such as a mobile phone, tablet, etc.

[0028] The following description will be made by taking a computer device as an example to execute the task arrangement execution method 100 of the human-computer interaction engine.

[0029] 1 , the method 100 includes steps S110 to S170 .

[0030] In step S110, the user's request is received. As an example, the user's request can be received through automatic speech recognition (ASR), that is, the vocabulary content in the user's voice is converted into computer-readable input. For another example, the user's request can also be received by the user manually typing text, which is not limited by the present disclosure. In some embodiments, the request may include asking the human-computer interaction engine to send WeChat, schedule a meeting, send an email, etc. For example, the user can say to the human-computer interaction engine, "Please tell my mother that I will not be home for dinner tonight." For another example, the user can say to the human-computer interaction engine, "Please book a meeting with Mr. Li for me on Wednesday afternoon."

[0031] In step S120, the system status information is obtained. In some embodiments, the system status information may include at least one of the focus, position, or size of each active user interface of the application. In some embodiments, the system status information may include the battery status or the current mode. For example, after obtaining the current system status information of "low battery state and do not disturb mode", when the user says to the human-computer interaction engine "please tell my mother that I will not be home for dinner tonight", the human-computer interaction engine is more inclined to choose a method of sending WeChat or text messages that saves power and does not disturb the user, rather than a method of making a phone call that consumes power and disturbs the user.

[0032] In step S130, the user's previous operation behaviors are analyzed and processed to obtain the user's frequently used operation behaviors. In some embodiments, the user's frequently used operation behaviors include at least one of the following: applications installed by the user, applications opened by the user, frequently used applications, and frequently communicated contacts. For example, in the case described above where the human-computer interaction engine tends to send WeChat or send text messages, if the human-computer interaction engine determines that the user's frequently used operation behavior is to use WeChat instead of text messages, then the human-computer interaction engine is more inclined to select the more frequently used application, i.e., to send WeChat.

[0033] In step S140, the user's intention is determined based on the user's request, the system's status information, and the user's common operating behaviors. In some embodiments, the human-computer interaction engine includes an artificial intelligence big model, and the determination of the user's intention is achieved through the artificial intelligence big model. According to some embodiments, the artificial intelligence big model can be deployed in the cloud. In some embodiments, the selection of a specific large-scale pre-trained model or a basic model for the artificial intelligence big model can be determined based on actual application requirements or experiments. For example, the artificial intelligence big model can determine that the user's intention is to use WeChat text to send a message to the mother to tell her that she will not be home for dinner in the evening based on the user's request after automatic voice recognition of "help me tell my mother that I will not be home for dinner in the evening", the system's "low power state and do not disturb mode" status information, and the fact that the user's common way of communication with his mother is WeChat text.

[0034] In step S150, according to the user's intention, at least one task to be executed is arranged and processed to obtain the execution order of each task. For example, in the case where it is determined that the user's intention is to use WeChat text to send a message to her mother to tell her that she will not be home for dinner in the evening, it is necessary to arrange and process multiple tasks of opening and editing WeChat to obtain the execution order of each task such as opening WeChat and the chat window of the user's mother, inputting messages, and sending information. In some embodiments, the human-computer interaction engine includes an artificial intelligence engine, and the arrangement and processing of at least one task to be executed to obtain the execution order of each task is achieved by the artificial intelligence engine. In some embodiments, the artificial intelligence engine, such as a neural network-like engine, can realize automatic speech recognition and transmit the recognition results to the artificial intelligence large model. The artificial intelligence engine can execute specific functions or applications based on the analysis results of the artificial intelligence large model.

[0035] In step S170 , the corresponding tasks are executed in sequence according to the execution order.

[0036] Method 100 has the ability to actively perceive the system status information and the user's common operating behaviors, and determines the user's intention through the user's request, the system status information and the user's common operating behaviors. Based on the intention, at least one task to be executed is orchestrated, which is conducive to more intelligent access to the automated task orchestration and execution of AI applications.

[0037] FIG2 is a flowchart illustrating a portion of an example process in method 100 of FIG1 according to an exemplary embodiment. As shown in FIG3, according to some embodiments, the system state information may include view tree information for each active user interface of an application. The view tree information includes the components of the active user interface and the hierarchical structure of different components. Step S170, executing the corresponding tasks in sequence according to the execution order, may include:

[0038] Step S210: adding a mark to the specific view tree information of the application to be executed for subsequent identification; and

[0039] Step S220: Based on the execution order and the mark, locate the components of the active user interface required for the operation and execute the corresponding task.

[0040] According to some embodiments, the component of the active user interface includes at least one of a text box, an edit view, a button, and an image view.

[0041] According to some embodiments, step S210, adding a mark to the specific view tree information of the application to be executed may include: adding metadata (meta) information to the specific view tree information of the application to be executed. In some embodiments, the metadata information includes at least one of a data element, a data attribute, a data structure, or related data. For example, when the execution order of each task such as opening WeChat and the chat window of the user's mother, inputting a message, and sending a message is obtained, the metadata information of the components of the active user interface such as WeChat, the chat window of the user's mother, the message input, and the message sending button can be marked. Therefore, in step S220, there is no need to search for the components of each active user interface one by one in the traditional way, but based on the execution order and the metadata information of the components of each active user interface in the window where the WeChat contact's mother is located in step S210, the components of the active user interface for the required operation can be quickly located and the corresponding text message can be sent.

[0042] Referring again to FIG. 1 , alternatively or additionally, the method 100 may further include step S160 .

[0043] In step S160, a user confirmation operation is received before the corresponding tasks are executed in sequence according to the execution order in step S170. However, it is understood that step S160 is not required. The human-computer interaction engine can also determine the user's default operation based on the user's repeated operation habits and directly execute the corresponding task in step S170.

[0044] FIG3 is a schematic block diagram illustrating a task scheduling execution device 300 of a human-computer interaction engine according to an exemplary embodiment.

[0045] As shown in FIG3 , the task scheduling execution device 300 of the human-computer interaction engine includes: a first receiving module 310 , an acquisition module 320 , an analysis module 330 , an artificial intelligence large model module 340 , an artificial intelligence engine module 350 and an execution module 370 .

[0046] The first receiving module 310 is configured to receive a user's request.

[0047] The acquisition module 320 is configured to acquire system status information.

[0048] The analysis module 330 is configured to analyze and process the user's previous operation behaviors to obtain the user's frequently used operation behaviors.

[0049] The artificial intelligence big model module 340 is configured to determine the user's intention based on the user's request, the system's status information, and the user's common operating behaviors.

[0050] The artificial intelligence engine module 350 is configured to arrange at least one task to be executed according to the user's intention and obtain the execution order of each task.

[0051] The execution module 370 is configured to execute the corresponding tasks in sequence according to the execution order.

[0052] Referring again to FIG. 3 , alternatively or additionally, the apparatus 300 may further include a second receiving module 360 ​​.

[0053] The second receiving module 360 ​​is configured to receive the user's confirmation operation before executing the corresponding tasks in sequence according to the execution order.

[0054] It should be understood that the various modules of the apparatus 300 shown in FIG3 may correspond to steps S110-S170 of the method 100 described with reference to FIG1 . Thus, the operations, features, and advantages described above for the method 100 are also applicable to the apparatus 300 and the modules included therein. For the sake of brevity, certain operations, features, and advantages are not described in detail herein.

[0055] While specific functionality has been discussed above with reference to specific modules, it should be noted that the functionality of the various modules discussed herein may be separated into multiple modules, and / or at least some functionality of multiple modules may be combined into a single module. A specific module as discussed herein performing an action includes the specific module itself performing the action, or alternatively, the specific module calling or otherwise accessing another component or module that performs the action (or performs the action in conjunction with the specific module). Thus, a specific module that performs an action may include the specific module itself that performs the action and / or another module that the specific module calls or otherwise accesses to perform the action.

[0056] It should also be understood that various technologies can be described herein in the general context of software hardware elements or program modules. The various modules described above with respect to FIG3 can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these modules can be implemented as computer program code / instructions, which are configured to be executed in one or more processors and stored in a computer-readable storage medium. Alternatively, these modules can be implemented as hardware logic / circuits. For example, in some embodiments, one or more of the first receiving module 310, the acquisition module 320, the analysis module 330, the artificial intelligence large model module 340, the artificial intelligence engine module 350, the second receiving module 360, and the execution module 370 can be implemented together in a system on chip (SoC). The SoC can include an integrated circuit chip (which includes a processor (e.g., a central processing unit (CPU), a microcontroller, a microprocessor, a digital signal processor (DSP), etc.), a memory, one or more communication interfaces, and / or one or more components in other circuits), and can optionally execute the received program code and / or include embedded firmware to perform functions.

[0057] According to one aspect of the present disclosure, a computer device is provided, comprising at least one memory, at least one processor, and a computer program stored in the at least one memory. The at least one processor is configured to execute the computer program to implement the steps of any of the method embodiments described above. A task orchestration execution device of a human-computer interaction engine may include the computer device.

[0058] According to one aspect of the present disclosure, a task orchestration execution device for a human-computer interaction engine is provided. The task orchestration execution device for the human-computer interaction engine is connected to at least one processor and at least one memory, wherein the at least one memory stores a computer program. The at least one processor is configured to execute the computer program to implement the steps of any of the method embodiments described above.

[0059] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any method embodiment described above are implemented.

[0060] According to one aspect of the present disclosure, a computer program product is provided, which includes a computer program. When the computer program is executed by a processor, the steps of any one of the method embodiments described above are implemented.

[0061] Illustrative examples of such a computer device, computer-readable storage medium, and computer program product are described below in conjunction with FIG. 4 .

[0062] FIG4 illustrates an example configuration of a computer device 400 that can be used to implement the methods described herein. For example, the power management system 100 shown in FIG1 may include an architecture similar to the computer device 400. The apparatus 300 described above may also be implemented in whole or in part by the computer device 400 or a similar device or system.

[0063] Computer device 400 may include at least one processor 402, memory 404, communication interface(s) 406, a display device 408, other input / output (I / O) devices 410, and one or more mass storage devices 412, all capable of communicating with one another, such as via a system bus 414 or other appropriate connections.

[0064] The processor 402 may be a single processing unit or multiple processing units, all of which may include a single or multiple computing units or multiple cores. The processor 402 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operational instructions. Among other capabilities, the processor 402 may be configured to retrieve and execute computer-readable instructions stored in the memory 404, mass storage device 412, or other computer-readable media, such as program code for an operating system 416, program code for application programs 418, program code for other programs 420, and the like.

[0065] Memory 404 and mass storage device 412 are examples of computer-readable storage media for storing instructions that are executed by processor 402 to implement the various functions described above. For example, memory 404 may generally include both volatile memory and non-volatile memory (e.g., RAM, ROM, etc.). In addition, mass storage device 412 may generally include a hard drive, a solid-state drive, removable media, including external and removable drives, memory cards, flash memory, floppy disks, optical disks (e.g., CDs, DVDs), storage arrays, network attached storage, storage area networks, etc. Memory 404 and mass storage device 412 may be collectively referred to herein as memory or computer-readable storage media, and may be non-transitory media capable of storing computer-readable, processor-executable program instructions as computer program code that may be executed by processor 402 as a specific machine configured to implement the operations and functions described in the examples herein.

[0066] A number of programs may be stored on the mass storage device 412. These programs include an operating system 416, one or more application programs 418, other programs 420, and program data 422, and they may be loaded into the memory 404 for execution. Examples of such applications or program modules may include, for example, computer program logic (e.g., computer program code or instructions) for implementing the following components / functionality: the method 200 (including any suitable steps of the method 200), and / or other embodiments described herein.

[0067] 4 as being stored in memory 404 of computer device 400, modules 310, 320, 330, 340, 350, 360, and 370, or portions thereof, may be implemented using any form of computer-readable media accessible by computer device 400. As used herein, "computer-readable media" includes at least two types of computer-readable media, namely, computer-readable storage media and communication media.

[0068] Computer-readable storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information, such as computer-readable instructions, data structures, program modules or other data. Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD), or other optical storage devices, magnetic cassettes, magnetic tape, magnetic disk storage devices or other magnetic storage devices, or any other non-transmission media that can be used to store information for access by a computer device. In contrast, communication media can embody computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism. Computer-readable storage media as defined herein does not include communication media.

[0069] One or more communication interfaces 406 are used to exchange data with other devices, such as via a network, direct connection, and the like. Such communication interfaces can be one or more of the following: any type of network interface (e.g., a network interface card (NIC)), a wired or wireless (such as an IEEE 802.11 wireless LAN (WLAN)) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth™ interface, a Near Field Communication (NFC) interface, and the like. The communication interface 406 can facilitate communication within a variety of network and protocol types, including wired networks (e.g., LAN, cable, and the like) and wireless networks (e.g., WLAN, cellular, satellite, and the like), the Internet, and the like. The communication interface 406 can also provide communication with external storage devices (not shown), such as storage arrays, network attached storage, storage area networks, and the like.

[0070] In some examples, a display device 408 such as a monitor may be included for displaying information and images to the user. Other I / O devices 410 may be devices that receive various inputs from the user and provide various outputs to the user, and may include a touch input device, a gesture input device, a camera, a keyboard, a remote control, a mouse, a printer, an audio input / output device, and the like.

[0071] The technology described herein can be supported by these various configurations of the computer device 400 and is not limited to the specific examples of the technology described herein. For example, the functionality can also be implemented in whole or in part on a "cloud" by using a distributed system. The cloud includes and / or represents a platform for resources. The platform abstracts the underlying functionality of the hardware (e.g., servers) and software resources of the cloud. Resources can include applications and / or data that can be used when performing computing processing on a server away from the computer device 400. Resources can also include services provided over the Internet and / or through a subscriber network such as a cellular or Wi-Fi network. The platform can abstract resources and functionality to connect the computer device 400 to other computer devices. Therefore, the implementation of the functionality described herein can be distributed throughout the cloud. For example, functionality can be implemented partially on the computer device 600 and partially through a platform that abstracts the functionality of the cloud.

[0072] Although the present disclosure has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description are to be considered illustrative and exemplary and not restrictive; the disclosure is not limited to the disclosed embodiments. Variations to the disclosed embodiments will be understood and effected by those skilled in the art in practicing the claimed subject matter by studying the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps not listed, and the term "based on" is to be interpreted as "based at least in part on". The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0073] The above descriptions are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for task scheduling and execution of a human-computer interaction engine, the method comprising: Receiving a request from a user; Obtaining the status information of the system; Analyzing and processing the previous operation behaviors of the user to obtain the commonly used operation behaviors of the user; Determining the intention of the user according to the request of the user, the status information of the system, and the commonly used operation behaviors of the user; According to the intention of the user, performing scheduling processing on at least one task to be executed to obtain the execution order of each task; And Sequentially executing the corresponding tasks according to the execution order.

2. The method according to claim 1, wherein, The status information of the system includes view tree information of each active user interface of the application. The view tree information includes components of the active user interface and the hierarchical structure of different components. And sequentially executing the corresponding tasks according to the execution order includes: Attaching a mark to the specific view tree information of the application to be executed for subsequent identification; and Based on the execution order and the mark, locating the components of the active user interface that need to be operated and performing the corresponding tasks.

3. The method according to claim 2, wherein, Attaching a mark to the specific view tree information of the application to be executed includes: Attaching metadata information to the specific view tree information of the application to be executed.

4. The method according to claim 3, wherein The metadata information includes at least one of data elements, data attributes, data structures, or related data.

5. The method according to claim 2, wherein The components of the active user interface include at least one of a text box, an edit view, a button, and an image view.

6. The method according to claim 1, wherein The human-computer interaction engine includes an artificial intelligence large model, and the determination of the intention of the user is implemented by the artificial intelligence large model.

7. The method according to claim 1, wherein, The human-computer interaction engine includes an artificial intelligence engine, and performing scheduling processing on at least one task to be executed to obtain the execution order of each task is implemented by the artificial intelligence engine.

8. The method according to any one of claims 1 to 7, wherein, The commonly used operation behaviors of the user include at least one of the applications installed by the user, the opened applications, the frequently used applications, and the frequently communicated contacts.

9. The method according to any one of claims 1 to 7, wherein The status information of the system includes at least one of the focus, position, or size of each active user interface of the application.

10. The method according to any one of claims 1 to 7, further comprising: Receiving a confirmation operation from the user before sequentially executing the corresponding tasks according to the execution order.

11. A task scheduling and execution device of a human-computer interaction engine, comprising: A first receiving module for receiving a request from a user; An obtaining module for obtaining the status information of the system; An analyzing module for analyzing and processing the previous operation behaviors of the user to obtain the commonly used operation behaviors of the user; An artificial intelligence large model module for determining the intention of the user according to the request of the user, the status information of the system, and the commonly used operation behaviors of the user; An artificial intelligence engine module for performing scheduling processing on at least one task to be executed according to the intention of the user to obtain the execution order of each task; And An execution module for sequentially executing the corresponding tasks according to the execution order.

12. The device according to claim 11, further comprising: A second receiving module, configured to receive a confirmation operation of the user before performing corresponding tasks in sequence according to the execution order.

13. A task scheduling and execution device for a human-computer interaction engine, the task scheduling and execution device being connected to at least one processor and at least one memory, and a computer program being stored on the at least one memory, Among them, when the computer program is executed by the at least one processor, the at least one processor is caused to execute the method according to any one of claims 1 to 10.

14. A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 10.

15. A computer program product, including a computer program, and when the computer program is executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 10.

Citation Information

Patent Citations

  • Context-awareness-based contact individuation recommending server, client and method

    CN103294801A

  • System and method for processing web-browsing information

    CN105453082A

  • User behavior prediction method, terminal, cloud equipment and storage medium

    CN116562926A

  • Artificial intelligence based dynamic orchestration engine for automatically processing application service requests

    US20220012605A1

  • System and method for recommendation of products and services

    US20230043314A1