Electronic device, information processing method, electronic device, and chip

Abstract

This application discloses an electronic device, an information processing method, and a chip, relating to the field of artificial intelligence technology. The specific technical solution is as follows: the electronic device includes a system-level artificial intelligence (AI) engine integrated within the operating system of the electronic device; wherein, the system-level AI engine is used to invoke hardware components and / or system services of the electronic device based on user intent information to execute a target task.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically relating to an electronic device, an information processing method, an electronic device, and a chip. Background Technology

[0002] With the development of artificial intelligence (AI) technology, AI technologies (such as AI assistants) have been widely used in various electronic devices.

[0003] In related technologies, AI components run as applications on the operating system. Taking an AI assistant as an example, its processing flow is as follows: first, it receives user instructions; then, it calls other applications to execute tasks through the application programming interface provided by the operating system; and finally, it renders and displays the execution results on the application interface. However, in the existing architecture, because AI components run at the application layer and are restricted by the application sandbox, all operations must be executed through the application agent, resulting in response delays and consequently poor task processing performance. Summary of the Invention

[0004] The purpose of this application is to provide an electronic device, an information processing method, an electronic device and a chip that can reduce the response latency of task processing, thereby improving task processing efficiency.

[0005] In a first aspect, embodiments of this application provide an electronic device, which includes: a system-level artificial intelligence (AI) engine integrated within the operating system of the electronic device; wherein the system-level AI engine is used to invoke hardware components and / or system services of the electronic device based on user intent information to execute a target task.

[0006] Secondly, embodiments of this application provide an information processing method, which includes: using a system-level AI engine to invoke hardware components and / or system services of the electronic device based on user intent information to execute a target task.

[0007] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions, when executed by the processor, implementing the steps of the method described in the second aspect.

[0008] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the second aspect.

[0009] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the second aspect.

[0010] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the second aspect.

[0011] In this embodiment, the electronic device includes a system-level AI engine integrated within the operating system of the electronic device. This system-level AI engine is used to invoke hardware components and / or system services of the electronic device based on user intent information to execute a target task. By integrating the system-level AI engine into the operating system of the electronic device, the system-level AI engine can directly invoke underlying hardware and system services to execute related tasks, eliminating data transmission and instruction conversion in intermediate stages, thereby reducing task processing response latency and improving task processing efficiency. Attached Figure Description

[0012] Figure 1A One of the schematic diagrams of the AI ​​terminal provided in the embodiments of this application;

[0013] Figure 1B A second schematic diagram of the architecture of the AI ​​terminal provided in the embodiments of this application;

[0014] Figure 1C One of the schematic diagrams of the AI ​​architecture of the electronic device provided in the embodiments of this application;

[0015] Figure 1D A second schematic diagram of the AI ​​architecture of the electronic device provided in the embodiments of this application;

[0016] Figure 1E A schematic diagram of the system hierarchy of the AI ​​engine provided in the embodiments of this application;

[0017] Figure 1F This is a schematic diagram of the AI ​​terminal system framework provided in the embodiments of this application;

[0018] Figure 2 This is one of the structural schematic diagrams of the electronic device provided in the embodiments of this application;

[0019] Figure 3 This is a second schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0020] Figure 4 This is the third schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0021] Figure 5 A timing diagram of shared memory provided for an embodiment of this application;

[0022] Figure 6 This is a schematic diagram of AI communication protocol types provided in the embodiments of this application;

[0023] Figure 7 A schematic diagram of the event notification subscription process provided in this application embodiment;

[0024] Figure 8 A schematic diagram of the streaming data subscription process provided in an embodiment of this application;

[0025] Figure 9 This is a flowchart illustrating the asynchronous call process provided in an embodiment of this application.

[0026] Figure 10 A flowchart illustrating the synchronous invocation process provided in an embodiment of this application;

[0027] Figure 11 One of the flowcharts illustrating the information processing method provided in the embodiments of this application;

[0028] Figure 12 A schematic diagram illustrating the process of AI calling system capabilities as provided in an embodiment of this application;

[0029] Figure 13 A schematic diagram illustrating the AI-controlled UI rendering pipeline process provided in an embodiment of this application;

[0030] Figure 14 A schematic diagram illustrating the direct conversion of AI decisions into rendering instructions, as provided in the embodiments of this application;

[0031] Figure 15 A schematic diagram of a multi-level context memory module provided in an embodiment of this application;

[0032] Figure 16 A schematic diagram of the context retrieval process provided for embodiments of this application;

[0033] Figure 17 A schematic diagram illustrating the priority protection of important rules provided in the embodiments of this application;

[0034] Figure 18 This is a schematic diagram illustrating the construction of a user preference profile as provided in an embodiment of this application.

[0035] Figure 19 This is a schematic diagram illustrating the real-time personalized adjustment of user preference profiles provided in the embodiments of this application;

[0036] Figure 20 A schematic diagram of the differential update rendering process provided in the embodiments of this application;

[0037] Figure 21 This is a schematic diagram of the pre-rendering process provided in an embodiment of this application;

[0038] Figure 22 A schematic diagram illustrating the interaction logic between the user and the AI-generated interface provided in this application embodiment;

[0039] Figure 23 A schematic diagram illustrating the switching logic of the full-screen AI interactive interface provided in the embodiments of this application;

[0040] Figure 24 This is a schematic diagram of the structure of another electronic device provided in an embodiment of this application;

[0041] Figure 25 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0043] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0044] The terms "at least one," "at least one of," etc., used in the specification and claims of this application refer to any one, any two, or a combination of two or more of the included items. For example, at least one of a, b, and c can mean: "a," "b," "c," "a and b," "a and c," "b and c," and "a, b, and c," where a, b, and c can be single or multiple. Similarly, "at least two" refers to two or more items, and its meaning is similar to that of "at least one."

[0045] With the rapid development of artificial intelligence technology, existing electronic devices are equipped with AI applications or AI agents. The relationship between these AI applications or AI agents and the system is one of external overlay, that is, AI capabilities are superimposed on the traditional operating system. In other words, AI applications or AI agents cannot directly access the underlying system (kernel, drivers, hardware abstraction layer, rendering pipeline, etc.) and need to call corresponding capabilities through standard interfaces. Specifically, existing AI applications rely on the pre-designed interface of the applications they call when displaying their interface. They cannot generate customized operation interfaces according to user intentions, nor can they adjust the interface in real time according to changes in user intentions. That is, existing AI applications need to rely on interfaces provided by third-party applications to operate, which increases response latency and leads to poor task processing performance.

[0046] like Figure 1A As shown, the existing AI terminal architecture mainly includes the application layer, AI assistant, operating system, and system capabilities (API), meaning that the relationship between AI and the system is "external superposition" rather than "internal integration".

[0047] like Figure 1B As shown, the existing architecture can be divided into the following layers:

[0048] AI layer: Located at the top of the architecture, responsible for generating content. Figure 1B The main output of this layer is represented by three "Generated Content" modules.

[0049] Application Layer: This layer includes multiple applications such as APP1, APP2, and APP3. These applications initiate requests to the system layer through call actions to obtain the necessary capabilities. The AI ​​Agent runs at the application layer and is responsible for scheduling or executing AI-related task logic.

[0050] System layer: Composed of the operating system, UI framework, and system capability APIs from bottom to top. The operating system provides the underlying infrastructure; the UI framework is responsible for interface presentation and interaction; and the system capability APIs expose various system function interfaces to upper-layer applications for application layer to call.

[0051] The purpose of this application is to provide an electronic device, an information processing method, an electronic device and a chip that incorporates AI as a core component of the operating system, achieving deep integration of AI (i.e., a system-level AI engine) with the system. This allows AI to directly call system capabilities, directly control UI rendering, and generate user interaction interfaces in real time, realizing a new human-computer interaction paradigm. This not only reduces the response latency of task processing and improves task processing efficiency, but also provides customized functions based on user intent and generates corresponding real-time interactive interfaces, thereby improving the flexibility and efficiency of interface display.

[0052] It should be noted that the AI ​​or AI engine in the embodiments of this application refers to the system-level AI engine, and the two can be used interchangeably in description.

[0053] In this embodiment, AI is presented as a system service, with the AI ​​engine deeply integrated into the operating system as a system-level service. Specifically, AI is part of the system, accessing all system capabilities and continuously running on the device side, sensing user status in real time. Secondly, AI can directly call underlying system APIs without going through an intermediate layer, such as directly calling hardware capabilities like cameras, positioning, and sensors, and directly calling system services like payment, communication, and file access, without going through an application sandbox, resulting in lower performance overhead. Thirdly, AI directly controls UI rendering; for example, AI outputs UI descriptions, which the system directly parses and renders, achieving millisecond-level UI response and a full-screen AI interactive interface for an immersive experience. Fourthly, it provides intention-driven proactive services, meaning AI no longer passively responds but actively senses and serves, for example, continuously sensing user status (location, time, scene), proactively identifying intents, predicting needs, and proactively providing services at appropriate times.

[0054] Figure 1C This is a schematic diagram of the AI ​​architecture of the electronic device provided in the embodiments of this application, such as... Figure 1C As shown, the electronic device includes Figure 1C A schematic diagram of the architecture of the electronic device provided in the embodiments of this application, such as... Figure 1C As shown, the electronic device includes: System Layer: This is the core of the architecture, where the "System-level AI Engine," "System Capability Bus," and "UI Rendering Pipeline" are interconnected to form a deeply integrated structure. Specifically: The System-level AI Engine can handle core driving functions such as business logic, data processing, and runtime environment. System Capability Bus: As a functional module embedded within the system layer, it provides AI-related services and capability support. UI Rendering Pipeline: Responsible for graphical interface rendering and display output. Operating System: Located at the bottom layer of the architecture, it provides the basic runtime environment for all upper-layer modules.

[0055] Specifically, in conjunction with the above Figure 1C , Figure 1D Another schematic diagram of the electronic device provided in the embodiments of this application, such as Figure 1DAs shown, the electronic device may include an AI fusion layer, a hardware component layer (i.e., a hardware abstraction layer), and a system service layer. The hardware component layer includes the hardware components of the electronic device, and the system service layer includes the system services of the electronic device. Optionally, the electronic device may also include an application layer. The AI ​​fusion layer includes a system-level AI engine, which directly calls the system capability bus and directly controls the system capability bus. The system capability bus directly accesses the camera, positioning, sensors, etc., and directly accesses the payment, calendar, and notification functions of the system service layer. The UI rendering pipeline directly calls View / UIView for rendering. That is, in this application, AI is no longer "running on top of the system" but "embedded in the system."

[0056] Figure 1E This is a schematic diagram of the system hierarchy of the AI ​​engine provided in the embodiments of this application, such as... Figure 1E As shown, the operating system's underlying architecture includes a system service layer. This layer comprises system services for electronic devices, including various system-level service modules such as telephone services, SMS services, and an AI engine (i.e., a system-level AI engine). These service modules are uniformly managed and scheduled through a system capability bus. This bus serves as a unified entry point for hardware capabilities, system services, and UI rendering, providing standardized calling interfaces for upper-layer applications and enabling centralized control and efficient access to system resources. This application embeds the AI ​​engine as a system-level service into the operating system, rather than as a standalone application. Specifically, the characteristics of AI as a system-level service include: being at the same level as basic services like telephone, SMS, and clock; having system-level permissions; no application sandbox restrictions; access to the underlying system (hardware abstraction layer, kernel interface); direct control of the UI rendering pipeline; and the ability to intercept and respond to system events. This layered architecture places the AI ​​engine and basic communication services at the same system service level, which improves the efficiency of AI functions in calling system capabilities and enhances the overall architecture's scalability and maintainability.

[0057] Figure 1F This is a schematic diagram of the AI ​​terminal system framework provided in the embodiments of this application, such as... Figure 1FAs shown, this framework is built around an AI engine, employing a multi-layered system architecture from top to bottom. The top layer consists of the AI ​​engine (i.e., the system-level AI engine) and its system-level services. It initiates calls to lower layers through a unified communication protocol layer, supporting various interaction methods such as synchronous calls (for low latency), asynchronous calls (for high throughput), streaming data (to meet real-time requirements), and event notifications. Below the communication protocol layer is the capability abstraction layer, which shields the differences between underlying hardware and system services, providing a unified capability interface for upper layers. The capability abstraction layer includes a Hardware Abstraction Layer (HAL), a system service layer, a sensor layer, and a UI rendering layer, respectively implementing unified management and invocation of hardware components, system services, sensor data, and interface views. The bottom layer consists of hardware components and system services, including cameras, gyroscopes, calendars, payment systems, messaging systems, GPS, and ViewUIView. Through layered design and standardized communication mechanisms, the overall architecture efficiently integrates the AI ​​engine with various terminal hardware and software capabilities, providing upper-layer AI applications with flexible, low-latency, and high-throughput system capability invocation. In this application, after the AI ​​engine is embedded in the system layer, the system-level AI engine is directly connected to the system capability bus and the UI rendering pipeline. The system-level AI engine can directly control the UI rendering pipeline, thereby generating images efficiently and in real time without intermediate steps. Specifically, the terminal directly calls the system UI API by controlling the UI rendering pipeline through AI, without relying on the UI components of the APP, and can generate a complete interactive interface in real time, thus achieving significant improvements in response efficiency, interface flexibility, and system integration.

[0058] The AI ​​architecture of the electronic device provided in this application differs from the AI ​​architecture of existing technologies in the following ways: In related technologies, the AI ​​Agent adopts an indirect interaction architecture: after the user input is understood by the AI, the AI ​​only generates results in the form of text or images, and then sends the results to a third-party APP. After receiving the results, the APP parses them and calls its own UI components to complete the final rendering. In this process, the AI ​​is only responsible for content generation; the AI ​​cannot directly control the generation of the screen. Its output depends on the UI components of the APP to be presented to the user, resulting in a long interaction path and the interface form being limited by the APP's predetermined components, making it difficult to achieve flexible real-time interaction.

[0059] In this application, the AI ​​terminal adopts a direct control architecture: after user input is understood by the system-level AI engine, the system-level AI engine directly controls the underlying UI rendering pipeline. By calling system-level View or UIView APIs, the AI ​​can generate a complete interactive interface in real time and render the results directly to the screen. In other words, the system-level AI engine has the ability to directly draw the interface, thereby achieving a low-latency, highly consistent native interactive experience, and thus improving response efficiency, interface flexibility, and system integration.

[0060] The electronic device provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0061] like Figure 2 As shown, an embodiment of the present invention provides an electronic device, which includes a system-level AI engine 100, which is integrated into the operating system of the electronic device.

[0062] In some embodiments of this application, the system-level AI engine 100 is used to invoke hardware components and / or system services of an electronic device to perform a target task based on user intent information.

[0063] In some embodiments of this application, the aforementioned electronic device is a terminal. This terminal integrates an artificial intelligence engine as a system component within the operating system. In this type of terminal, the AI ​​engine no longer runs as an independent application at the application layer, but exists as a basic service parallel to the system kernel and hardware abstraction layer, possessing system-level permissions to directly access underlying hardware resources and system services. The intelligent native terminal uniformly manages the invocation of hardware components and system services through the system capability bus, and the AI ​​engine directly controls the interface generation and rendering of the UI rendering pipeline, thereby achieving a closed-loop process from user intent information to system execution, and from execution results to interface presentation. Compared to existing AI-enabled terminals, the terminal proposed in this application has lower interaction latency, a wider capability boundary, and a more immersive interactive experience that better meets user needs.

[0064] In some embodiments of this application, the aforementioned hardware components may be hardware components of the operating system of an electronic device. For example, the hardware component may include a camera, microphone, sensor, positioning module, etc.

[0065] In some embodiments of this application, the aforementioned system services may be system services running on electronic devices.

[0066] In some embodiments of this application, the system services of the electronic device may include services for calling third-party interfaces, calendar services, payment services, push notification services, and location services. The services for calling third-party interfaces encapsulate the interface addresses and interaction rules required for interacting with external services, providing a standardized entry point for the system-level AI engine to call third-party capabilities; the calendar service manages scheduled events and reminders; the payment service completes transaction processes; the push notification service sends notification messages to users; and the location service obtains the device's current location information.

[0067] It should be noted that the aforementioned hardware components and system services can also be other hardware components or system services, and this application embodiment does not limit them.

[0068] In some embodiments of this application, the system-level AI engine 100 is integrated within the operating system and runs as a system-level service.

[0069] In some embodiments of this application, the system-level AI engine has the authority to invoke hardware components and / or system services of the electronic device.

[0070] In some embodiments of this application, the system-level AI engine 100 receives text input, voice input, touch input or other interactive input from the user, identifies user intent information, and then, based on the user intent information, invokes hardware components and / or system services of the electronic device to execute the target task.

[0071] In some embodiments of this application, the aforementioned target tasks include, but are not limited to: ticket booking, navigation, information query, device control, and content recommendation.

[0072] It should be noted that the objective may also include other tasks, which are not limited in this application.

[0073] For example, the system-level AI engine 100 receives a user's voice command, "Book me a flight from Shanghai to Beijing at 9:00 AM tomorrow." The system-level AI engine 100 parses and determines that the user's intent is "to book a flight." Then, based on this intent, the system-level AI engine 100 calls a flight query service and obtains flight information that meets the time and route requirements through a pre-defined ticketing system interface. After obtaining the flight data, the system-level AI engine 100 generates an interactive interface containing a flight list and selection controls for the user to select and confirm.

[0074] For example, the system-level AI engine 100 receives the text information "Help me set a meeting reminder for tomorrow afternoon at 3 pm, in meeting room A" input by the user. The system-level AI engine 100 determines the user's intent as "create a schedule reminder" through semantic analysis. Based on this intent, the system-level AI engine 100 calls the calendar service interface to complete the schedule creation and reminder settings.

[0075] In this way, the system-level AI engine 100 can determine the user's intent based on the user's input information and call various system services as needed to execute the target task. It can complete the entire task process without the participation of third-party applications, eliminating data transmission and instruction conversion in the intermediate links, thereby reducing the response delay of task processing.

[0076] The electronic device provided in this application includes a system-level AI engine integrated within the operating system of the electronic device. This system-level AI engine is used to invoke hardware components and / or system services of the electronic device based on user intent information to execute a target task. Through this solution, the system-level AI engine can directly invoke underlying hardware and system services, eliminating data transmission and instruction conversion in intermediate stages, thereby reducing task processing response latency and improving task processing efficiency.

[0077] In combination with the above Figure 2 ,like Figure 3 As shown, the electronic device also includes a UI rendering pipeline 200, which is connected to the system-level AI engine 100.

[0078] Specifically, the system-level AI engine 100 is used to receive task information sent by hardware components and / or system services, generate first UI data based on the task information, and transmit the first UI data to the UI rendering pipeline 200; the UI rendering pipeline 200 is used to generate and render the user interface based on the first UI data.

[0079] In some embodiments of this application, the UI rendering pipeline 200 can parse the first UI data and generate and render the user interface.

[0080] In some embodiments of this application, the UI rendering pipeline 200 is coupled to the display hardware (e.g., display screen) of the electronic device and is responsible for the generation and rendering of the screen.

[0081] In some embodiments of this application, the UI rendering pipeline 200 parses the received UI data, synthesizes and renders a dynamic user interface in real time, and outputs it to the display screen for display.

[0082] For example, when performing a target task, the system-level AI engine can invoke the UI rendering pipeline 200 of the electronic device to dynamically generate a real-time user-customized interactive interface (i.e., user interface), and the interface content can be flexibly adjusted in real time according to user needs.

[0083] For example, referring to the above example, the system-level AI engine 100 receives the user's voice command "Book me a flight from Shanghai to Beijing at 9:00 AM tomorrow." The system-level AI engine 100 parses and determines that the user's intent is "book a flight." Then, based on this intent, the system-level AI engine 100 calls the flight query service to obtain flight information that meets the time and route requirements through a pre-defined ticketing system interface. After obtaining the flight data, the system-level AI engine 100 further calls the UI rendering pipeline to generate an interactive interface containing a flight list and selection controls in real time, and outputs it to the display screen for the user to select and confirm. After user confirmation, the system-level AI engine 100 continues to call the ticketing booking interface to complete subsequent steps in the booking process, such as seat selection, passenger information entry, and payment redirection.

[0084] In some embodiments of this application, the system-level AI engine can also obtain task information from the application at the mobile application layer, or obtain task information jointly from the application at the mobile application layer, hardware components, and / or system services, based on the user's intent, generate first UI data, and transmit the first UI data to the UI rendering pipeline; the UI rendering pipeline generates and renders a customized interactive interface that can be flexibly adjusted according to user needs based on the first UI data.

[0085] Thus, since the system-level AI engine can directly call relevant services to execute tasks and directly call the UI rendering pipeline to render task results, the entire processing latency is low. Furthermore, the display interface is generated in real time by the system-level AI engine based on user intent information, ensuring that the displayed interface meets the user's actual needs.

[0086] In combination with the above Figure 3 ,like Figure 4 As shown, the electronic device also includes a system capability bus 300, which is connected to the system-level AI engine 100, the hardware components of the electronic device, and system services.

[0087] The system-level AI engine 100 is used to generate a first instruction based on user intent information and send the first instruction to hardware components and / or system services through the system capability bus 300.

[0088] In some embodiments of this application, the first instruction described above is used to trigger or execute a target task.

[0089] In some embodiments of this application, the system capability bus 300 serves as a unified hardware and system service access channel, providing a standardized calling interface for the upper layer.

[0090] In some embodiments of this application, after the system-level AI engine 100 sends a first instruction to the hardware components and / or system services through the system capability bus 300, it can receive task information returned by the hardware components and / or system services, generate first UI data based on the task information, and transmit the first UI data to the UI rendering pipeline 200.

[0091] For example, the system capability bus 300 communicates with multiple hardware components (such as cameras, microphones, sensors, positioning modules, etc.) and multiple system services (such as payment services, communication services, file management services, etc.) at the operating system level.

[0092] In some embodiments of this application, the system-level AI engine establishes communication connections with the system capability bus 300 and the UI rendering pipeline 200, respectively.

[0093] In some embodiments of this application, the system-level AI engine has the authority to invoke hardware components and / or system services on the system capability bus. For example, the system-level AI engine 100 has the authority to directly invoke hardware components and system services mounted on the system capability bus 300, without going through an application sandbox or third-party applications.

[0094] In some embodiments of this application, the system-level AI engine 100 receives text input, voice input, touch input, or other interactive input from a user, identifies the user's intent information, and then generates first UI data (such as UI description data) and a first instruction (such as a first instruction sequence) corresponding to the intent. Next, the system-level AI engine 100 directly transmits the UI description data to the UI rendering pipeline 200, and directly sends the first instruction sequence to the corresponding hardware components and / or system services for execution via the system capability bus 300.

[0095] In some embodiments of this application, the first UI data mentioned above may include at least one of the following: interface layout information, controls, content, etc.

[0096] It should be noted that the aforementioned first UI data can be determined according to actual task requirements, and this application embodiment does not limit this.

[0097] In some embodiments of this application, the first instruction may include at least one of the following: booking a ticket, activating a camera, obtaining location, performing payment, etc.

[0098] It should be noted that the first instruction mentioned above can be determined according to actual task requirements, and this application embodiment does not limit it.

[0099] In this embodiment, the electronic device includes a system capability bus, a system-level AI engine, and a UI rendering pipeline. The system capability bus is connected to the hardware components and system services of the electronic device; the system-level AI engine is integrated within the operating system of the electronic device and is connected to both the system capability bus and the UI rendering pipeline. The system-level AI engine generates a first instruction based on user intent information and transmits the first instruction to system hardware or services via the system capability bus. It also receives task information from the system hardware or services, generates first UI data accordingly, and transmits the first UI data to the UI rendering pipeline. The UI rendering pipeline parses the first UI data and generates and renders the user interface. Through this approach, the system-level AI engine can directly call the underlying hardware and system services, eliminating data transmission and instruction conversion in intermediate stages, thereby reducing task processing response latency and improving task processing efficiency.

[0100] In some embodiments of this application, the system-level AI engine 100 includes: an intent recognition module, used to receive and analyze user input information to identify user intent information; a decision module, connected to the intent recognition module and the system capability bus respectively, used to generate a first instruction based on the user intent information and send the first instruction to hardware components and / or system services through the system capability bus; the decision module is also used to generate first UI data based on task information returned by hardware components and / or system services, and transmit the first UI data to the UI rendering pipeline.

[0101] For example, the intent recognition module is used to receive and analyze user input, such as performing speech recognition on speech and semantic understanding on text, so as to output structured user intent information.

[0102] For example, the decision-making and planning modules are connected to the intent recognition module and the system capability bus 300, respectively. After receiving user intent information, the decision-making module decomposes the user intent information into an executable first instruction sequence and generates first UI data that matches the current interaction scenario. For example, when the user intent information is "Book me a flight", the decision-making module will generate: instruction sequence 1 (call the ticket booking system interface) and instruction sequence 2 (book a flight); the generated first UI data includes the ticket booking interface, ticket booking information, etc.

[0103] In this embodiment, the system-level AI engine includes an intent recognition module and a decision module. The intent recognition module analyzes and identifies user input to obtain structured user intent information. The decision module decomposes this user intent information into an executable first instruction sequence and simultaneously generates UI description data matching the interaction requirements. Through this modular collaborative processing, the system-level AI engine can convert ambiguous natural language instructions or touch operations into a low-level executable instruction sequence and a high-level renderable interface data, avoiding the complexity and latency caused by multiple applications collaboratively parsing and transmitting data, thereby improving the accuracy of intent parsing and the efficiency of task execution.

[0104] In some embodiments of this application, the system-level AI engine 100 is specifically used for: upon receiving user input information, searching for contextual information related to the input information; and identifying user intent information based on the input information and the contextual information.

[0105] The context information includes at least one of the following: interaction session context information, user behavior information, and user profile information.

[0106] In some embodiments of this application, the system-level AI engine 100 retrieves and associates relevant contextual information from the context memory module, thereby making the recognition results more accurate. For example, when a user says "navigate me to there", the system completes the user intent information as "navigate to that location" by obtaining the most recently mentioned location in the history.

[0107] In some embodiments of this application, the electronic device includes working memory, long-term memory, and a rule storage area. The working memory is used to store the short-term context of the current interactive session (e.g., the last 5 conversations); the long-term memory is used to store user behavior patterns and personal preference profiles (e.g., frequently visited locations, preferred applications, and habitual operation sequences); and the rule storage area is used to store preset priority protection rules (e.g., "unauthorized invocation of payment services is not allowed").

[0108] In some embodiments of this application, the system-level AI engine 100 further includes: a context memory module connected to the intent recognition module, used to store and manage context information; the intent recognition module is also used to search for context information related to the input information from the context memory module when receiving user input information; the intent recognition module is also used to identify user intent information based on the input information and the context information.

[0109] In some embodiments of this application, the context memory module is used to store and manage context information. For example, the context information may include the short-term history of the current session, the user's long-term behavioral habits, and preset important rules.

[0110] In some embodiments of this application, when recognizing user intent information, the intent recognition module retrieves and associates relevant contextual information from the context memory module, thereby making the recognition results more accurate. For example, when a user says "navigate me to there," the intent recognition module combines the most recently mentioned location in the history to complete the user intent information as "navigate to that location."

[0111] In some embodiments of this application, the context memory module includes working memory, long-term memory, and rule storage area. Working memory is used to store the short-term context of the current interaction session (e.g., the last 5 conversations); long-term memory is used to store user behavior patterns and personal preference profiles (e.g., frequently visited locations, preferred applications, and habitual operation sequences); and rule storage area is used to store preset priority protection rules (e.g., "unauthorized invocation of payment services is not allowed").

[0112] In some embodiments of this application, data access priority in the rule storage area is higher than that in working memory and long-term memory, ensuring that critical rules are not forgotten.

[0113] In this embodiment, by storing context information, such as storing the short-term context of the current session in working memory, the AI ​​engine can accurately understand the user's intent in the current interaction. By storing user behavior patterns and personal preference profiles in a long-term memory, the AI ​​engine can provide personalized services and interface presentation. In one possible implementation, priority-protected rules are stored in a rule storage area, giving them a higher data access priority than other memory modules to prevent important rules from being forgotten. This improves the accuracy of the AI ​​engine's understanding of user intent information and the intelligence of the interaction. The specific priority order can be adjusted according to actual needs, and this application does not impose specific limitations on it.

[0114] In some embodiments of this application, the system-level AI engine 100 is further configured to store the first UI data in a shared memory region; the system-level AI engine 100 is further configured to send a notification message to the UI rendering pipeline 200; the notification message is used to notify the reading of the first UI data from the shared memory region.

[0115] In some embodiments of this application, the aforementioned electronic device includes a shared memory region (i.e., a zero-copy shared memory region) connected to the system-level AI engine 100 and the UI rendering pipeline 200, respectively. After the system-level AI engine 100 generates the first UI data, it writes the data into the zero-copy shared memory region and passes the memory address to the UI rendering pipeline 200. The UI rendering pipeline 200 reads the data through direct mapping, avoiding the overhead of multiple data copies between processes in the traditional method.

[0116] Figure 5For shared memory timing diagrams, such as Figure 5 As shown, this sequence diagram illustrates the process of zero-copy data transfer between the AI ​​engine and the UI renderer via a shared memory mechanism, including the following steps:

[0117] Step 1: Write UI description data.

[0118] For example, after making interface decisions, the system-level AI engine directly writes the UI description data (ai_generated_data) into a pre-allocated shared memory area. This shared memory area is jointly created by the AI ​​engine and the UI renderer during system initialization, providing a common data space that both can directly access.

[0119] Step 2: Notify the renderer.

[0120] For example, after the system-level AI engine completes the data writing, it notifies the UI renderer (i.e., the UI rendering pipeline) through a lightweight notification mechanism that the new UI description data is ready for rendering. This notification mechanism has extremely low communication overhead compared to the data transmission itself.

[0121] Step 3: Read the data directly (no copying required).

[0122] For example, after receiving the notification, the UI renderer directly reads the UI description data from the shared memory area. Since the shared memory is accessed by both processes, the UI renderer does not need to perform additional data copying operations, avoiding the memory bandwidth consumption and latency overhead caused by multiple data copies in traditional inter-process communication.

[0123] Step 4: Call the rendering API.

[0124] For example, the UI renderer calls the underlying graphics rendering API (such as OpenGLES, Vulkan, etc.) based on the read UI description data to convert the interface description into rendering instructions that the GPU can recognize.

[0125] Step 5: Rasterization.

[0126] After receiving the rendering instructions, the GPU performs rasterization, converting the vectorized UI elements into screen pixel data and completing graphics processing operations such as geometric transformation, texture mapping, and shading.

[0127] Step 6: Frame buffer writing.

[0128] For example, after rasterization is completed, the GPU writes the generated pixel data into the frame buffer and waits for the display controller to output the frame buffer content to the screen to complete the final interface display.

[0129] In this embodiment, the electronic device further includes a zero-copy shared memory area, which is connected to both the system-level AI engine and the UI rendering pipeline. After the system-level AI engine generates UI description data, it maps this data directly from the AI ​​engine's memory space to the UI rendering pipeline's memory space through the shared memory area. This achieves zero-copy data transmission, reduces data transmission latency between the AI ​​engine and the UI rendering pipeline, and thus improves the real-time performance and smoothness of interface rendering.

[0130] In some embodiments of this application, the system-level AI engine 100 and the system capability bus 300 interact via a dedicated communication protocol;

[0131] The aforementioned dedicated communication protocol includes at least one of the following:

[0132] Synchronous invocation protocol, used by system-level AI engines to synchronously invoke hardware components and / or system services and wait for return information;

[0133] An asynchronous call protocol is used for system-level AI engines to asynchronously call hardware components and / or system services without waiting for return information;

[0134] The event subscription protocol is used by the system-level AI engine to subscribe to system events from the system capability bus and receive notifications when system events occur.

[0135] A streaming subscription protocol used for system-level AI engines to receive real-time data streams.

[0136] For example, the synchronous call protocol is used in scenarios where the system-level AI engine 100 needs to obtain the return result immediately, such as reading the current value of the sensor. After the system-level AI engine 100 issues the call, it blocks and waits, and the system capability bus 300 performs the operation and returns the result.

[0137] For example, the asynchronous call protocol is used in scenarios where there is no need to wait for the result, such as sending a notification, and the system-level AI engine 100 immediately continues to execute subsequent operations after issuing the call.

[0138] For example, the event subscription protocol is used for the system-level AI engine 100 to subscribe to specific system events from the system capability bus 300, such as when a Bluetooth device is connected or when the battery level is below a threshold. When the event occurs, the system capability bus 300 actively pushes a notification to the system-level AI engine 100.

[0139] like Figure 6 The diagram shown illustrates the types of AI communication protocols. Figure 6As shown, the AI ​​engine interacts with the hardware abstraction layer, system service layer, sensor layer, and UI rendering layer through an AI-specific communication protocol layer. This protocol layer provides four communication modes for different interaction scenarios: Event Subscription: This mode is used for scenarios that require receiving system event notifications. The AI ​​engine obtains asynchronous events such as incoming call notifications and system state changes through a subscription mechanism, ensuring real-time perception of system state changes without active polling. Stream Subscription: This mode is suitable for scenarios that require continuously receiving real-time data streams. For example, in scenarios such as camera preview data and sensor data streams, the AI ​​engine establishes a continuous data channel through stream subscription to achieve high-frequency, low-latency data transmission. Asynchronous Call: This mode is used to handle high-throughput, non-blocking call scenarios. When the AI ​​engine initiates time-consuming operations such as network requests and file processing, asynchronous calls are used to avoid blocking the main thread and improve the overall concurrency processing capability of the system. Synchronous Call: This mode is suitable for scenarios with extremely high latency requirements. For example, in scenarios such as UI interaction and real-time sensor reading, the AI ​​engine adopts a synchronous calling method to ensure accurate return results in a short time, thus guaranteeing the real-time performance and smoothness of the interaction.

[0140] like Figure 7 The diagram shown illustrates the event notification subscription process. Figure 7 As shown, the AI ​​engine first initiates an event subscription request (event_subscribe) to register with the system to listen for specific events. After completing the subscription, the AI ​​engine enters an event waiting state, continuously listening for events occurring at the system level. When an incoming call event occurs, the subscription mechanism triggers the corresponding callback function, passing the incoming call event information to the AI ​​engine. When a new notification event occurs, the notification content is also reported to the AI ​​engine through the callback mechanism. When the system state changes (such as network switching, battery level changes, etc.), the system state change event also triggers a callback, notifying the AI ​​engine of the current state change. After receiving callbacks for various events, the AI ​​engine parses and processes the event content, determining whether the user interface needs to be updated based on the event type and context information. For example, an incoming call event may require displaying a floating incoming call interface, a new notification event may require displaying a notification card, and a system state change may require informing the user of the current network status. If the AI ​​engine determines that a UI update is needed, it immediately calls the rendering mechanism to generate a new interactive interface and outputs it to the screen; if it determines that a UI update is not needed, it directly returns to the waiting state and continues listening for subsequent events. After completing UI rendering, the AI ​​engine also returns to a waiting state, continuing to listen for system events, forming a closed loop of "subscription - waiting - event triggering - processing - (on-demand rendering) - returning to waiting".

[0141] like Figure 8 The diagram shown illustrates the process of streaming data subscription. Figure 8 As shown, the AI ​​engine first initiates a streaming subscription request (stream_subscribe), specifying the sensor ID (sensor_id) to be monitored and the callback function for data processing in the request. This subscription request is sent to the corresponding sensor module through the AI-specific communication protocol layer. After the subscription is established, a continuous data channel is formed between the AI ​​engine and the sensor. The sensor continuously collects data in the form of a stream and continuously pushes the real-time data stream to the AI ​​engine. Each time a frame of data is collected, the sensor triggers the callback function, reporting the real-time data to the AI ​​engine. After receiving the real-time data, the AI ​​engine processes the data stream in real time. Typical application scenarios include camera preview scenarios, where the AI ​​engine continuously receives image frames captured by the camera and performs real-time image recognition, beautification processing, or scene analysis; and GPS positioning scenarios, where the AI ​​engine continuously receives location data to achieve real-time positioning functions such as navigation trajectory tracking and geofencing determination.

[0142] like Figure 9 The diagram shown illustrates the asynchronous call process. Figure 9 As shown, the AI ​​engine first initiates an asynchronous call request (async_call), specifying the system capability identifier (capability_id), the passed parameters (params), and the callback function (callback) for handling the returned result. This asynchronous call request is sent to the corresponding system capability module through the AI-specific communication protocol layer. The core feature of asynchronous calls is their non-blocking nature: the AI ​​engine receives a return immediately after initiating the call and will not wait for the operation to complete before continuing to execute other tasks. This mechanism ensures that the AI ​​engine can still respond to other user inputs, handle other system events, or execute parallel tasks while processing time-consuming operations. After receiving the asynchronous call request, the system capability module executes the time-consuming operation in the background. Typical application scenarios include network request scenarios, where the background initiates an HTTP request to obtain flight data; and file processing scenarios, where the background performs file reading and writing, image compression, and other operations. During background operations, the AI ​​engine remains active and is not affected by blocking. When the background time-consuming operation is completed, the system capability module actively returns the operation result, triggering the pre-set callback function (callback). The AI ​​engine obtains the operation results through callback functions and performs subsequent processing based on the results, such as updating the first UI data or triggering the next task.

[0143] like Figure 10 The diagram shown illustrates the synchronous call process. Figure 10 As shown, Figure 10The diagram shown illustrates the synchronous call process. Figure 10 As shown, this process demonstrates the complete procedure by which the AI ​​engine directly obtains the results returned by system capabilities through a synchronous call mechanism. The AI ​​engine initiates a synchronous call request (sync_call), specifying the system capability identifier (capability_id) and corresponding parameters (params) to be invoked. This synchronous call request is sent to the hardware abstraction layer or the corresponding system capability module through the AI-specific communication protocol layer. The core feature of synchronous calls lies in their blocking and waiting mechanism: after initiating the call, the AI ​​engine enters a waiting state until the system capability completes its operation and returns the result before continuing to execute subsequent tasks. This mechanism ensures that the AI ​​engine can immediately obtain the required accurate results, meeting the needs of scenarios extremely sensitive to latency. After receiving the synchronous call request, the hardware abstraction layer or system capability module executes the corresponding system capability operation, such as reading sensor data or obtaining the UI state. After the operation is completed, the result is returned to the AI ​​engine along the original path. Typical application scenarios include UI interaction scenarios where the AI ​​engine needs to obtain the current interface state or user input information in real time; and sensor reading scenarios where the AI ​​engine needs to immediately obtain the current values ​​of sensors such as gyroscopes and accelerometers. Synchronous calls ensure that the latency of these operations is kept within 10ms, guaranteeing the real-time performance and smoothness of the interaction. After obtaining the return result, the AI ​​engine continues to perform subsequent processing, such as updating the interface based on sensor data and triggering the next step of logic based on user input.

[0144] In this embodiment, the system-level AI engine interacts with the system capability bus via a dedicated communication protocol, which includes at least one of a synchronous call protocol, an asynchronous call protocol, an event subscription protocol, and a streaming subscription protocol. Through the synchronous call protocol, the AI ​​engine can block and wait when it needs to obtain hardware status or system service return results in real time, ensuring the data integrity of critical operations. Through the asynchronous call protocol, the AI ​​engine can execute non-blockingly in scenarios where waiting for results is not required, avoiding resource idleness. Through the event subscription protocol, the AI ​​engine can actively subscribe to specific system events and receive notifications when events occur, achieving real-time status awareness. Through the streaming subscription protocol, the system-level AI engine can receive real-time data streams.

[0145] In some embodiments of this application, the UI rendering pipeline 200 is further configured to: acquire second UI data; the second UI data is the UI data of the next frame to be rendered; and perform re-rendering based on the difference data between the first UI data and the second UI data to update the user interface.

[0146] In some embodiments of this application, the UI rendering pipeline obtains the first UI data of the current interface and the UI data of the next frame to be rendered, then compares the first UI data with the second UI data to identify the differences, and only performs re-rendering on the differences, thereby reducing the rendering load.

[0147] In some embodiments of this application, the UI rendering pipeline 200 includes: a differential update module, used to acquire first UI data and second UI data of the user interface; the second UI data is the UI data of the next frame to be rendered; the differential update module is also used to perform re-rendering based on the difference data between the first UI data and the second UI data to update the user interface.

[0148] In this embodiment, the UI rendering pipeline compares the current display interface with the UI description data of the next frame to be rendered, identifies the differences, and only performs re-rendering on the differences, thus avoiding the waste of computing resources caused by re-rendering the entire frame and improving rendering efficiency.

[0149] In some embodiments of this application, the UI rendering pipeline 200 is specifically used for: receiving a pre-rendering instruction generated by the system-level AI engine based on the user intent information; and pre-rendering at least a portion of the user interface based on the pre-rendering instruction.

[0150] In some embodiments of this application, the UI rendering pipeline 200 includes: a pre-rendering module, which is used to receive pre-rendering instructions generated by a system-level AI engine based on user intent information; the pre-rendering module is also used to pre-render at least a portion of the user interface content of the next frame based on the pre-rendering instructions.

[0151] In this embodiment, at least a portion of the user interface for the next frame is pre-rendered by a pre-rendering instruction generated based on user intent information. This enables interface updates when user intent information is clear, thereby reducing the perceived response delay and improving the rendering smoothness of dynamic interfaces.

[0152] In some embodiments of this application, the system-level AI engine 100 is further configured to generate a second instruction based on the input event when an input event is received, the second instruction being used to update the first UI data.

[0153] In some embodiments of this application, the electronic device further includes: an event listening module connected to the system-level AI engine 100; the UI rendering pipeline 200 is also used to output the user interface to the display screen; the event listening module is used to receive input events when they are detected and send the input events to the system-level AI engine 100; the system-level AI engine 100 is also used to parse the input events and generate a second instruction based on the input events, the second instruction being used to update the first UI data.

[0154] In some embodiments of this application, after the UI rendering pipeline 200 outputs the generated user interface to the display screen, the user can perform touch, voice, and other inputs on the interface. The event listening module 400 receives user input events on the user interface (such as the user clicking a button) and sends the input event to the system-level AI engine 100.

[0155] In some embodiments of this application, the system-level AI engine 100 parses the input event, updates the user intent information according to the event content (e.g., changing the intent to "browse photos" to "delete photos"), and then generates new instructions and new UI data according to the updated user intent information. For example, the UI data is the UI data of the interface after the deletion operation is performed. Then, the system-level AI engine calls the system capability bus 300 and the UI rendering pipeline 200 again to execute and render, so as to display the interface after deleting the photos, thereby realizing the closed loop of the interaction process.

[0156] In this embodiment, after the user interface is output to the display screen via the UI rendering pipeline, user input events are received and sent to the system-level AI engine. The system-level AI engine parses the input event, updates the user intent information based on the event, and generates new instructions and UI data according to the updated user intent information. The instructions are then executed again via the system capability bus, and the interface is rendered via the UI rendering pipeline. This allows the AI ​​engine to respond to every user operation in real time and dynamically adjust subsequent interface and behavior, improving the consistency and intelligence of human-computer interaction.

[0157] In some embodiments of this application, the system-level AI engine 100 is specifically used to determine a first task based on user intent information; determine the complexity of the first task; if the complexity is lower than a first threshold, call a first model to process the first task; if the complexity is higher than or equal to the first threshold, call a cloud server to run a second model to process the first task; and generate a first instruction based on the output of the first model or the second model.

[0158] In some embodiments of this application, the intelligent native terminal further includes an edge processing module and a cloud processing interface, both of which are connected to the system-level AI engine 100. The edge processing module is used to interact with a lightweight first model (e.g., a multimodal model with 3B parameters) deployed on the edge, and the cloud processing interface is used to communicate with a cloud server, which deploys a second model with a larger number of parameters.

[0159] In some embodiments of this application, the system-level AI engine 100 determines a first task to be performed (e.g., recognizing objects in an image) based on user intent information, and then judges the complexity of the task. If the complexity is below a threshold (e.g., simple object recognition), the edge processing module is invoked to run a first model to process the task; if the complexity is higher than or equal to the threshold (e.g., complex logical reasoning is required), the task is sent to a cloud server through a cloud processing interface, where a second model is run and processed by the cloud. After processing, the result is returned to the system-level AI engine 100, which then generates a first instruction and first UI data based on the result.

[0160] In this embodiment, the electronic device further includes an edge processing module and a cloud processing interface. After determining the first task based on user intent information, the system-level AI engine assesses the complexity of the task. Specifically, if the complexity is below a preset threshold, the edge processing module runs a locally deployed lightweight model to process the task; if the complexity is higher than or equal to the preset threshold, the task is sent to a cloud server via the cloud processing interface for processing by a larger model in the cloud. Through this edge-cloud collaborative processing mechanism, simple tasks are responded to quickly on the edge, while complex tasks are sent to the cloud for processing, ensuring processing efficiency.

[0161] like Figure 11 As shown, this application embodiment provides an information processing method, which may include the following steps 501:

[0162] Step 501: The electronic device executes the target task by calling the hardware components and / or system services of the electronic device based on the user intent information through the system-level AI engine.

[0163] In some embodiments of this application, the system-level AI engine 100 receives text input, voice input, touch input or other interactive input from the user, identifies user intent information, and then, based on the user intent information, invokes hardware components and / or system services of the electronic device to execute the target task.

[0164] In some embodiments of this application, the aforementioned target tasks include, but are not limited to: ticket booking, navigation, information query, device control, and content recommendation.

[0165] It should be noted that the objective may also include other tasks, which are not limited in this application.

[0166] For example, the system-level AI engine 100 receives a user's voice command, "Book me a flight from Shanghai to Beijing at 9:00 AM tomorrow." The system-level AI engine 100 parses and determines that the user's intent is "to book a flight." Then, based on this intent, the system-level AI engine 100 calls a flight query service and obtains flight information that meets the time and route requirements through a pre-defined ticketing system interface. After obtaining the flight data, the system-level AI engine 100 generates an interactive interface containing a flight list and selection controls for the user to select and confirm.

[0167] For example, the system-level AI engine 100 receives the text information "Help me set a meeting reminder for tomorrow afternoon at 3 pm, in meeting room A" input by the user. The system-level AI engine 100 determines the user's intent as "create a schedule reminder" through semantic analysis. Based on this intent, the system-level AI engine 100 calls the calendar service interface to complete the schedule creation and reminder settings.

[0168] In this way, the system-level AI engine 100 can determine the user's intent based on the user's input information and call various system services as needed to execute the target task, thereby improving task processing efficiency.

[0169] The information processing method provided in this application embodiment allows an electronic device to execute a target task by calling its hardware components and / or system services based on user intent information through a system-level AI engine. This approach allows the electronic device to directly call underlying hardware and system services via the system-level AI engine, eliminating the need for application proxy and thus reducing response latency and improving processing efficiency.

[0170] In some embodiments of this application, step 501 may include steps 501a and 501b:

[0171] Step 501a: The electronic device receives task information sent by hardware components and / or system services through the system-level AI engine, generates first UI data based on the task information, and transmits the first UI data to the UI rendering pipeline.

[0172] Step 501b: The electronic device generates and renders the user interface based on the first UI data through the UI rendering pipeline.

[0173] In some embodiments of this application, the electronic device parses first UI data through the UI rendering pipeline to generate and render the user interface.

[0174] In some embodiments of this application, the AI ​​engine directly controls the UI rendering pipeline to render images.

[0175] In some embodiments of this application, the UI rendering pipeline receives first UI data transmitted by the system-level AI engine, parses the interface description information (such as the list of train services to be displayed, seat selection controls, and payment buttons), and synthesizes and renders the user interface in real time. This user interface includes a train information display area, a seat selection interface, a user identity information confirmation area, and a payment entry point, and is ultimately output to a display screen for the user. The user can select a specific seat and complete the payment operation on this interface.

[0176] The following specific examples illustrate the information processing method provided in the embodiments of this application.

[0177] For example, after a user inputs the voice command "book a flight," the electronic device first understands and parses the user's intent. Once the system-level AI engine understands the user's intent, it can directly call the underlying flight query service interface or relevant applications to obtain relevant flight data. Subsequently, the AI ​​autonomously decides on the required interactive interface based on the returned data, determining that it should be displayed as a flight list. During the interface generation phase, the AI ​​engine directly calls the system's UI rendering pipeline, sequentially creating list components, populating flight data, and generating corresponding button components through the system-level UI API, completing the real-time construction and rendering of the entire flight list interface. Finally, the interface is directly displayed on the screen.

[0178] For example, when a user clicks on a flight on the interface, the AI ​​engine directly responds to the user's interaction and calls the UI rendering pipeline again to update the interface in real time based on the click result, realizing a closed loop of the entire process from intent understanding and capability invocation to interface generation and interaction response.

[0179] Using the above example, taking the user's input of "book me a flight ticket" as an example, the intent parsing and capability scheduling process of the electronic device in this application embodiment will be illustrated by way of example.

[0180] For example, after receiving user input, the AI ​​engine first performs intent analysis on the instruction to identify the core task of "booking a flight." Then, the AI ​​engine dynamically determines the required system capabilities based on the task requirements:

[0181] During the capability decision-making phase, the AI ​​sequentially evaluates key conditions in the task execution process. When it determines that the user's current location is needed, the AI ​​adds a location capability call (location.get) to the task planning; when it determines that network access is needed to query flight data, the AI ​​adds a network request capability (network.http); and when it determines that an interactive interface needs to be presented to the user, the AI ​​adds a UI rendering capability (ui.render). In the UI construction phase, the AI ​​determines the interface presentation format based on the task type "booking a flight," creating a corresponding list of UI elements, including a list component for displaying flight information (list: flight list) and a button component for user selection (button: select flight). If the task type matches other scenarios, the AI ​​can create other corresponding UI elements. Finally, the AI ​​integrates the above decision results with the capability call sequence, outputting a complete Plan object. This Plan object encapsulates the system capabilities, UI construction instructions, and execution order required for task execution, for subsequent execution engines to call sequentially. This example fully demonstrates the process of dynamic capability decision-making and interface planning by the AI ​​terminal before task execution, realizing structured scheduling from intent understanding to capability combination and then to interface generation.

[0182] Combining the examples above, such as Figure 12 As shown, taking the above-mentioned flight booking scenario as an example, the process of AI directly calling system capabilities in this application embodiment is illustrated by way of example.

[0183] For example, after completing intent parsing and task planning, the AI ​​generates and outputs a Plan object, which encapsulates a series of system capability call sequences required for task execution. Upon receiving the Plan object, the execution engine iterates through the system capability items in a preset order. For the location capability (location.get), the execution engine calls the GPS module to obtain the user's current location information. For the network request capability (network.http), the execution engine initiates an HTTP request to query flight data from the flight service provider that meets the criteria. If the Plan object also contains other system capabilities (such as user preference reading, payment capability preloading, etc.), the execution engine calls them sequentially. After each system capability completes its call, the returned results are uniformly stored in the results dictionary, forming a structured data set. The AI ​​then extracts the required data from the results dictionary, performs data processing and integration, including filtering, sorting, and format conversion of flight information. After data processing is complete, the AI ​​calls the rendering method (render_ui) to pass the processed flight data to the UI rendering pipeline, driving the creation and layout of interface components, ultimately presenting a complete flight list interface on the screen. The entire process realizes a complete closed loop from parsing the Plan object, sequentially calling system capabilities, unified data collection and processing, to the final rendering of the interface, fully demonstrating the AI ​​terminal's ability to directly schedule system capabilities and its efficient execution process.

[0184] like Figure 13 As shown, the process of AI directly controlling the UI rendering pipeline in the embodiments of this application is illustrated by way of example.

[0185] For example, after the AI ​​completes data processing, it calls the `render_ui` function to trigger the interface rendering process. This function first obtains an instance of the system's UI rendering pipeline and creates a screen object as the root container of the interface. Then, the AI ​​traverses each interface component according to the UI element structure defined in the `Plan` object. For the header component, the AI ​​creates a `HeaderView` and sets the title text; for the button component, the AI ​​creates a `ButtonView`, sets the button text, and binds a click event, which defines the response logic after the user clicks; for the list component, the AI ​​creates a `ListView`, iterates through the flight data items, creates a corresponding `ListItemView` for each flight data item, and adds the list items one by one to the `ListView`. After all UI components are created, the AI ​​adds these view components to the screen object according to their hierarchy, forming a complete view tree. The `render_ui` function then submits the view tree to the GPU for rendering, ultimately presenting a complete flight list interface on the screen, including a title, clickable buttons, and a list of flight data.

[0186] The following is an example of the AI ​​terminal interface generation process, which includes:

[0187] AI Decision-Making Phase: Based on user intent information and task requirements, the AI ​​engine plans and makes decisions on the interface, determines the UI structure, component types, data binding relationships, and interaction logic to be presented, and generates corresponding rendering instructions.

[0188] UI Description Phase: The output of AI decisions is transformed into structured UI description data. This description, in a declarative or imperative format, defines the interface layout, component properties, event bindings, and other details, serving as input for subsequent rendering pipelines.

[0189] Rendering pipeline stage: The rendering engine receives UI description data and converts it into a View Tree. The View Tree organizes all view components in the interface in a tree structure, clarifying the hierarchical relationship, layout parameters, and display attributes between components.

[0190] Rendering instruction conversion and GPU rendering stage: The rendering pipeline further converts the View Tree into rendering instructions that can be recognized by the underlying graphics API, and then hands them over to the GPU for rasterization, texture mapping, shading calculation and other graphics operations to complete the pixel-level rendering of the interface.

[0191] Display phase: The GPU writes the rendered frame data into the frame buffer, which is then output to the screen by the display controller, so that the end user can see the complete interactive interface.

[0192] like Figure 14 The diagram illustrates how AI's decisions are directly translated into rendering instructions. Figure 14 As shown, the process mainly includes:

[0193] AI Decision-Making Phase: The AI ​​engine completes interface planning based on user intent information and task requirements, and outputs a set of UI operation instructions.

[0194] The AI ​​decision-making process is further translated into a specific sequence of UI operations, including three types of atomic operations: creation (new), modification (modified), and deletion (deleted), which correspond to the addition, modification, and deletion of interface elements, respectively.

[0195] UI element traversal stage: The system traverses each UI element according to the operation sequence and performs corresponding view processing based on the element type and operation type.

[0196] For example, when the operation is "new" and the type is "header", a HeaderView is created; when the type is "list", a ListView is created; and when the type is "button", a ButtonView is created. For the "modified" operation, the view is updated; for the "deleted" operation, the view is deleted. After creation or update, all types of views are added to the rendering queue.

[0197] Rendering queue stage: The views corresponding to all UI operations are added to the rendering queue in sequence, forming a list of rendering tasks to be processed.

[0198] Rendering submission phase: Tasks in the rendering queue are submitted to the GPU in batches for rendering, completing the final presentation of the interface.

[0199] In this embodiment, the electronic device generates a first instruction based on user intent information through a system-level AI engine, and sends it directly to hardware components and / or system services for execution via the system capability bus, eliminating the need for application agents and removing intermediate instruction conversion and data transmission. Subsequently, the system-level AI engine receives task information, generates first UI data based on this execution information, and directly transmits it to the UI rendering pipeline for parsing and rendering. This approach achieves a complete closed loop from user intent information to instruction execution, and from execution results to interface presentation, significantly reducing response latency and improving task processing efficiency and system response performance.

[0200] In some embodiments of this application, before receiving task information sent by hardware components and / or system services through the system-level AI engine in step 501a above, the information processing method provided in this application embodiment may further include the following step 502:

[0201] Step 502: The electronic device generates a first instruction based on the user intent information through the system-level AI engine, and sends the first instruction to the hardware components and / or system services for execution through the system capability bus.

[0202] In some embodiments of this application, the user issues a request via voice or text. The AI ​​engine (as a system service) directly receives the input, further understanding the user's intent and determining what system capabilities and UI interfaces are needed.

[0203] For example, the system-level AI engine identifies the user's intent based on the dialogue information input by the user (such as "Help me book a high-speed rail ticket from Beijing to Shanghai tomorrow morning"), determines that ticketing information needs to be queried, and completes the booking operation. The system-level AI engine generates a first instruction (such as "Call the ticketing service to query train schedules and available tickets"), and sends the instruction directly to the ticketing system service for execution through the system capability bus, without going through any application proxy.

[0204] For example, in response to the first instruction, the ticketing system service returns information such as available train numbers, seat types, and fares as task information. Upon receiving this information, the system-level AI engine, combined with the user's initial dialogue ("high-speed rail ticket from Beijing to Shanghai tomorrow morning"), analyzes and filters the ticketing information using a multimodal visual perception model to determine train numbers that meet the user's needs and generates first UI data containing train details, seat selection, and payment options. Subsequently, the system-level AI engine directly transmits this first UI data to the UI rendering pipeline.

[0205] In some embodiments of this application, AI directly invokes system capabilities. For example, it directly invokes the system capability bus.

[0206] In some embodiments of this application, the information processing method provided in this application may further include steps 503 and 504:

[0207] Step 503: Upon receiving user input information, the electronic device searches for contextual information related to the input information using a system-level AI engine.

[0208] Step 504: The electronic device obtains user intent information based on input information and context information through a system-level AI engine.

[0209] The aforementioned contextual information includes at least one of the following: interactive session contextual information, user behavior information, and user profile information.

[0210] In some embodiments of this application, when an electronic device receives user input information, it searches for context information related to the input information from the context memory module.

[0211] In some embodiments of this application, the aforementioned context memory module may be a context memory unit.

[0212] In some embodiments of this application, the system-level AI engine continuously analyzes user behavior, constructs user preference profiles, and obtains user profile information.

[0213] In some embodiments of this application, the electronic device may store the user profile information in a context memory module.

[0214] In some embodiments of this application, the system-level AI engine can adjust the task output information of the target task in real time based on context information.

[0215] For example, the system-level AI engine obtains the dialogue information input by the user (such as "Help me book a high-speed rail ticket from Beijing to Shanghai tomorrow morning"), then obtains the context information related to the booking task from the context memory module. For example, the user's booking time preference is to book a high-speed rail ticket at 9 am. Then, based on the dialogue information and the context information, the system identifies the user's intent information as "book a high-speed rail ticket from Beijing to Shanghai at 9 am tomorrow morning", and calls system hardware components and / or system services based on the user's intent information, or it can call relevant applications to execute the booking task.

[0216] It should be noted that steps 503 and 504 can be executed after step 501, after step 502, or before step 502. This application embodiment does not limit the timing of the execution of steps 503 and 504.

[0217] In some embodiments of this application, the AI ​​engine includes a multi-level context memory module (i.e., a context memory module) which is responsible for storing and managing different types of context information.

[0218] like Figure 15 The diagram shown is a schematic of this multi-level context memory module. Figure 15As shown, the system includes: Working memory, which stores the latest context information in the current session and is managed using a sliding window mechanism with a maximum token limit of 32,000 tokens. When new information enters, the window slides forward, and information exceeding the window's range is removed from the working memory, ensuring controllable memory usage and optimal response speed. Session memory stores context information for a single complete session, with a maximum token limit of 1,000,000 tokens, supporting complete traceability. This level retains all user interaction history within a single conversation, facilitating the AI ​​engine's understanding of long-term context and maintaining conversational coherence. Persistent memory stores key information such as user preferences and historical behavior over the long term, implemented using a vector database with unlimited storage capacity. By vectorizing and persistently storing important information, the AI ​​engine can retain user-specific characteristics across different sessions, enabling cross-session memory reuse. The memory management mechanism employs an importance-weighted eviction strategy: dynamic management is performed based on the importance weight of information; low-importance memories are prioritized for eviction when the capacity reaches its limit, while high-importance memories are retained. This mechanism ensures that storage resources are focused on critical information while preventing unlimited expansion. Once the memory write process is complete, the storage status is confirmed, and the updated memory context is provided to the AI ​​engine for subsequent decision-making and response.

[0219] like Figure 16 The diagram shown illustrates the contextual retrieval process. Figure 16 As shown, the process includes: User Query: Receiving user input query requests as the starting point for retrieval. Multi-level Search: The system performs information retrieval in parallel or collaboratively across three memory levels: working memory: Retrieving immediate memory to obtain the most recent contextual information in the current session; session memory: Retrieving session memory to obtain complete historical interaction information in the current session; persistent memory: Based on vector similarity search, retrieving user preferences, historical behaviors, and other information related to the semantics of the current query from long-term storage, and selecting the top K results with the highest similarity (top_k=5). Result Merging: Merging the results retrieved from the three memory levels to form a unified candidate result set. Relevance Ranking: Ranking the merged result set based on relevance, comprehensively evaluating factors such as matching degree with the current query, timeliness, and importance to determine the priority order of the final results. Returning Results: Returning the ranked retrieval results to the AI ​​engine as contextual basis for subsequent decisions and responses.

[0220] Figure 17 A schematic diagram illustrating the priority protection of important rules provided in the embodiments of this application, such as... Figure 17 As shown, it includes:

[0221] The system receives three types of rule inputs: system-level rules, which have the highest priority and are used to ensure system stability and security; user rules, whose priority can be customized by the user to reflect their personalized preferences; and AI learning rules, which are automatically discovered and generated by the system to continuously optimize the service experience. The method for setting rule priorities can also be adaptively adjusted according to actual needs, and this application does not impose specific limitations on this.

[0222] The priority queue is used to manage rules according to their priority scores: rules with a priority score greater than 90 are assigned to the highest priority, rules with a priority score between 80 and 90 are assigned to high priority, rules with a priority score between 50 and 80 are assigned to medium priority, and rules with a priority score less than 50 are assigned to low priority.

[0223] Context injection and filtering: For rules with a priority of 50 or higher (i.e., highest, high, and medium priority), the system injects them into the System Prompt as contextual constraints for the large language model, ensuring that key rules are strictly followed during model inference. For low-priority rules with a priority below 50, the system performs a skip operation and does not inject them to avoid irrelevant information interfering with the model output.

[0224] The large language model, after being injected into the System Prompt, carries a set of priority-filtered rules, which serve as the basic constraints for generating responses by the large language model. This ensures that important system-level and user-level rules are executed first, while preventing low-priority rules from interfering with critical logic.

[0225] In some embodiments of this application, the AI ​​engine continuously analyzes user behavior and builds user preference profiles.

[0226] Figure 18 This application provides a possible schematic diagram for constructing a user preference profile, such as... Figure 18 As shown, it includes:

[0227] Learning sources: AI terminals continuously learn user preferences from multiple dimensions:

[0228] User choice: When a user adopts or selects a recommendation, the corresponding preference weight increases by 0.1;

[0229] User rejection: When a user rejects or ignores a recommendation, the corresponding preference weight decreases by 0.1;

[0230] Time pattern: Identify periodic preferences by learning users' habitual behaviors during specific time periods;

[0231] Feedback: By combining explicit and implicit user feedback, the preference model is continuously optimized.

[0232] User preference profiles are constructed by AI terminals based on data accumulated from learning sources, creating multi-dimensional user preference profiles:

[0233] Price sensitivity: Measured by statistics on the price that users are willing to accept, with a value ranging from 0 to 1.

[0234] Brand preferences: Records users' preferred airlines, hotels, product brands, etc., for personalized recommendations;

[0235] Preferred Time: Identify users' habitual time periods in scenarios such as calls, shopping, and dining, and provide timely services;

[0236] Communication style: Learn the details, tone, and language style that users prefer, such as concise or detailed, formal or friendly, etc.

[0237] Personalized output: Based on user preference profiles, AI terminals personalize the interactive interface, content recommendations, and response tone to deliver customized service experiences that match user preferences.

[0238] Figure 19 This is a schematic diagram illustrating the real-time personalized adjustment of user preference profiles provided in the embodiments of this application, such as... Figure 19 As shown, it includes:

[0239] User input: Users send commands or queries to the AI ​​engine, such as "Book me a flight" or "Recommend a restaurant".

[0240] Get current task preferences: The AI ​​engine calls the preference retrieval interface (get_preference_for_task()) in the user preference profile to retrieve personalized settings related to the current task type, such as the price range and brand preference in the context of booking air tickets.

[0241] Return to Preferences: User Preference Profile returns the preferences for the current task, including price range, preferred brands (such as specific airlines), communication style (such as concise or detailed, formal or friendly), and other personalized constraints.

[0242] Build personalized prompts: The AI ​​engine integrates the returned preference settings and builds prompts that include personalized constraints to ensure that the output of the large language model meets the personalized needs of the user's current task.

[0243] Sending Constrained Hints: The AI ​​engine sends the pre-built personalized hints to the large language model as the basis for response generation.

[0244] Generate personalized responses: The large language model generates personalized response content that matches user preferences based on constrained prompts.

[0245] Learning from this interaction: The AI ​​engine calls the learning interface (learn_from_interaction()) in the user preference profile, and passes in the user's explicit feedback (such as selection, rejection) or implicit feedback (such as dwell time, subsequent actions) in this interaction as learning samples.

[0246] Update Preference Weights: The user preference profile dynamically adjusts the weight values ​​of relevant preferences based on the feedback from this interaction. For example, if a user accepts a recommendation, the corresponding preference weight increases; if they reject it, the weight decreases, enabling the profile to evolve in real time.

[0247] Returning personalized results: The AI ​​engine returns the personalized response results generated by the large language model to the user, completing this interaction.

[0248] In this embodiment, when identifying user intent information, contextual information related to the current input information is actively retrieved from the electronic device, including the interaction session context, user behavior information, and user profile information. By combining contextual information for intent recognition, the system-level AI engine can accurately understand the user's references and omissions in the current interaction and provide personalized services and interface presentations based on the user's historical behavioral habits, thereby significantly improving the accuracy of intent recognition and the personalization of the interaction.

[0249] In some embodiments of this application, after generating and rendering the user interface in step 501b above, the following steps 505 and 506 are further included:

[0250] Step 506: The electronic device acquires the second UI data.

[0251] The second UI data mentioned above is the UI data for the next frame to be rendered.

[0252] In some embodiments of this application, the electronic device obtains second UI data through a differential update module.

[0253] The aforementioned first UI data is the first UI data of the next frame to be rendered;

[0254] Step 507: The electronic device performs a re-render based on the difference between the first UI data and the second UI data to update the user interface.

[0255] For example, the electronic device performs a re-render based on the difference between the first UI data and the second UI data through the differential update module to update the user interface.

[0256] Figure 20The diagram shown illustrates the Diff Update rendering process. Figure 20 As shown, the process includes:

[0257] Step 1: Obtain the old UI tree and the new UI description.

[0258] The system saves the currently rendered old UI tree structure and receives new UI description data generated by the AI ​​engine as the basis for updates.

[0259] Step 2: Compare the old and new UIs and calculate the differences.

[0260] By traversing the old and new UI structures using a difference comparison algorithm, the changes between the two are identified, and the differences are summarized into three basic operations: add, modify, and remove.

[0261] Step 3: Generate the corresponding rendering instructions.

[0262] Based on the type of difference, corresponding rendering commands are generated: the new operation corresponds to the create command (CreateCmd), the modification operation corresponds to the update command (UpdateCmd), and the deletion operation corresponds to the delete command (DeleteCmd).

[0263] Step 4: Perform incremental update.

[0264] The rendering engine executes rendering instructions in sequence: creating and adding new elements, updating attributes or content for modified elements, and removing deleted elements from the view tree.

[0265] Step 5: Generate a rendering instruction set, rendering only the changed parts.

[0266] The final rendering instruction set only includes the UI elements that have changed. The rendering pipeline only redraws these areas, while the unchanged parts retain their original display.

[0267] In this embodiment, the differential update module obtains the first UI data of the next frame to be rendered and compares it with the current first UI data to identify the differences. Only the differences are re-rendered. This mechanism avoids the waste of computational resources caused by redrawing the entire frame, reduces the load on the UI rendering pipeline, and significantly improves rendering efficiency and the smoothness of interface updates.

[0268] In some embodiments of this application, the information processing method may further include steps 508 and 509:

[0269] Step 508: The electronic device receives pre-rendering instructions generated by the system-level AI engine based on user intent information through the UI rendering pipeline.

[0270] Step 509: The electronic device pre-renders at least a portion of the user interface based on pre-rendering instructions via the UI rendering pipeline.

[0271] Figure 21 The diagram shown illustrates the pre-rendering process. Figure 21 As shown, the process includes: Input phase: The pre-rendering engine receives the current UI state and user context information as the basis for prediction. Prediction phase: The prediction engine performs AI prediction based on the input information, generating the user's most likely next action, and selects the top K candidates with the highest confidence (top_k=3 or top_k=5, the value of k can be set according to requirements). If K=3, it predicts the three most likely user actions. Pre-rendering phase: The pre-rendering engine performs pre-rendering for the three predicted actions respectively, generating corresponding interface rendering results, and storing these pre-rendering results in the cache. Caching phase: The pre-rendering result cache stores the pre-rendered interface data. When the user actually performs a click operation, the system first looks up the matching pre-rendering result in the cache. Hit response: If the user's click operation happens to match the pre-rendering result in the cache, the system directly uses the result for display without waiting for real-time rendering, thus achieving zero-latency interface switching. If it does not match, it falls back to the real-time rendering process.

[0272] In this embodiment, a pre-rendering module receives a pre-rendering instruction generated by a system-level AI engine based on user intent information, and pre-renders at least a portion of the user interface content for the next frame based on this instruction. This mechanism enables instantaneous interface switching when user intent information is clear, further reducing the perceived response latency and significantly improving the rendering smoothness and interaction response speed of dynamic interfaces.

[0273] In some embodiments of this application, the information processing method may further include the following step 510:

[0274] Step 510: Upon receiving an input event, the electronic device generates a second instruction based on the input event through the system-level AI engine.

[0275] The second instruction mentioned above is used to update the first UI data.

[0276] In some embodiments of this application, after the AI ​​engine completes the interface construction, it submits the view tree to be rendered to the system's UI rendering pipeline. The UI rendering pipeline performs layout calculations, layer compositing, and other processing on the view tree, and finally outputs the rendering result to the display screen. At this time, the user can see the complete interactive interface generated by AI in real time on the screen. This process is completed by the AI ​​directly controlling the rendering pipeline and does not depend on the UI components of third-party apps.

[0277] In some embodiments of this application, the electronic device is equipped with an event listening module (such as a touch event listener, a button event listener, etc.) to continuously monitor the user's input operations. When the user performs operations such as clicking or swiping on the AI-generated interface, the event listening module captures the input event, encapsulates it, and sends it to the system-level AI engine.

[0278] In some embodiments of this application, after receiving an input event, the system-level AI engine parses the event and identifies the user's interaction intent (e.g., clicking a flight option, clicking a function button, etc.). Subsequently, based on the parsing results and the current application state, the AI ​​engine autonomously makes a decision and generates a second instruction. This instruction is used to update the first UI data and may include obtaining new data, modifying the state of interface components, creating new interface elements, etc. The AI ​​engine sends this instruction to the UI rendering pipeline, triggering a dynamic update of the interface.

[0279] like Figure 22 The diagram shown illustrates the interaction logic between the user and the AI-generated interface. Figure 22 As shown, for example, when a user clicks a button (such as selecting a flight) on the AI-generated interface, this click triggers a click event, which is sent to the AI ​​engine for processing. Upon receiving the event, the AI ​​engine performs a logical judgment based on the event type and the current context. If the interaction requires obtaining new data (such as querying detailed ticket prices or seat information for the flight), the AI ​​engine directly calls the corresponding system capabilities (such as network requests, payment interfaces, etc.) to obtain the required data. After obtaining the new data, the AI ​​engine calls the `render_ui` method, passing in the new UI element definition and the updated data. Upon receiving the update instruction, the rendering pipeline creates or updates the corresponding UI components and submits the updated view tree to the GPU for re-rendering. Finally, the updated interface is displayed on the screen, and the user can see the dynamically refreshed content.

[0280] In some examples, during the "stating needs" phase, users input voice or text commands such as "Book me a flight." During the "response" phase, the AI ​​Agent in related technologies only returns recommendations in text or image format; users need to wait for a third-party app to open before viewing specific information. In contrast, the AI ​​terminal in this application, after understanding the intent, directly controls the UI rendering pipeline, and the screen immediately displays the complete booking interface without waiting for the app to launch. During the "waiting" phase, in related technologies, users experience the loading and launching process of the app, resulting in significant waiting time; in this application, the AI ​​terminal directly presents the interface, achieving instant response. During the "operation" phase, in related technologies, users need to complete subsequent operations such as flight selection and information entry within a third-party app; in this application, users operate directly within the AI-generated interface, with an interface style consistent with the native system experience. During the "feedback" phase, in related technologies, each user operation is responded to by the app, resulting in a lengthy response path; in this application, the AI ​​directly responds to user interaction events, achieving instant feedback. In the "end" phase, in related technologies, users need to manually exit the APP after completing the task; in this application, users can directly exit the AI ​​interface, and the interaction process does not involve opening and closing the APP.

[0281] Figure 23 A diagram illustrating the switching logic of the full-screen AI interactive interface, as shown below. Figure 23 As shown, the process includes: the AI ​​terminal is in desktop mode, waiting for user input. The user inputs their needs via voice or text, triggering the AI ​​terminal's response process. The AI ​​engine begins processing the user's needs, preparing to enter full-screen interactive mode. The system initiates the interface rendering process with a fade-in animation effect, calls the rendering function to begin building the initial interface, and after completing the initial interface preparation, executes a fade-out animation to smoothly transition to the formal interface display stage, entering the core rendering process. The interface is presented sequentially according to a progressive strategy of "skeleton screen → first screen → complete interface": first, the skeleton screen is displayed to provide a placeholder preview of the interface framework; then, the key content of the first screen is loaded; finally, the complete interactive interface is rendered to ensure the user's perceived response speed and the continuity of content filling. If the AI ​​terminal can autonomously complete the user's needs (such as simple queries, system settings, etc.), the task directly enters the completed state without additional user interaction. If the task requires user participation (such as selecting a flight or filling in information), it enters the interactive state, where the user operates on the AI-generated interface. During the interaction, if the user's operation triggers the acquisition of new data (such as changing flight dates), the system calls the update() method to refresh the interface based on the new data and returns to the rendering step to achieve incremental interface updates. When the user actively exits the interactive interface, or when the entire task flow is completed, the system ends the current full-screen AI interaction process and returns to the desktop or enters standby mode. Upon termination of the process, the system releases relevant resources.

[0282] In this embodiment, after the UI rendering pipeline outputs the user interface to the display screen, the event listening module listens for user input events on the interface in real time and sends the listened input events to the system-level AI engine. The system-level AI engine parses the input event and generates a second instruction based on the event content to update the first UI data. Through the above event listening and feedback mechanism, a complete interactive closed loop of "display-interaction-perception-response-redisplay" is formed, enabling the system-level AI engine to respond to every user operation in real time and dynamically adjust the subsequent interface presentation, realizing an intent-driven continuous interactive experience and significantly improving the coherence and intelligence of human-computer interaction.

[0283] It should be noted that each of the above method embodiments, or various possible implementations of each method embodiment, can be executed individually or in combination of any two or more. The specific implementation can be determined according to actual usage requirements, and this application embodiment does not impose any restrictions on this.

[0284] Optionally, such as Figure 24 As shown, this application embodiment also provides an electronic device 600, including a processor 601 and a memory 602. The memory 602 stores a program or instructions that can run on the processor 601. When the program or instructions are executed by the processor 601, they implement the various steps of the above-described information processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0285] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0286] Figure 25 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0287] The electronic device 100 includes, but is not limited to, components such as: radio frequency unit 101, network module 102, audio output unit 103, input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, and processor 110.

[0288] Those skilled in the art will understand that the electronic device 100 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 110 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 7 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0289] The processor 110 is used to: execute the target task by calling the hardware components and / or system services of the electronic device based on the user intent information through the system-level AI engine.

[0290] In some embodiments of this application, the processor 110 is specifically configured to receive task information sent by hardware components and / or system services through a system-level AI engine, generate first UI data based on the task information, and transmit the first UI data to the UI rendering pipeline; and generate and render the user interface based on the first UI data through the UI rendering pipeline.

[0291] In some embodiments of this application, the processor 110 is further configured to generate a first instruction based on user intent information through a system-level AI engine, and send the first instruction to hardware components and / or system services for execution via a system capability bus.

[0292] In some embodiments of this application, the processor 110 is further configured to: upon receiving user input information, search for contextual information related to the input information using a system-level AI engine; and obtain user intent information based on the input information and contextual information using the system-level AI engine; wherein the contextual information includes at least one of the following: interactive session contextual information, user behavior information, and user profile information.

[0293] In some embodiments of this application, the system-level AI engine interacts with the system capability bus via a dedicated communication protocol;

[0294] The dedicated communication protocol includes at least one of the following:

[0295] Synchronous invocation protocol, used by system-level AI engines to synchronously invoke hardware components and / or system services and wait for return information;

[0296] An asynchronous call protocol is used for system-level AI engines to asynchronously call hardware components and / or system services without waiting for return information;

[0297] The event subscription protocol is used by the system-level AI engine to subscribe to system events from the system capability bus and receive notifications when system events occur.

[0298] A streaming subscription protocol used for system-level AI engines to receive real-time data streams.

[0299] In some embodiments of this application, the processor 110 is further configured to acquire second UI data; the second UI data is the UI data of the next frame to be rendered; and to perform re-rendering based on the difference data between the first UI data and the second UI data to update the user interface.

[0300] In some embodiments of this application, the processor 110 is further configured to receive pre-rendering instructions generated by the system-level AI engine based on user intent information through the UI rendering pipeline; and pre-render at least a portion of the user interface based on the pre-rendering instructions through the UI rendering pipeline.

[0301] In some embodiments of this application, the processor 110 is further configured to generate a second instruction based on the input event through a system-level AI engine when an input event is received, the second instruction being used to update the first UI data.

[0302] It should be understood that, in this embodiment, the input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 107 includes at least one of a touch panel 1071 and other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0303] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 109 may include volatile memory or non-volatile memory, or it may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 109 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0304] Processor 110 may include one or more processing units; optionally, processor 110 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 110.

[0305] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described information processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0306] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0307] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above information processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0308] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0309] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the information processing method embodiments described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0310] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0311] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0312] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An electronic device, characterized in that, include: A system-level artificial intelligence (AI) engine is integrated into the operating system of the electronic device. The system-level AI engine is used to invoke the hardware components and / or system services of the electronic device based on user intent information to execute the target task.

2. The electronic device according to claim 1, characterized in that, The electronic device also includes: The UI rendering pipeline is connected to the system-level AI engine. Specifically, the system-level AI engine is used to receive task information sent by the hardware components and / or the system services, generate first UI data based on the task information, and transmit the first UI data to the UI rendering pipeline. The UI rendering pipeline is used to generate and render the user interface based on the first UI data.

3. The electronic device according to claim 2, characterized in that, The electronic device also includes a system capability bus, which is connected to the system-level AI engine, the hardware components of the electronic device, and the system services, respectively. Specifically, the system-level AI engine is used to generate a first instruction based on the user intent information, and send the first instruction to the hardware component and / or the system service for execution via the system capability bus.

4. The electronic device according to claim 2 or 3, characterized in that, The system-level AI engine is specifically used for: Upon receiving user input, search for contextual information related to the input. Based on the input information and the context information, user intent information is identified; The context information includes at least one of the following: interactive session context information, user behavior information, and user profile information.

5. The electronic device according to claim 2, characterized in that, The system-level AI engine is also used to store the first UI data in a shared memory area; The system-level AI engine is also used to send a notification message to the UI rendering pipeline; the notification message is used to notify the reading of the first UI data from the shared memory area.

6. The electronic device according to claim 3, characterized in that, The system-level AI engine and the system capability bus interact via a dedicated communication protocol; The dedicated communication protocol includes at least one of the following: A synchronous invocation protocol is used for the system-level AI engine to synchronously invoke the hardware components and / or the system services and wait for return information; An asynchronous call protocol is used for the system-level AI engine to asynchronously call the hardware components and / or the system services without waiting for return information; An event subscription protocol is used for the system-level AI engine to subscribe to system events from the system capability bus and to receive notifications when the system events occur. A streaming subscription protocol is used for the system-level AI engine to receive real-time data streams.

7. The electronic device according to claim 2, characterized in that, The UI rendering pipeline is also used for: Obtain the second UI data; the second UI data is the UI data of the next frame to be rendered; Based on the differences between the first UI data and the second UI data, a re-render is performed to update the user interface.

8. The electronic device according to claim 2, characterized in that, The UI rendering pipeline is specifically used for: Receive the pre-rendering instruction generated by the system-level AI engine based on the user intent information; At least a portion of the user interface is pre-rendered based on the pre-rendering instructions.

9. The electronic device according to any one of claims 2 to 8, characterized in that, The system-level AI engine is also used to generate a second instruction based on the input event when an input event is received, and the second instruction is used to update the first UI data.

10. An information processing method, characterized in that, Applied to an electronic device as described in any one of claims 1 to 9, the method comprises: The system-level AI engine invokes the hardware components and / or system services of the electronic device based on user intent information to execute the target task.

11. The method according to claim 10, characterized in that, The step of calling the hardware components and / or system services of the electronic device through a system-level AI engine to execute the target task includes: The system-level AI engine receives task information sent by the hardware components and / or the system services, generates first UI data based on the task information, and transmits the first UI data to the UI rendering pipeline. The UI rendering pipeline generates and renders the user interface based on the first UI data.

12. The method according to claim 11, characterized in that, Before receiving task information sent by the hardware components and / or the system services through the system-level AI engine, the method further includes: The system-level AI engine generates a first instruction based on the user intent information, and sends the first instruction to the hardware component and / or the system service for execution via the system capability bus.

13. The method according to claim 11 or 12, characterized in that, The method further includes: Upon receiving user input, the system-level AI engine searches for contextual information related to the input. The system-level AI engine obtains user intent information based on the input information and the context information; The context information includes at least one of the following: interactive session context information, user behavior information, and user profile information.

14. The method according to claim 12, characterized in that, The system-level AI engine and the system capability bus interact via a dedicated communication protocol; The dedicated communication protocol includes at least one of the following: A synchronous invocation protocol is used for the system-level AI engine to synchronously invoke the hardware components and / or the system services and wait for return information; An asynchronous call protocol is used for the system-level AI engine to asynchronously call the hardware components and / or the system services without waiting for return information; An event subscription protocol is used for the system-level AI engine to subscribe to system events from the system capability bus and to receive notifications when the system events occur. A streaming subscription protocol is used for the system-level AI engine to receive real-time data streams.

15. The method according to claim 11, characterized in that, After generating and rendering the user interface, the method further includes: Obtain the second UI data; the second UI data is the UI data of the next frame to be rendered; Based on the differences between the first UI data and the second UI data, a re-render is performed to update the user interface.

16. The method according to claim 11, characterized in that, The method further includes: The UI rendering pipeline receives pre-rendering instructions generated by the system-level AI engine based on the user intent information; At least a portion of the user interface is pre-rendered using the UI rendering pipeline based on the pre-rendering instructions.

17. The method according to any one of claims 11 to 16, characterized in that, The method further includes: Upon receiving an input event, the system-level AI engine generates a second instruction based on the input event, which is used to update the first UI data.

18. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the information processing method as described in any one of claims 10-17.

19. A chip comprising a processor and a communication interface coupled to the processor, the processor being configured to run a program or instructions to implement the steps of the information processing method as described in any one of claims 10-17.

Images