Desktop component generation method, apparatus, and electronic device

Abstract

The application discloses a desktop component generation method and device and electronic equipment, and belongs to the technical field of artificial intelligence. The method comprises the following steps: generating constraint rules based on collected user multi-modal demand information and desktop running environment restriction information; generating description codes and list incremental patches corresponding to desktop components based on the constraint rules; generating a desktop component installation package according to the description codes and the list incremental patches; and running the desktop component installation package and rendering the desktop components.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically relating to a desktop component generation method, apparatus, and electronic device. Background Technology

[0002] Existing AI interface generation solutions typically target web pages, web container pages, script pages, in-app cards, or general interface descriptions. The model directly generates page code based on natural language requirements, or first generates an intermediate description, which is then adapted to the specific runtime environment by a converter or developer. However, this approach is often only suitable for ordinary web pages or unrestricted containers, and its applicability to generating native desktop components of electronic device operating systems is poor. Summary of the Invention

[0003] The purpose of this application is to provide a desktop component generation method, apparatus, and electronic device that can improve the accuracy and environmental adaptability of desktop component generation.

[0004] In a first aspect, embodiments of this application provide a desktop component generation method, the method comprising: Based on the collected user multimodal demand information and desktop operating environment constraint information, constraint rules are generated; Based on constraint rules, iteratively generate description code and incremental patch manifest corresponding to desktop components; Generate a desktop component installation package based on the description code and manifest incremental patches; Run the desktop component installation package and render and generate the desktop component.

[0005] Secondly, embodiments of this application provide a desktop component generation apparatus, the apparatus comprising: The first generation module is used to generate constraint rules based on the collected user multimodal demand information and desktop operating environment constraint information; The second generation module is used to iteratively generate description code and manifest incremental patches corresponding to desktop components based on constraint rules; The third generation module is used to generate desktop component installation packages based on description codes and manifest incremental patches; The fourth generation module is used to run the desktop component installation package and render and generate the desktop component.

[0006] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0007] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0008] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0009] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0010] In this embodiment, constraint rules are generated by comprehensively considering the user's multimodal needs and the limitations of the desktop operating environment. Based on the constraint rules, description code and incremental patch of the manifest are generated iteratively. Finally, a desktop component installation package is generated and rendered, so that the generated desktop component can adapt to the user's actual needs and the native desktop environment of the operating system, thereby improving the accuracy of desktop component generation and environment adaptability. Attached Figure Description

[0011] Figure 1 These are schematic diagrams of a desktop component generation system provided in some embodiments of this application; Figure 2 This is a flowchart illustrating a desktop component generation method provided in some embodiments of this application; Figure 3 This is a schematic diagram illustrating the multi-round iterative generation in the desktop component generation method provided in some embodiments of this application; Figure 4 This is a schematic diagram of the same-origin verification in the desktop component generation method provided in some embodiments of this application; Figure 5 These are schematic diagrams illustrating the desktop component rendering process in some embodiments of this application. Figure 6 This is a schematic diagram of the structure of a desktop component generation apparatus provided in some embodiments of this application; Figure 7 These are schematic diagrams of the structure of electronic devices provided in some embodiments of this application; Figure 8 These are schematic diagrams of the hardware structure of electronic devices provided in some embodiments of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0013] The terms "first," "second," etc., used in this application's specification are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, in the specification, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects have an "or" relationship.

[0014] Existing AI interface generation solutions typically target web pages, web container pages, script pages, in-app cards, or general interface descriptions. The model directly generates page code based on natural language requirements, or it first generates an intermediate description, which is then adapted to the specific runtime environment by a converter or developer. However, this approach is often only suitable for ordinary web pages or unrestricted containers. When generating native desktop components for electronic device operating systems, it often encounters different types of host limitations, resulting in poor applicability.

[0015] Specifically, desktop widgets typically run within the launcher process, the negative one screen process, or a system-specified card runtime. The desktop host usually only exposes a subset of components, attributes, selectors, refresh mechanisms, and system capabilities. Furthermore, widget sizes are fixed, and the number of nodes, font size, spacing, data update paths, and cross-process communication methods may also be limited. If processed using ordinary page generation methods, the model may easily output components, style attributes, external resource links, or system capabilities not supported by the desktop host. The desktop host refers to the application responsible for hosting and managing widgets; that is, a desktop application running on an electronic device, such as the system launcher.

[0016] The inventors discovered that, for desktop component generation, some existing solutions maintain separate constraint rules for cloud generation, browser preview, compilation service, and real device rendering. Due to differences in protocol fields, default values, capability whitelists, or runtime versions used by different platforms, situations may arise where browser preview is normal but real device rendering fails, or the generated result can be compiled but cannot be added to the desktop or updated.

[0017] Additionally, some quick app cards or widgets use a method of launching a subprocess on each request. This method requires repeatedly loading the toolchain and rebuilding the temporary project directory, which is not conducive to fast previewing in conversational generation. Moreover, the operating system, such as adding a desktop on Android, may also be affected by the caller's installation package name, the content provider's installation package name, the timing of pending configuration writes, and cross-process state synchronization.

[0018] As built-in system capabilities are gradually opened to higher layers through WebView, JSBridge, Android Interface Definition Language (AIDL), ContentProvider, or plug-in modules, users no longer just need a static display card. Instead, they want to combine weather, calendar, notifications, media, voice, health, payment gateways, theme materials, and other application capabilities into desktop components tailored to the current task using natural language. Existing user interface generation methods lack a unified model of the operating system's (OS) native skills, host security boundaries, and desktop runtime lifecycle, making it difficult to stably support personalized user tasks.

[0019] In other words, the current generation of desktop components has the following problems: 1. Lack of host constraints in the generated target: The model is generated based on free pages and may output components, attributes, selectors, external resource links (external links) or system capabilities that are not supported by the host.

[0020] 2. Inconsistent protocol sources across multiple platforms: Rules are maintained separately in the cloud, browser preview, compilation, and real device, which can easily lead to field drift and runtime differences.

[0021] 3. Lack of automatic repair path after verification failure: When the generated result does not meet the constraints, it usually relies on manual modification, and the error information is not transformed into a structured input that the model can process again.

[0022] 4. High compilation chain startup cost: Each request starts a child process that repeatedly loads the toolchain and rebuilds the cache, increasing preview waiting time and hindering high-frequency interactive modifications.

[0023] 5. Adding a desktop to the operating system may encounter issues such as using the same installation package and race conditions: different installation packages between the generator and the content provider, asynchronous writing of pending configurations, or missing unique identifier mappings for desktop components may all lead to failures in adding or updating desktops.

[0024] 6. Lack of pre-verification and automatic repair mechanism in the model generation chain: The description code and incremental patch of the manifest generated by the model may have problems such as syntax exceptions, component structure errors, non-component user interface specifications, size out-of-bounds or misuse of disabled components. If these problems are not fixed until the real operating system rendering container fails to load, it will reduce the success rate of component generation and lengthen the user feedback chain.

[0025] Based on this, embodiments of this application provide a desktop component generation method, apparatus, and electronic device, enabling users to express personalized task requirements through a natural language interactive component generation development mode (vibecoding), and to generate stable and runnable desktop components within the size, components, style, system capabilities, and cross-process desktop addition limitations of the target desktop host.

[0026] The desktop component generation method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0027] like Figure 1 As shown, the desktop component generation method can be applied to a desktop generation system. The desktop generation system can include a generation-side APP101 (Application), a generation engine 102, a compilation service 103, and a desktop host App104. The generation-side APP101 can include a web page (H5), and the desktop host App104 can include a desktop launcher process.

[0028] Users can input their natural language requirements on the web page of the generation-side App 101. The generation engine 102 converts these requirements into a Domain-Specific Language (DSL), manifest patch, and optional operating system (OS) skill call configuration. The compilation service 103 can generate access addresses. The generation-side App 101 triggers real-device preview, screenshots, or desktop addition operations via a JavaScript communication bridge layer. The desktop host App 104 runs in the Launcher process and renders the generated desktop components for display.

[0029] In some examples, the system may also include a web preview client or a local quick app build scheduler (hapbroker), a native skill module on the generation side, etc.

[0030] like Figure 2 As shown, desktop component generation methods may include: Step 201: Generate constraint rules based on the collected user multimodal demand information and desktop operating environment constraint information.

[0031] In step 201, the web page of the generating app receives natural language input or multimodal tasks from the user, such as images, voice, or continuous dialogue, to obtain user multimodal requirement information, which may include at least one of the following: component size, style materials, and component modification instructions.

[0032] For example, the input can be a task description such as "create a 2x2 component that displays today's weather", "put my exercise, sleep and medication reminders on the desktop", or "create a pre-meeting reminder that can open meeting materials with one click". It can also include user-selected size, style, image materials, voice input results, or modification instructions for the desktop component generated in the previous round.

[0033] Based on the collected multimodal user demand information, it can output user requirements, size intentions, and optional material information as input for subsequent constraint rule normalization.

[0034] The constraints for this generation can be jointly determined by the generation app and the generation engine. The generation app can transmit user multimodal requirements and desktop runtime environment constraints, such as component size, runtime environment, output mode, whether to add the desktop to a real device, whether to preview on a webpage, and whether to modify existing desktop components, to the generation engine.

[0035] The generation engine normalizes this information into host constraint objects and binds them to finite size levels such as 2×2, 4×2, and 4×4, corresponding canvas widths, available components, minimum font size, number of lists, system capabilities, and the range of fields in the card manifest file, generating the final constraint rules. It's understandable that if the information sent by the generation app to the generation engine has already locked the size, subsequent generation, validation, and submission will all be based on that locked size.

[0036] In some examples, constraints may include component size tiers, a whitelist of available components, style restrictions, a list of system capabilities, and a range of fields in the manifest file. By using these constraints to generate subsequent desktop components, the model's generation space converges from that of ordinary pages to a restricted domain-specific description language acceptable to desktop components, ensuring that the generated target always remains within the capabilities of the desktop host.

[0037] Step 202: Based on the constraint rules, iteratively generate the description code and manifest incremental patch corresponding to the desktop component.

[0038] In step 202, based on constraint rules, multiple rounds of iterative generation can be performed through virtual tools. The verification results returned by the virtual tools can be used to guide model repair. If the requirement information is insufficient, follow-up questions are asked. If the request exceeds the capabilities of the desktop host, it is rejected. This process continues until the description code and manifest incremental patch corresponding to the desktop component that meets the requirements and is within the capabilities of the desktop host are generated.

[0039] The description code corresponding to the desktop component is a domain-specific description language. It can be Quick App card code, or it can be JSON Schema, XML layout, intermediate rendering tree, or host-customized component description code generated by other desktop card protocols. No specific restrictions are made here.

[0040] Taking the description code as the Quick App card code as an example, the description code can be a three-part structure, including template code, style code, and script code.

[0041] Among them, the inventory incremental patch can record the incremental configuration of system capability declaration fields (features).

[0042] In some embodiments, step 202 above may include: Construct a virtual tool runtime environment; the virtual tool runtime environment includes a read-only knowledge base and a read-write memory workspace; In the virtual tool runtime environment, based on constraint rules and a read-only knowledge base, the virtual tool performs multiple rounds of iterative generation processing in the read-write memory workspace to obtain the description code and manifest incremental patch corresponding to the generated desktop component. The multi-round iterative generation process includes context assembly executed by round, model streaming output parsing, incremental merging of virtual tool calls, execution of virtual tools, and backfeeding of virtual tool results.

[0043] In this embodiment, the generation engine can construct a virtual tool runtime environment for the intelligent execution agent. The virtual tool runtime environment can include a read-only knowledge base and a read-write memory workspace. That is, when the generation engine starts the current task, it provides the model with a read-only knowledge base and a read-write memory workspace. The read-only knowledge base is used to read quick app component specifications, style rules, operating system skill descriptions, and design constraints, while the read-write memory workspace is used to write description code and manifest incremental patches.

[0044] Agents can only interact through pre-defined virtual tools and cannot directly write to the final business deliverables. These virtual tools can include at least a workspace writing tool (ws_write), a workspace editing tool (ws_edit), a code verification tool (ws_lint), a component submission tool (report_quickapp_card), a requirement inquiry tool (report_ask), and an illegal requirement rejection tool (report_reject). In other words, the model can only read knowledge, write descriptive code and incremental patch manifests, call the code verification tool (lint), submit components, inquire with users, or reject out-of-bounds requirements through these virtual tools, thus restricting the model's free output to an auditable tool call trajectory.

[0045] It can run the Agent Core Loop mechanism to perform multiple rounds of iterative generation, such as... Figure 3 As shown, multi-round iterative generation can include context assembly 301, model streaming output parsing 302, incremental merging of virtual tool calls 303, virtual tool execution 304, and virtual tool result backfeeding 305. The callable tools include knowledge base reading tools, workspace reading, writing, and editing tools, code verification tools, card component submission tools, user requirement follow-up tools, and illegal requirement rejection tools. The generation engine can continue to guide model repair based on the verification results returned by the tools. When requirement information is insufficient, the requirement follow-up tool is invoked to initiate inquiries to the user; when the task exceeds the host's constraints, the illegal requirement rejection tool is invoked to terminate the process.

[0046] During the multi-round iterative generation process, the model can write description code and incremental manifest patches to the read-write memory workspace using a workspace writing tool or workspace editing tool, based on constraint rules, knowledge base specifications read from the read-only knowledge base, and selected skills. The description code can be a three-part .ux code, including at least template code, style code, and script logic code.

[0047] Manifest patches can record incremental manifest information such as system feature declaration fields. For example, a manifest patch can include two types of configuration content aligned with the generated artifact specifications: First, configurations aligned with the restricted domain-specific description language artifacts (RDLs) of the desktop component's restricted runtime environment, used to synchronize host constraint parameters such as component size levels, available component whitelists, style restrictions, and node quantity limits. Second, configurations aligned with the restricted domain-specific description language artifacts of the operating system design specifications, uniformly matching system interface standards such as minimum font size, spacing specifications, user interface (UI) layout boundaries, and disabled component lists.

[0048] In this way, by constructing a virtual tool runtime environment that includes a read-only knowledge base and a read-write memory workspace, and performing multiple rounds of iterative generation processing, we can make full use of the prior knowledge and constraint rules in the knowledge base, and gradually optimize the generation results in the read-write memory workspace, thereby improving the stability and accuracy of the generation of description code and manifest incremental patches.

[0049] Step 203: Generate the desktop component installation package based on the description code and manifest incremental patch.

[0050] In step 203, a real-time compilation can be performed based on the description code and manifest incremental patches via a compilation service to generate a desktop component installation package. It is understood that real-time compilation can be performed by a Quick App Package (RPK) compilation service or by other build executors; no specific limitation is made here. For the sake of understanding the technical solution provided in this application's embodiments, the following description will use the RPK compilation service performing real-time compilation as an example.

[0051] In some embodiments, prior to step 203 above, the method may further include: The description code and manifest incremental patches are verified using preset verification rules and restricted container runtime verification rules to obtain verification results; If the verification result indicates that the verification failed, a structured verification error message is output; the structured verification error message includes the verification rule code, error description information, and repair prompts. Based on the structured validation error information, partial repair processing is performed on the description code and manifest incremental patch to obtain the repaired description code and manifest incremental patch.

[0052] In this embodiment, same-origin structured verification can be performed before real-time compilation. For example, such as... Figure 4 As shown, a component task 401 can be submitted, and a same-origin structured verification 402 is performed based on the submitted component task. That is, the verification module in the generation engine calls the in-process code verification tool to perform syntax, component structure, logic script, style, card manifest file, size level, and disabled component rules verification on the description code and manifest incremental patch, and obtain the verification results.

[0053] Among them, the preset verification rules are pre-fixed basic specifications used to uniformly constrain the common UI, code syntax, basic size and other common standards of components.

[0054] The restricted container runtime verification rules rely on the Rust kernel verifier to implement dynamic and generalized verification logic. They dynamically perform verification judgments based on the current task, the device's available system capability pool, and the actual runtime environment of the desktop component. In other words, the restricted container runtime verification rules do not depend on static fixed configurations. They can dynamically verify the matching relationship between resources and capabilities based on user generation requirements and the actual operating system skills that the device can invoke. For example, if the user's requirement is to generate a desktop component displaying the weather of a certain location, the Rust kernel verifier will query the local weather skill's regional data acquisition capability pool in real time. If the device does not support fetching weather data for a certain location, the restricted container runtime verification rules will identify the missing capability and simultaneously provide a recommended solution for invoking appropriate system capabilities, allowing the Agent to select the appropriate capability to complete component generation.

[0055] If the validation fails, a structured validation error message 403 will be output. This structured validation error message 403 may include the validation rule code, error description information, and a hint. The error description information may include at least one of the following: error severity level, code location (such as code block, line number, column number), and error description text.

[0056] Whether it is a static specification error triggered by preset verification rules or a dynamic error of device capability or runtime environment adaptation identified by restricted container runtime verification rules, they will all be uniformly encapsulated into standardized structured verification error information output to support subsequent local automatic repair of incremental patches to description code and manifest.

[0057] An example of a structured validation error message 403 is as follows: "Validation rule code (rule): Minimum font size of style (STYLE_FONT_MIN)" Code location: style code.weather temperature.font size (style.weather-temp.font-size) Error message: The font size (font-size) in the 2x2 component is 8px, which is lower than the minimum font size of 9px. Repair suggestion: Adjust the font size to at least 9px and resubmit the card component submission tool (report_quickapp_card).

[0058] The structured validation error message 403 can be fed back into the model 404, driving the model to partially repair the code using the workspace editing tool and resubmit 405, and the validation-repair process is executed cyclically until the validation passes 406.

[0059] In this way, to address potential issues such as syntax errors, component structure errors, style out-of-bounds errors, size mismatches, and interface specification inconsistencies that may occur during model generation and description code, the system no longer waits until the real operating system's rendering container fails to load or throws runtime errors before manual troubleshooting. Instead, it performs pre-verification during the generation phase using a code verification tool (lint) module. The verification rules are derived from the final submission gate, covering both self-checks during generation and final checks at submission. The verification output is not a regular log or manual error report, but is converted into structured repair input that the model can process. The structured verification error information can include verification rule encoding, error description information, and repair suggestions. The generation engine can feed this information back to the model, allowing the model to partially repair and resubmit within the same workspace using the workspace editing tool. This forms a "generation—verification—repair—resubmit" loop, improving the success rate of desktop component generation and final compilation and rendering.

[0060] In this way, errors in the generation process can be detected and corrected in a timely manner, reducing the generation of invalid or erroneous installation packages and effectively improving the correctness and robustness of desktop component generation.

[0061] In some embodiments, step 203 above may include: If the verification result indicates that the verification passed, a unique installation package identifier, resource path, and version code are generated. Generate a desktop component installation package based on the unique package identifier, resource path, version code, description code, and manifest incremental patch.

[0062] In this embodiment, after the description code and manifest incremental patch pass the verification, a legitimate component task can be submitted through the process termination tool to generate a unique installation package identifier (package), resource path (path), and version code (versionCode), and enter RPK compilation. At the same time, the same-origin protocol artifact is returned in the task payload. The same-origin protocol artifact may include restricted domain-specific description language code, manifest incremental patch, and card package access address (rpk_url).

[0063] The RPK compilation service performs real-time compilation of resident RPK. That is, when the RPK compilation service starts, it binds a stable working directory to each resident worker process, copies the template project, loads the hap-toolkit, and performs a one-time throwaway compile for warmup initialization.

[0064] Once a real compilation request arrives, the RPK compilation service receives a unique installation package identifier, resource path, version code, description code, and manifest incremental patch, and generates an RPK, i.e., a desktop component installation package, by following a process of 1 / 4 overwriting the card manifest file, 2 / 4 writing the card source code, 3 / 4 calling the compilation, and 4 / 4 moving the artifacts.

[0065] In some examples, the build request verifies the installer identifier, resource path, version encoding, size specification, and template tags. The build service can be configured with a single worker process serial queue, multiple worker processes parallel processing, and real-time clock timeout protection to reduce the risk of upstream processes waiting indefinitely due to build hangs. In this way, the build service reduces the waiting cost and failure uncertainty of restarting the toolchain for each preview in conversational generation by using resident worker processes, a stable working directory, startup pre-warm-up initialization, a single resident worker process serial task queue, multiple resident worker processes parallel processing, card manifest file security filtering, and real-time clock timeout protection.

[0066] In this way, after verification, a unique installation package identifier, resource path and version code are generated, and real-time compilation is performed based on the stable working directory of the preheating initialization, which makes the generation process efficient and consistent. At the same time, the unique identifier and version code facilitate subsequent installation management and version updates, enhancing the manageability and reliability of the desktop component installation package.

[0067] Step 204: Run the desktop component installation package and render and generate the desktop component.

[0068] In step 204, the task to add is sent to the desktop host APP via the operating system's inter-process communication interface, and the transparent proxy activity page (PinWidgetActivity) within the desktop host is launched. The transparent proxy activity page synchronously writes the configuration to be applied before calling the add desktop card request interface (requestPinAppWidget), satisfying the constraint that the caller and the content provider (AppWidgetProvider) are in the same package.

[0069] Desktop components can be rendered and displayed within the Launcher process based on the desktop component installer.

[0070] In this embodiment, the desktop component generation method generates constraint rules by comprehensively considering the user's multimodal needs and the limitations of the desktop operating environment, and iteratively generates description code and manifest incremental patches based on the constraint rules, and finally generates and renders the desktop component installation package, so that the generated desktop component can adapt to the user's actual needs and the operating system's native desktop environment, thereby improving the accuracy of desktop component generation and environment adaptability.

[0071] In some embodiments, step 204 above may include: The interface for adding desktop cards, registered by the native system, is invoked through a communication bridge layer between the web page and the operating system. In response to the call to the Add Desktop Card interface, a task to add desktop cards is generated based on the desktop component installation package; The desktop task is sent to the desktop host through the operating system's inter-process communication interface, and the transparent agent activity page in the desktop host is explicitly started, writing the configuration to be applied into the desktop host. The desktop host determines the rendering parameters based on the configuration to be implemented, and then renders the desktop component based on the rendering parameters. The configurations that are yet to take effect are determined based on the addition of desktop tasks.

[0072] In this embodiment, the generating app can establish an agent-oriented JavaScript communication bridge layer (JS Bridge) within the embedded webpage. For example, the generating app uses a webpage view to host the webpage. The webpage uniformly calls native system capabilities through the JSBridge API and receives native push notifications by registering processing functions through the JS Bridge. The native side uniformly registers processing functions, verifies the source, executes coroutines, and generates a unified response envelope through the JS Bridge manager, using the standard channel of the general webpage view support library instead of using native object injection interfaces or self-developed webpage message channels.

[0073] Understandably, JS Bridge can integrate with skill sets from different operating systems. For example, capabilities such as notifications, media, voice, accounts, health, payments, themes, calendars, location, and deep app navigation links can be extended through a unified processing function registration and permission confirmation mechanism. The Agent only needs to generate the call configuration based on the skill description and input parameter protocol.

[0074] Operating system (OS) skills that can be scheduled by the intelligent execution agent or the web page (H5) are registered by the native system module on the generation side. The native system can register skills through the HandlerRegistry, WebExtensionRegistry, application initialization interface (IAppInitializer), or equivalent mechanisms. The notification module registers push notification capabilities (sendNotification), the media module registers photo taking capabilities (takePhoto) and image selection capabilities (pickImages), the quick app card pack module registers card pack preview interfaces (previewRpk), card screenshot interfaces (captureRpk), and desktop card adding interfaces (pinRpk), and the voice module can register voice recognition start capabilities (startAsr), voice recognition stop capabilities (stopAsr), voice broadcast capabilities (speak), etc. The web page (H5) / intelligent execution agent only perceives the processing function name (handler name) and input parameter structure. The specific permission requests, lifecycle and operating system (OS) application programming interface (API) calls are completed by the native system skill module.

[0075] The preview, screenshot, and addition of the card to the desktop are triggered by the web client or the agent. After receiving the Quick App card task (quickapp_card), the web client selects the card package preview interface (previewRpk), card screenshot interface (captureRpk), or desktop card addition interface (pinRpk) based on the runtime environment. Before calling, it checks necessary or optional fields such as card package name (rpkName), card resource path (rpkCardPath), card access address (cardUrl), version code (versionCode), component size (widgetSize), component name (widgetName), and component material (widgetMaterial).

[0076] The card preview interface (previewRpk) opens a user-mode preview page, the card screenshot interface (captureRpk) triggers off-screen rendering and returns a screenshot, and the add desktop card interface (pinRpk) generates a unique task configuration identifier (configId) and starts the subsequent desktop addition process. In some examples, whether the user confirms the addition of the desktop in the system pop-up window, and the actual unique identifier of the desktop component (widgetId) callback, are completed asynchronously by the operating system's cross-process communication interface and the desktop host process.

[0077] A communication bridge layer between the web client and the native system can be used to call the add desktop card interface registered by the native system. The call to the add desktop card interface includes checking at least one of the following: card package name, card resource path, card access address, version code, component size, component name, and component assets.

[0078] In response to the call to the Add Desktop Card interface, a Desktop Add Task can be generated based on the desktop component installation package. The Add Desktop Task can carry a unique identifier for the task configuration and structured data of the card package information.

[0079] like Figure 5 As shown, the generation side App501 constructs structured data of card package information and generates a unique identifier for task configuration. It then sends the added desktop task to the desktop host App503 via the task distribution interface of the operating system inter-process communication interface 502. Simultaneously, it explicitly launches the transparent proxy activity page within the desktop host App503. This transparent proxy activity page is installed in the same package as the desktop component content provider (AppWidgetProvider).

[0080] Before calling the Add Desktop Card interface (requestPinAppWidget), the configuration to be applied is synchronously written to the desktop host. The configuration to be applied may include a unique identifier for the task configuration, structured data of the card package information, component size, component name, component material, and content provider component identifier.

[0081] For example, you can write the task configuration unique identifier (configId), card package information structured data (rpkInfoJson), component size (widgetSize), component name (widgetName), component material (widgetMaterial), and content provider component identifier (providerComponent) into the transparent proxy activity page. This reduces the risk of race conditions where the configuration to be applied is not yet visible when the desktop launcher immediately triggers the update callback function (onUpdate).

[0082] In some examples, the ContentProvider can be used as an optional cross-process interface such as prepareAdd, status query, update address, and snapshot saving.

[0083] In the update callback function (onUpdate), the AppWidgetProvider of the desktop host App503 reads the configuration to be activated, associates the unique identifier of the desktop component (widgetId) with the unique identifier of the task configuration (configId), saves the activation configuration, and passes the rendering parameters such as the card package information structured data (rpkInfoJson), task name, component size (widgetSize), unique identifier of the desktop component (widgetId), outer corner radius value, and component material (widgetMaterial) to the desktop card root rendering view (PhxWidgetRootView) through the remote view container data packet binding method (RemoteViews.setBundle). The desktop card root rendering view (PhxWidgetRootView) runs within the desktop launcher 504 process. Through the HybridCardManager, it first executes the card installation method (installCard) and then the card creation method (createCard), creating card views according to the desktop grid specification (grid), design base width (designWidth), outer corner radius, and material or color mode to obtain the desktop component.

[0084] Understandably, desktop hosts are not limited to the Launcher. The negative one screen, lock screen, in-vehicle infotainment system desktop, tablet desktop, or foldable screen external screen can also serve as host environments, requiring only the replacement and addition of desktop proxies, size constraints, and rendering software development kits.

[0085] In some examples, if loading fails, a backoff strategy can be used to retry, and a fallback display can be shown after a threshold is exceeded.

[0086] In this way, the generating app does not directly act as the desktop content provider caller, but instead issues tasks and starts a transparent proxy activity page in the desktop host through the operating system's inter-process communication interface. This transparent proxy activity page calls the add desktop card interface in the same installation package context and writes the configuration to be effective synchronously before the call, thereby simultaneously satisfying the same installation package restriction and the race condition constraint of the desktop launcher updating the callback function too early.

[0087] In some embodiments, before calling the add desktop card interface registered by the native system through the communication bridge layer between the web page and the operating system, the method may further include: Based on the description code and manifest incremental patch, generate static preview resources and return the preview window address corresponding to the static preview resources.

[0088] In this embodiment, a preview of the same-sized QuickApp without adding a desktop path can be provided.

[0089] For example, in a browser or debugging environment, the web preview client can send the description code and manifest incremental patch output by the build engine to the local HAP broker. The HAP broker creates a separate build directory with a unique chatId, merges the default desktop component manifest file and manifest incremental patch, calls the HAP build operation, publishes static preview resources, and returns the address of a fixed-size embedded preview window (iframe).

[0090] Understandably, the same output—the description code and the manifest incremental patch—can be reused in web preview, RPK compilation, and the real device pipeline. This preview path does not depend on the application installer or JS Bridge and can use containers such as 2×2, 4×2, and 4×4 that match the target size. It can detect issues such as layout overflow, blank spaces, and size mismatches before adding the desktop on the real device.

[0091] In some examples, the preview process can be configured to select between web page preview, real device RPK preview, or screenshot preview, depending on the runtime environment. For environments that cannot be compiled in real time, off-screen rendering can be performed first using a container of the same specifications and a static runtime, and then the actual RPK compilation and desktop addition process can proceed after user confirmation.

[0092] In this way, before officially adding desktop components, static preview resources are generated based on the description code and incremental configuration of the manifest, and the preview window address is returned. This allows users to preview the appearance and effect of desktop components in advance, improving the user experience and making it easier for users to confirm before installation, effectively reducing accidental operations.

[0093] In some embodiments, after running the desktop component installation package and rendering the desktop component, the method may further include: Save a rendering snapshot of the desktop component; the rendering snapshot is used to preview the desktop component after it is generated and before it is added to the desktop.

[0094] In this embodiment, after the card view is created and mounted to the Launcher process, the desktop card root rendering view (PhxWidgetRootView) captures a frame of the actual rendered screen that has been laid out after a preset delay, and persists it as a rendering snapshot in lossless image format through the content provider snapshot persistence interface (saveSnapshot).

[0095] Render snapshots can be used to preview desktop components after they are generated but before they are added to the desktop. For example, when the system restarts, fonts are changed, themes are changed, or the Launcher process is rebuilt, the base desktop card content provider can display the most recent render snapshot when redeploying the RemoteViews containers, and then wait for the actual desktop components to be remounted.

[0096] When users modify desktop components and generate new card pack information structured data on the generation side, the generation side App can push update instructions through the operating system's cross-process communication interface based on the mapping between the unique identifier of the task configuration and the unique identifier of the desktop component. The desktop host updates the local configuration and triggers the corresponding content provider to re-issue the remote view container through the card view refresh broadcast, so that the desktop card root rendering view in the Launcher process is re-rendered.

[0097] In this way, a rendering snapshot is saved after the desktop component is generated, and the snapshot is displayed directly when the desktop component restarts. This improves the blank or flickering phenomenon caused by component loading delay and significantly improves the startup speed of the desktop component and the user's visual experience.

[0098] In some embodiments, after running the desktop component installation package and rendering the desktop component, the method may further include: Receive modification instructions from the user; Update desktop components in response to modification commands.

[0099] In this embodiment, the app can receive modification instructions input by the user. For example, when the user continues to input requirements such as "make the font bigger", "change to blue", or "add a meeting reminder", the generation side app can submit the modification instructions to the generation engine.

[0100] The generation engine can regenerate or partially adjust the current desktop components and re-enter the aforementioned generation, verification, and compilation process to update the content or configuration of the desktop components.

[0101] This allows the system to receive user input for modification and respond by updating desktop components, enabling the generated desktop components to be dynamically adjusted according to user needs. This enhances the flexibility and customizability of desktop components and meets the ever-changing user requirements.

[0102] In a specific example, taking "Generate Today's Weather 2×2 Desktop Component" as an example, after the user inputs the requirements, the system locks the 2×2 size, reads the weather display, font size, list quantity and network capability rules, and the model generates description code and incremental patch for the list.

[0103] If the temperature font size generated by the model is smaller than the minimum 2×2 font size, the code verification tool based on the Rust language kernel will return a structured error. The model should be modified by changing the font size and then resubmitted.

[0104] After verification, the RPK compilation service generates the installation package identifier, resource path, and card package access address. On the real device side, the desktop is added and rendered through the desktop card interface, operating system cross-process communication interface, transparent proxy activity page, desktop component content provider, and desktop card root rendering view. The weather card is displayed in the desktop launcher.

[0105] In one optional human-computer interaction implementation, the generating app displays natural language input, size and style selection, generation process status, webpage preview, real device preview, screenshots, saving, and adding desktop entry points. The desktop host app displays desktop components that have been added to the desktop, and users can jump back to the generating app through entry points to view more AI components or initiate further modifications.

[0106] The desktop host app does not directly edit description codes or manifest incremental patches. Content adjustments still require returning to the generation-side app to input modification requirements and re-entering the generation process.

[0107] In some examples, native system skills combined with natural language interactive components can generate task-oriented components that integrate with the development model. The system can register different operating system skills as skill descriptions that can be invoked by agents. Users can aggregate multiple skills with a single sentence to generate the desktop components required for the current task.

[0108] For example, a payment code aggregation component: If a user says, "Make a payment component that combines WeChat, Alipay, and bank card payment codes," the system generates a desktop component that only displays the entry point and does not cache sensitive code values, based on the application-deep redirection links of authorized payment applications, installation package name detection, and security confirmation capabilities. After clicking, the user is redirected to the corresponding application or system payment panel according to the selected payment method.

[0109] Family Health Data Dashboard: Users say, "Make my and my family's steps, sleep, heart rate, and medication reminders into a desktop dashboard." After user authorization, the system calls on health or family account-related skills to generate a 4x4 component based on family members and indicators, and reminds users of abnormal indicators or medication times through local notification skills.

[0110] Travel Task Component: When a user says, "I'm going on a business trip tomorrow morning, please make a travel component for me," the system aggregates calendar, weather, location, ride-hailing, flight or train information, payment entry, and notification reminders to generate a desktop component that is only valid during the travel period.

[0111] In some examples, themes, raw image capabilities, and desktop widgets can be linked. The system can access the operating system's built-in themes or theme store capabilities, combining raw image models and user-submitted materials to generate wallpapers, icons, widget backgrounds, and overall desktop themes. For example, if a user says, "Make a blue desktop theme using my beach photos, and generate weather, to-do, and photo album widgets," the system first generates theme materials, then generates corresponding desktop widgets according to the theme color and widget size, and submits them to the theme or desktop application pipeline after previewing and approval.

[0112] In some examples, users can freely aggregate any built-in system skills and capabilities to generate a short-lived desktop application or set of components based on real-time needs. For example, if the user wants to "apply for a visa today, and wants to put their passport photo, document checklist, appointment time, navigation, and payment entry on their desktop," the system can combine media, calendar, files, maps, notifications, and payment entry into a single task desktop, which is then automatically archived or converted into a history template after the task is completed.

[0113] In some examples, third-party applications can register some security capabilities with the system as skill descriptions, permission declarations, and processing function protocols. When the model generates components, it only selects the authorized skills and generates the call configuration. In this way, different applications do not need to develop fixed desktop components themselves, and users can combine the capabilities of multiple applications into a single desktop entry point according to their own tasks.

[0114] In some examples, self-optimizing components can be generated based on actual rendering results. This means that screenshots, layout trees, first-screen rendering time, and error codes can be uploaded after web preview or real-device rendering as input for the next round of model repair. The system not only validates syntax but can also fix issues such as whitespace, overflow, occlusion, excessively small font sizes, and unreasonable information density based on actual rendering results.

[0115] The desktop component generation method provided in this application can be executed by a desktop component generation device. This application uses an example of a desktop component generation device executing the desktop component generation method to illustrate the desktop component generation device provided in this application.

[0116] like Figure 6 As shown, the desktop component generation apparatus 600 provided in this application embodiment may include: The first generation module 601 is used to generate constraint rules based on the collected user multimodal demand information and desktop operating environment constraint information; The second generation module 602 is used to iteratively generate description code and manifest incremental patches corresponding to desktop components based on constraint rules; The third generation module 603 is used to generate desktop component installation packages based on description codes and manifest incremental patches; The fourth generation module 604 is used to run the desktop component installation package and render and generate the desktop component.

[0117] In this embodiment, the desktop component generation method generates constraint rules by comprehensively considering the user's multimodal needs and the limitations of the desktop operating environment, and iteratively generates description code and manifest incremental patches based on the constraint rules, and finally generates and renders the desktop component installation package, so that the generated desktop component can adapt to the user's actual needs and the operating system's native desktop environment, thereby improving the accuracy of desktop component generation and environment adaptability.

[0118] In some embodiments, the second generation module 602 may be specifically used for: Construct a virtual tool runtime environment; the virtual tool runtime environment includes a read-only knowledge base and a read-write memory workspace; In the virtual tool runtime environment, based on constraint rules and a read-only knowledge base, the virtual tool performs multiple rounds of iterative generation processing in the read-write memory workspace to obtain the description code and manifest incremental patch corresponding to the desktop component. The multi-round iterative generation process includes context assembly executed by round, model streaming output parsing, incremental merging of virtual tool calls, execution of virtual tools, and backfeeding of virtual tool results.

[0119] In this way, by constructing a virtual tool runtime environment that includes a read-only knowledge base and a read-write memory workspace, and performing multiple rounds of iterative generation processing, we can make full use of the prior knowledge and constraint rules in the knowledge base, and gradually optimize the generation results in the read-write memory workspace, thereby improving the stability and accuracy of the generation of description code and manifest incremental patches.

[0120] In some embodiments, the desktop component generation apparatus 600 further includes a verification module, which can be used for: Before generating the desktop component installation package based on the description code and manifest incremental patch, the description code and manifest incremental patch are verified using preset verification rules and restricted container runtime verification rules to obtain the verification results. If the verification result indicates that the verification failed, a structured verification error message is output; the structured verification error message includes the verification rule code, error description information, and repair prompts. Based on the structured validation error information, partial repair processing is performed on the description code and manifest incremental patch to obtain the repaired description code and manifest incremental patch.

[0121] In this way, errors in the generation process can be detected and corrected in a timely manner, reducing the generation of invalid or erroneous installation packages and effectively improving the correctness and robustness of desktop component generation.

[0122] In some embodiments, the third generation module 603 may be specifically used for: If the verification result indicates that the verification passed, a unique installation package identifier, resource path, and version code are generated. Generate a desktop component installation package based on the unique package identifier, resource path, version code, description code, and manifest incremental patch.

[0123] In this way, after verification, a unique installation package identifier, resource path and version code are generated, and real-time compilation is performed based on the stable working directory of the preheating initialization, which makes the generation process efficient and consistent. At the same time, the unique identifier and version code facilitate subsequent installation management and version updates, enhancing the manageability and reliability of the desktop component installation package.

[0124] In some embodiments, the fourth generation module 604 may be specifically used for: The interface for adding desktop cards, registered by the native system, is called through a communication bridge layer between the web page and the native system. In response to the call to the Add Desktop Card interface, a task to add desktop cards is generated based on the desktop component installation package; The desktop task is sent to the desktop host through the operating system's inter-process communication interface, and the transparent agent activity page in the desktop host is explicitly started, writing the configuration to be applied into the desktop host. The desktop host determines the rendering parameters based on the configuration to be applied, and then renders the desktop component based on the rendering parameters. The configurations that are yet to take effect are determined based on the addition of desktop tasks.

[0125] In this way, the generating app does not directly act as the desktop content provider caller, but instead issues tasks and starts a transparent proxy activity page in the desktop host through the operating system's inter-process communication interface. This transparent proxy activity page calls the add desktop card interface in the same installation package context and writes the configuration to be effective synchronously before the call, thereby simultaneously satisfying the same installation package restriction and the race condition constraint of the desktop launcher updating the callback function too early.

[0126] In some embodiments, the desktop component generation apparatus 600 further includes a preview module, which can be used for: Before calling the add desktop card interface registered by the native system through the communication bridge layer between the web page and the native system, static preview resources are generated based on the description code and manifest incremental patch, and the preview window address corresponding to the static preview resources is returned.

[0127] In this way, before officially adding desktop components, static preview resources are generated based on the description code and incremental configuration of the manifest, and the preview window address is returned. This allows users to preview the appearance and effect of desktop components in advance, improving the user experience and making it easier for users to confirm before installation, effectively reducing accidental operations.

[0128] In some embodiments, the desktop component generation apparatus 600 further includes a storage module, which can be used for: Run the desktop component installation package and render the desktop component, then save a rendering snapshot of the desktop component; the rendering snapshot is used to preview and display the desktop component after it is generated and before it is added to the desktop.

[0129] In this way, a rendering snapshot is saved after the desktop component is generated, and the snapshot is displayed directly when the desktop component restarts. This improves the blank or flickering phenomenon caused by component loading delay and significantly improves the startup speed of the desktop component and the user's visual experience.

[0130] In some embodiments, the desktop component generation apparatus 600 further includes a modification module, which can be used for: After running the desktop component installation package and rendering and generating the desktop component, it receives modification commands input by the user. Update desktop components in response to modification commands.

[0131] This allows the system to receive user input for modification and respond by updating desktop components, enabling the generated desktop components to be dynamically adjusted according to user needs. This enhances the flexibility and customizability of desktop components and meets the ever-changing user requirements.

[0132] The desktop component generation device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0133] The desktop component generation device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0134] The desktop component generation apparatus provided in this application embodiment can implement the various processes implemented in the method embodiment, and will not be described again here to avoid repetition.

[0135] Optionally, such as Figure 7 As shown, this application embodiment also provides an electronic device 700, including a processor 701 and a memory 702. The memory 702 stores a program or instructions that can run on the processor 701. When the program or instructions are executed by the processor 701, they implement the various steps of the desktop component generation method embodiment described above and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0136] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0137] Figure 8 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application.

[0138] The electronic device 800 includes, but is not limited to, components such as: radio frequency unit 801, network module 802, audio output unit 803, input unit 804, sensor 805, display unit 806, user input unit 807, interface unit 808, memory 809, and processor 810.

[0139] Those skilled in the art will understand that the electronic device 800 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 810 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 8 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0140] The processor 810 can be used for: Based on the collected user multimodal demand information and desktop operating environment constraint information, constraint rules are generated; Based on constraint rules, iteratively generate description code and incremental patch manifest corresponding to desktop components; Generate a desktop component installation package based on the description code and manifest incremental patches; Run the desktop component installation package and render and generate the desktop component.

[0141] In this embodiment, the desktop component generation method generates constraint rules by comprehensively considering the user's multimodal needs and the limitations of the desktop operating environment, and iteratively generates description code and manifest incremental patches based on the constraint rules, and finally generates and renders the desktop component installation package, so that the generated desktop component can adapt to the user's actual needs and the operating system's native desktop environment, thereby improving the accuracy of desktop component generation and environment adaptability.

[0142] In some embodiments, the processor 810 can also be used for: Construct a virtual tool runtime environment; the virtual tool runtime environment includes a read-only knowledge base and a read-write memory workspace; In the virtual tool runtime environment, based on constraint rules and a read-only knowledge base, the virtual tool performs multiple rounds of iterative generation processing in the read-write memory workspace to obtain the description code and manifest incremental patch corresponding to the desktop component. The multi-round iterative generation process includes context assembly executed by round, model streaming output parsing, incremental merging of virtual tool calls, execution of virtual tools, and backfeeding of virtual tool results.

[0143] In this way, by constructing a virtual tool runtime environment that includes a read-only knowledge base and a read-write memory workspace, and performing multiple rounds of iterative generation processing, we can make full use of the prior knowledge and constraint rules in the knowledge base, and gradually optimize the generation results in the read-write memory workspace, thereby improving the stability and accuracy of the generation of description code and manifest incremental patches.

[0144] In some embodiments, the processor 810 can also be used for: Before generating the desktop component installation package based on the description code and manifest incremental patch, the description code and manifest incremental patch are verified using preset verification rules and restricted container runtime verification rules to obtain the verification results. If the verification result indicates that the verification failed, a structured verification error message is output; the structured verification error message includes the verification rule code, error description information, and repair prompts. Based on the structured validation error information, partial repair processing is performed on the description code and manifest incremental patch to obtain the repaired description code and manifest incremental patch.

[0145] In this way, errors in the generation process can be detected and corrected in a timely manner, reducing the generation of invalid or erroneous installation packages and effectively improving the correctness and robustness of desktop component generation.

[0146] In some embodiments, the processor 810 can also be used for: If the verification result indicates that the verification passed, a unique installation package identifier, resource path, and version code are generated. Generate a desktop component installation package based on the unique package identifier, resource path, version code, description code, and manifest incremental patch.

[0147] In this way, after verification, a unique installation package identifier, resource path and version code are generated, and real-time compilation is performed based on the stable working directory of the preheating initialization, which makes the generation process efficient and consistent. At the same time, the unique identifier and version code facilitate subsequent installation management and version updates, enhancing the manageability and reliability of the desktop component installation package.

[0148] In some embodiments, the processor 810 can also be used for: The interface for adding desktop cards, registered by the native system, is called through a communication bridge layer between the web page and the native system. In response to the call to the Add Desktop Card interface, a task to add desktop cards is generated based on the desktop component installation package; The desktop task is sent to the desktop host through the operating system's inter-process communication interface, and the transparent agent activity page in the desktop host is explicitly started, writing the configuration to be applied into the desktop host. The desktop host determines the rendering parameters based on the configuration to be applied, and then renders the desktop component based on the rendering parameters. The configurations that are yet to take effect are determined based on the addition of desktop tasks.

[0149] In this way, the generating app does not directly act as the desktop content provider caller, but instead issues tasks and starts a transparent proxy activity page in the desktop host through the operating system's inter-process communication interface. This transparent proxy activity page calls the add desktop card interface in the same installation package context and writes the configuration to be effective synchronously before the call, thereby simultaneously satisfying the same installation package restriction and the race condition constraint of the desktop launcher updating the callback function too early.

[0150] In some embodiments, the processor 810 can also be used for: Before calling the add desktop card interface registered by the native system through the communication bridge layer between the web page and the native system, static preview resources are generated based on the description code and manifest incremental patch, and the preview window address corresponding to the static preview resources is returned.

[0151] In this way, before officially adding desktop components, static preview resources are generated based on the description code and incremental configuration of the manifest, and the preview window address is returned. This allows users to preview the appearance and effect of desktop components in advance, improving the user experience and making it easier for users to confirm before installation, effectively reducing accidental operations.

[0152] In some embodiments, the processor 810 can also be used for: After running the desktop component installation package and rendering the desktop component, save a rendering snapshot of the desktop component; the rendering snapshot is used to preview and display the desktop component after it is generated and before it is added to the desktop.

[0153] In this way, a rendering snapshot is saved after the desktop component is generated, and the snapshot is displayed directly when the desktop component restarts. This improves the blank or flickering phenomenon caused by component loading delay and significantly improves the startup speed of the desktop component and the user's visual experience.

[0154] In some embodiments, the processor 810 can also be used for: After running the desktop component installation package and rendering the desktop component, it receives modification instructions from the user. Update desktop components in response to modification commands.

[0155] This allows the system to receive user input for modification and respond by updating desktop components, enabling the generated desktop components to be dynamically adjusted according to user needs. This enhances the flexibility and customizability of desktop components and meets the ever-changing user requirements.

[0156] It should be understood that, in this embodiment, the input unit 804 may include a graphics processing unit (GPU) 8041 and a microphone 8042. The GPU 8041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 806 may include a display panel 8061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 807 includes at least one of a touch panel 8071 and other input devices 8072. The touch panel 8071 is also called a touch screen. The touch panel 8071 may include a touch detection device and a touch controller. Other input devices 8072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0157] The memory 809 can be used to store software programs and various data. The memory 809 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 809 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 809 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0158] Processor 810 may include one or more processing units; optionally, processor 810 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 810.

[0159] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the desktop component generation method embodiments described above and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0160] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0161] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described desktop component generation method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0162] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0163] This application provides a computer program product that is stored in a storage medium and executed by at least one processor to implement the various processes of the desktop component generation method embodiment described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0164] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0165] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0166] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for generating desktop components, characterized in that, include: Based on the collected user multimodal demand information and desktop operating environment constraint information, constraint rules are generated; Based on the aforementioned constraint rules, the description code and manifest incremental patches corresponding to the desktop components are generated iteratively. Based on the description code and the inventory incremental patch, generate a desktop component installation package; Run the desktop component installation package and render and generate the desktop component.

2. The method according to claim 1, characterized in that, The step of iteratively generating description code and manifest incremental patches corresponding to desktop components based on the constraint rules includes: Construct a virtual tool runtime environment; the virtual tool runtime environment includes a read-only knowledge base and a read-write memory workspace; In the virtual tool runtime environment, based on the constraint rules and the read-only knowledge base, the virtual tool performs multiple rounds of iterative generation processing in the read-write memory workspace to obtain the description code and manifest incremental patch corresponding to the desktop component; The multi-round iterative generation process includes context assembly executed by round, model streaming output parsing, incremental merging of virtual tool calls, execution of virtual tools, and backfeeding of virtual tool results.

3. The method according to claim 1, characterized in that, Before generating the desktop component installation package based on the description code and the manifest incremental patch, the method further includes: The description code and the manifest incremental patch are verified using preset verification rules and restricted container runtime verification rules to obtain verification results; If the verification result indicates that the verification failed, a structured verification error message is output; the structured verification error message includes the verification rule code, error description information, and repair prompts; Based on the structured validation error information, the description code and the manifest incremental patch are partially repaired to obtain the repaired description code and manifest incremental patch.

4. The method according to claim 3, characterized in that, The step of generating a desktop component installation package based on the description code and the manifest incremental patch includes: If the verification result indicates that the verification passed, a unique installation package identifier, resource path, and version code are generated; The desktop component installation package is generated based on the unique installation package identifier, the resource path, the version code, the description code, and the manifest incremental patch.

5. The method according to claim 1, characterized in that, The step of running the desktop component installation package and rendering and generating the desktop component includes: The interface for adding desktop cards, registered by the native system, is invoked through a communication bridge layer between the web page and the native system. In response to the call to the Add Desktop Card interface, an Add Desktop task is generated based on the desktop component installation package; The desktop task is sent to the desktop host via the operating system's inter-process communication interface, and the transparent agent activity page in the desktop host is explicitly launched to write the configuration to be applied into the desktop host. The desktop host determines the rendering parameters based on the configuration to be activated, and performs rendering based on the rendering parameters to obtain the desktop component; The configuration to be activated is determined based on the addition of desktop tasks.

6. The method according to claim 5, characterized in that, Before calling the add desktop card interface registered by the native system through the communication bridge layer between the web page and the native system, the method further includes: Based on the description code and the manifest incremental patch, a static preview resource is generated, and the preview window address corresponding to the static preview resource is returned.

7. The method according to claim 1, characterized in that, After running the desktop component installation package and rendering the desktop component, the method further includes: Save a rendering snapshot of the desktop component; wherein the rendering snapshot is used to preview and display the desktop component after it is generated and before it is added to the desktop.

8. The method according to claim 1, characterized in that, After running the desktop component installation package and rendering the desktop component, the method further includes: Receive modification instructions from the user; In response to the modification command, update the desktop component.

9. A desktop component generation device, characterized in that, include: The first generation module is used to generate constraint rules based on the collected user multimodal demand information and desktop operating environment constraint information; The second generation module is used to iteratively generate description code and manifest incremental patches corresponding to the desktop components based on the constraint rules. The third generation module is used to generate a desktop component installation package based on the description code and the manifest incremental patch; The fourth generation module is used to run the desktop component installation package and render and generate the desktop component.

10. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the method as described in any one of claims 1-8.

Images