Image processing method, device, storage medium and program product

Abstract

The application discloses an image processing method and device, a storage medium and a program product, and belongs to the technical field of terminals. The method comprises the following steps: acquiring an image editing instruction when a first interface of an intelligent voice assistant is displayed, the image editing instruction corresponding to at least two tasks; acquiring a first image; displaying a second interface of the intelligent voice assistant, displaying a first prompt message in the second interface when each task in the at least two tasks is executed, and the first prompt message being used for reminding the identity of the task being executed and the task progress; and displaying a first reply and a second image in the second interface after the at least two tasks are all executed. The first reply is used for indicating the processing result of the first image. The second image is an image obtained by processing the first image through at least one task. The application can not only improve the image editing efficiency and the user experience, but also make the image editing process more intuitive and natural, and can meet the demand for fast and accurate image editing.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an image processing method, device, storage medium, and program product. Background Technology

[0002] In everyday photography, users often encounter unsatisfactory photos, such as those containing unwanted elements like passersby or clutter. To improve the aesthetics, users typically edit their photos. For example, to remove unwanted elements, users can open a dedicated image editing application, select the photo, and manually remove the elements by selecting or smearing. However, manual removal requires a certain level of skill and patience, especially when dealing with complex backgrounds or images rich in detail, where inaccurate removal is common. Furthermore, manual removal is less efficient for smaller or more complex objects. Therefore, while manual removal can meet user needs to some extent, it still has significant shortcomings in terms of ease of use and ability to handle complex scenes. Summary of the Invention

[0003] This application provides an image processing method, apparatus, storage medium, and program product, which can meet the needs for fast and accurate image editing. The technical solution is as follows:

[0004] In a first aspect, an image processing method is provided. In this method, an electronic device, while displaying a first interface of a smart voice assistant, acquires an image editing instruction corresponding to at least two tasks. In response to the image editing instruction, a first image is acquired. Subsequently, a second interface of the smart voice assistant is displayed. During the execution of each of the at least two tasks, a first reminder message is displayed on the second interface, indicating the identifier and progress of the currently executing task. After both tasks have been completed, a first response and a second image are displayed on the second interface. The first response indicates the processing result of the first image. The second image is an image obtained by processing the first image through at least one of the at least two tasks.

[0005] Understandably, the image described in this application may exist in the electronic device as image data before being displayed; that is, the image may be transmitted and processed internally by the electronic device as image data. When displayed, the electronic device can present the image on the screen based on the image data.

[0006] Furthermore, the image to be displayed in this application can be the original image or a thumbnail of the image, depending on the specific requirements of the scenario. That is, displaying an image as described in this application may refer to displaying the original image or a thumbnail of the image; in some cases, displaying a thumbnail of an image may also refer to displaying the original image.

[0007] In this application, the electronic device can obtain image editing instructions through a smart voice assistant, and then execute at least two tasks according to the instructions, thereby improving image processing efficiency. During task execution, a first reminder message can be displayed through the smart voice assistant. The first reminder message can indicate the identity and progress of the task being executed, allowing users to be promptly informed of the current image processing status during complex image editing using the smart voice assistant. This not only improves system transparency but also alleviates user anxiety. After the at least two tasks are completed, the electronic device can also display a first response and a second image through the smart voice assistant, enhancing the smoothness of user interaction. The image editing function provided by the smart voice assistant in this application not only improves image editing efficiency and user experience but also makes the image editing process more intuitive and natural, thus meeting the needs for fast and accurate image editing in various complex scenarios.

[0008] In one possible approach, the at least two tasks include at least two of the following: elimination task, skin smoothing task, background removal task, ID photo generation task, element addition task, and basic editing task.

[0009] Removal tasks can include pedestrian removal tasks, intelligent removal tasks, etc. Pedestrian removal tasks are used to remove pedestrians from an image. Intelligent removal tasks are used to remove specific targets from an image. Skin smoothing tasks are used to smooth the skin of people in an image. Background removal tasks are used to separate the subject from an image. ID photo generation tasks are used to generate ID photos of people in an image. Element addition tasks are used to add specific elements to an image. Basic editing tasks are used to perform common image editing, such as cropping, applying filters, mosaic effects, and background blurring.

[0010] In this application, a variety of processing of the first image can be achieved through an intelligent voice assistant, thereby significantly improving image editing efficiency and user experience.

[0011] In one possible approach, the electronic device can display a large image page of a gallery application, on which the first image is displayed; an input box for a smart voice assistant is also displayed on the large image page, and the first interface is an interface displaying this input box. In response to the image editing instruction, the electronic device can retrieve the first image by: retrieving the first image from its local storage address in response to the image editing instruction.

[0012] In this application, users can decide whether to process images while viewing them in the gallery application, and instantly initiate image processing functions via a smart voice assistant when such processing is needed. This improves ease of use and makes image processing smoother and more intuitive.

[0013] In one possible approach, the electronic device displays a third interface showing a first thumbnail of the first image. In response to receiving an operation applied to the first thumbnail of the first image, a target element is displayed, displaying the content of the first image. In response to detecting a drag operation on the target element, an identifier (such as an icon) of the intelligent voice assistant is displayed. In response to receiving an operation where the target element is dragged to the location of the intelligent voice assistant's identifier and released, a chat window of the intelligent voice assistant is displayed, showing a second thumbnail of the first image, with the first interface being the interface displaying this chat window. The electronic device's operation of acquiring the first image in response to the image editing instruction can be: in response to the image editing instruction, acquiring the first image according to its local storage address.

[0014] In this application, users can quickly transfer images from any interface to the intelligent voice assistant for processing. This improves operational flexibility and convenience.

[0015] In one possible approach, the first interface displays one or more AI editing controls. Before acquiring an image editing instruction, the electronic device receives an operation applied to any one of the AI ​​editing controls. The operation by which the electronic device acquires the image editing instruction can be: acquiring the image editing instruction based on the received operation applied to the AI ​​editing control. The user instruction corresponding to the AI ​​editing control is the image editing instruction.

[0016] In this embodiment, users can quickly determine which image processing functions the intelligent voice assistant can display through one or more displayed AI editing controls. Furthermore, users can quickly trigger the corresponding image processing using any of the AI ​​editing controls. This reduces operational complexity, improves image processing efficiency, and enhances the user experience.

[0017] In one possible approach, the first notification message includes at least two status controls, each corresponding to one of the at least two tasks. Each status control indicates the task status of its corresponding task, which may include one or more of the following: not started, in progress, successful, or failed. This provides a more intuitive way to remind the user of the task status of each task.

[0018] In one possible approach, after the electronic device displays the first response and the second image on the second interface, for any one of the at least two status controls, if the task status indicated by the status control is successful, in response to receiving an operation performed on the status control, the second interface displays the image obtained through the task processing corresponding to the status control.

[0019] In this application, the user can selectively view the process diagrams obtained through various task processing using at least two status controls. This improves the user experience.

[0020] In one possible approach, the operation of the electronic device displaying the first reply and the second image on the second interface can be: displaying the first reply, the second image, and AI image editing controls on the second interface.

[0021] In response to receiving an operation applied to the AI ​​image editing control, the electronic device displays multiple AI editing controls on a second interface. For any one of these AI editing controls, in response to receiving an operation applied to that AI editing control, a second response and a third image are displayed on the second interface. The second response indicates the processing result of the second image. The third image is the image obtained after processing the second image according to the user instruction corresponding to the AI ​​editing control.

[0022] In this application, users can quickly determine what processing can be performed on a second image through the AI ​​image editing control. This improves ease of use and user efficiency.

[0023] In one possible approach, after the electronic device displays a first response and a second image on a second interface, in response to receiving an operation applied to the first image, it displays an editing interface for the first image, which includes AI image editing controls. In response to receiving an operation applied to the AI ​​image editing controls, multiple AI editing controls are displayed on the editing interface. For any one of these multiple AI editing controls, in response to receiving an operation applied to that AI editing control, the second interface is displayed, showing a third response and a fourth image. The third response indicates the processing result of the first image. The fourth image is the image obtained after processing the first image according to the user instruction corresponding to the AI ​​editing control.

[0024] In this application, users can access the editing interface of the first image by manipulating it. Within this editing interface, users can quickly learn about various operations that can be performed on the first image and initiate processing of the first image according to their needs. This improves ease of operation and enhances the user experience.

[0025] In one possible approach, after the electronic device acquires the first image, the intelligent voice assistant sends the image editing instruction and the storage address of the first image to the target intelligent agent. Upon receiving the image editing instruction and the storage address of the first image, the target intelligent agent retrieves at least two tasks based on the image editing instruction. The target intelligent agent executes the at least two tasks based on the storage address of the first image, obtaining the task execution result and a second image. The target intelligent agent obtains a first response based on the task execution result. The target intelligent agent then sends the first response and the second image to the intelligent voice assistant.

[0026] In this application, the intelligent voice assistant can process the first image and respond to dialogue with the help of the target intelligent agent. This division of labor and cooperation can improve image processing efficiency and make the overall processing flow smoother.

[0027] In one possible approach, during the execution of the at least two tasks, the target intelligent agent sends an instruction message to the intelligent voice assistant to indicate task nodes, which indicate whether the task has started, succeeded, or failed. The electronic device displays the first reminder message on the second interface as follows: after receiving the instruction message, the intelligent voice assistant displays the first reminder message based on the task node.

[0028] In this application, by displaying the first notification message on the second interface, users can understand the task's execution progress in real time. For example, the status of the task starting, succeeding, or failing can be promptly fed back to the user. This transparent status update helps users control the dynamics of the entire task execution, avoiding unnecessary confusion or anxiety during the task execution process.

[0029] In one possible approach, the storage address of the first image includes both a local storage address and a cloud storage address. Before sending the image editing instructions and the storage address of the first image to the target intelligent agent, the intelligent voice assistant sends the first image to a first cloud service. The intelligent voice assistant receives the cloud storage address sent by the first cloud service. This cloud storage address is the storage address of the first image within the first cloud service.

[0030] In this application, by storing the first image in a first cloud service in advance, the first image can be directly obtained from the first cloud service when image processing is required later, without having to repeatedly upload it from the local machine to the cloud, thereby saving communication resources and improving processing efficiency.

[0031] In one possible approach, before the intelligent voice assistant sends the image editing instruction and the storage address of the first image to the target intelligent agent, if the image editing instruction is triggered by an AI editing control, the intelligent voice assistant obtains the intelligent agent identifier corresponding to the AI ​​editing control, and the intelligent agent identifier corresponding to the AI ​​editing control is the identifier of the target intelligent agent; or, the intelligent voice assistant sends the image editing instruction to a second cloud service and receives the identifier of the target intelligent agent sent by the second cloud service.

[0032] In this application, when a user triggers the image editing command through an AI editing control, the target intelligent agent can be directly determined based on the AI ​​editing control without parsing the command. This reduces the user's waiting time. Even when the user does not trigger the image editing command through the AI ​​editing control, a second cloud service can accurately determine the target intelligent agent capable of processing the command.

[0033] In one possible approach, the operation of the target agent to obtain the at least two tasks according to the image editing instruction can be as follows: the target agent sends a task orchestration request to a third cloud service, the task orchestration request including the image editing instruction; the target agent receives the at least two tasks sent by the third cloud service.

[0034] In this application, the target intelligent agent can accurately orchestrate the task of editing the image through a third cloud service, thereby helping to accurately process the first image in the future and meet the user's image processing needs.

[0035] In one possible approach, the target agent includes a task engine and one or more task tools. The operation of the target agent to perform the at least two tasks based on the storage address of the first image can be as follows: for the first task among the at least two tasks, the task engine calls the task tool corresponding to the first task to execute the first task based on the storage address of the first image; for any other task among the at least two tasks besides the first task, the task engine calls the task tool corresponding to the task to execute the task based on the image obtained from the task executed before that task.

[0036] In this application, the task engine can sequentially call the task tools corresponding to each of the multiple tasks, so that each task tool can execute its corresponding task. This multi-task execution method can meet the image processing needs of complex user commands.

[0037] In one possible approach, the storage address of the first image includes both local storage and cloud storage. The first task is a pedestrian removal task. The corresponding task tool for this pedestrian removal task is a removal tool. The operation of the task engine calling the task tool corresponding to the first task to execute the first task based on the storage address of the first image can be as follows: the task engine sends the pedestrian removal task and the storage address of the first image to the removal tool. After receiving the pedestrian removal task and the storage address of the first image, the removal tool sends a pedestrian recognition request to the visual platform. This pedestrian recognition request includes the pedestrian removal task and the cloud storage address. After receiving the pedestrian recognition request, the visual platform sends the pedestrian recognition request to the fourth cloud service; after receiving the mask image of the pedestrian in the first image sent by the fourth cloud service, it sends the mask image to the removal tool. After receiving the mask image, the removal tool sends a pedestrian removal request to the visual platform. This pedestrian removal request includes the local storage address and the mask image. After receiving the pedestrian removal request, the visual platform retrieves the first image based on the local storage address, extracts the pedestrian image from the first image based on the mask image, and sends a pedestrian removal request including the pedestrian image to the fourth cloud service. Upon receiving the fifth image from the fourth cloud service, it merges the fifth image with the first image based on the mask image to obtain a sixth image, which is then sent to the removal tool. The removal tool, upon receiving the sixth image, sends it to the task engine.

[0038] In this application, during pedestrian identification, the powerful processing capabilities of the cloud side enable efficient and accurate identification of pedestrians in the first image. Furthermore, during pedestrian removal, only a portion of the first image containing the pedestrian (i.e., the pedestrian image) is sent to the cloud side for processing, thereby saving transmission resources, reducing the processing pressure on the cloud side, and increasing processing speed. This end-to-cloud collaborative approach effectively completes the pedestrian removal task, improving the quality and efficiency of image processing.

[0039] In one possible approach, the task engine's operation of calling the corresponding task tool to execute the first task based on the storage address of the first image can be as follows: The task engine sends the first task to the corresponding task tool. Upon receiving the first task, if a person's name exists in the first task, the corresponding task tool sends a first query request to the learning and memory platform, including the person's name; if it receives a person's identifier from the learning and memory platform, it sends a second query request to the image library application, including the person's identifier; if it receives a target face bounding box from the image library application, it sends a person recognition request to the visual platform, including the cloud storage address of the first image and the target face bounding box. Upon receiving the person recognition request, the visual platform sends it to a fourth cloud service; upon receiving the target face bounding box from the fourth cloud service, it sends the target face bounding box to the corresponding task tool. The corresponding task tool then executes the first task based on the target face bounding box.

[0040] In this application, by associating person titles with person identifiers, the specific person corresponding to the person title in the first image can be quickly and accurately identified locally without user clarification. This allows for the precise and automated completion of the entire task, reducing the complexity of user operations and improving image processing efficiency.

[0041] In one possible approach, after the task tool corresponding to the first task sends a first query request to the learning and memory platform, upon receiving a query failure message from the learning and memory platform, the task tool sends the query failure message to the task engine. Upon receiving the query failure message, the task engine retrieves a fourth response based on the message, which is used to inquire about the person's name in the first image; it then sends the fourth response to the intelligent voice assistant. Upon receiving the fourth response, the intelligent voice assistant displays it on a second interface; upon receiving a user statement entered on the second interface, it sends the user statement to the task engine. Upon receiving the user statement, the task engine generates a person description based on the fourth response and the user statement, and sends this description to the task tool corresponding to the first task. Upon receiving the person description, the task tool sends a person recognition request to the visual platform, which includes the cloud storage address of the first image and the person description. Upon receiving the person recognition request, the visual platform sends it to a fourth cloud service; upon receiving a target human bounding box from the fourth cloud service, it sends the target human bounding box to the task tool corresponding to the first task. The task tool corresponding to the first task executes the first task based on the target human body outline.

[0042] In this application, for tasks involving the naming of individuals, if the specific individual cannot be found locally, a fourth response can be used to proactively interact with the user and inquire about the specific characteristics of that individual. This not only enhances the interactivity between the user and the system but also helps the system accurately understand user needs, thereby ensuring the correct execution of the task to a certain extent.

[0043] In a second aspect, an image processing apparatus is provided, which has the function of implementing the image processing method described in the first aspect. The image processing apparatus includes at least one module for implementing the image processing method provided in the first aspect.

[0044] Thirdly, an electronic device is provided, comprising: one or more processors, and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code including computer instructions, the one or more processors calling the computer instructions to cause the electronic device to perform the image processing method provided in the first aspect.

[0045] Fourthly, a chip system is provided for use in an electronic device, the chip system including one or more processors for invoking computer instructions to cause the electronic device to perform the image processing method provided in the first aspect.

[0046] Fifthly, a computer-readable storage medium is provided, the computer-readable storage medium including instructions that, when executed on an electronic device, cause the electronic device to perform the image processing method provided in the first aspect.

[0047] In a sixth aspect, a computer program product is provided that, when the computer program product is run on an electronic device, causes the electronic device to perform the image processing method provided in the first aspect.

[0048] The technical effects achieved by the second, third, fourth, fifth, and sixth aspects mentioned above are similar to those achieved by the corresponding technical means in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the interface related to an AI editing function provided in an embodiment of this application;

[0050] Figure 2 This is a schematic diagram of the interface related to another AI editing function provided in the embodiments of this application;

[0051] Figure 3This is a schematic diagram of the interface related to another AI editing function provided in the embodiments of this application;

[0052] Figure 4 This is a schematic diagram of the interface related to another AI editing function provided in the embodiments of this application;

[0053] Figure 5 This is a schematic diagram of the interface related to another AI editing function provided in the embodiments of this application;

[0054] Figure 6 This is a schematic diagram of the interface related to another AI editing function provided in the embodiments of this application;

[0055] Figure 7 This is a schematic diagram of the interface related to another AI editing function provided in the embodiments of this application;

[0056] Figure 8 This is a schematic diagram of the interface related to another AI editing function provided in the embodiments of this application;

[0057] Figure 9 This is a schematic diagram of the interface related to another AI editing function provided in the embodiments of this application;

[0058] Figure 10 This is a schematic diagram of a smart voice assistant message provided in an embodiment of this application;

[0059] Figure 11 This is a schematic diagram of another intelligent voice assistant message provided in an embodiment of this application;

[0060] Figure 12 This is a schematic diagram of another intelligent voice assistant message provided in an embodiment of this application;

[0061] Figure 13 This is a schematic diagram of the interface related to another AI editing function provided in the embodiments of this application;

[0062] Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0063] Figure 15 This is a block diagram of a software system for an electronic device provided in an embodiment of this application;

[0064] Figure 16 This is a schematic diagram of an end-to-cloud collaborative architecture provided by an embodiment of this application;

[0065] Figure 17 This is a flowchart of an image processing method provided in an embodiment of this application;

[0066] Figure 18This is a flowchart of a task execution process provided in an embodiment of this application;

[0067] Figure 19 This is a flowchart of another task execution process provided by an embodiment of this application;

[0068] Figure 20 This is a flowchart of another task execution process provided by an embodiment of this application;

[0069] Figure 21 This is a flowchart of another task execution process provided by an embodiment of this application;

[0070] Figure 22 This is a flowchart of another task execution process provided in the embodiments of this application. Detailed Implementation

[0071] In the following description, specific details such as particular system architectures and technologies are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details.

[0072] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0073] It should be understood that "one or more" as mentioned in this application refers to one, two, or more, and "multiple" as mentioned in this application refers to two or more. In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0074] To facilitate a clear description of the technical solutions of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.

[0075] The terms "one embodiment" or "some embodiments" used in this application mean that one or more embodiments of this application include the specific features, structures, or characteristics described in that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this application do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0076] With the widespread use of mobile phones, tablets, and other electronic devices, taking photos has become increasingly convenient. However, when using these devices, unwanted elements are often captured, such as passersby, clutter, or other distracting objects, which can severely detract from the aesthetics of the photo. In such cases, users typically need to remove these unwanted elements. Traditional removal methods require users to open a gallery application and manually select or paint over the object to be removed in the image display interface. This manual method is not only time-consuming and laborious but also demands a high level of skill and patience from the user. Especially when multiple elements need to be removed or the photo background is complex, manual removal becomes even more cumbersome, significantly impacting the user experience.

[0077] Therefore, this application provides an image processing method to achieve artificial intelligence (AI) editing functions for images. In this method, users can use a smart voice assistant to perform image editing through conversational interaction, such as removing passersby or applying skin smoothing effects with a single click. This significantly improves image editing efficiency and user experience. Furthermore, the introduction of the smart voice assistant makes the operation process more intuitive and natural, lowering the technical barrier to image editing and allowing ordinary users to easily get started, thus meeting the needs for fast and accurate image editing in various complex scenarios. Optionally, the smart voice assistant can be called "YOYO".

[0078] Next, combine Figures 1 to 13 The relevant interfaces of the AI ​​editing function provided in the embodiments of this application are described.

[0079] In this application embodiment, the image processing function of the intelligent voice assistant can be triggered in various ways, and the following will be combined with Figures 1 to 5 This is illustrated by example:

[0080] The following is combined Figure 1 The first method is illustrated by example:

[0081] like Figure 1As shown in (a), the electronic device displays interface 21, which can display icons for multiple applications, assuming it includes an icon 211 for the gallery application. Optionally, interface 21 can be referred to as the main interface or desktop. Subsequently, the electronic device receives an operation applied to the icon 211 of the gallery application in interface 21, and in response to this operation, as shown in (a). Figure 1 As shown in (b), the electronic device can display an interface 22 of a gallery application. Interface 22 can display thumbnails of each of multiple images, assuming it includes thumbnail 181 of image a. Subsequently, the electronic device receives an operation applied to thumbnail 181 of image a, and in response to this operation, as shown in (b). Figure 1 As shown in (c), the electronic device can display an interface 23 of a gallery application. Interface 23 can display image a, as well as multiple controls. For example, these multiple controls may include share, favorite, AI edit, delete, more, etc. Then, the electronic device receives an operation applied to image a and responds to that operation as follows: Figure 1 As shown in (d), the electronic device can display the interface 24 of the gallery application. The interface 24 can display a full-screen view of image a. Optionally, the interface 24 can be referred to as a large page of image a. For example, in this embodiment, the operation received by the electronic device acting on the content displayed on the electronic device (such as icons, thumbnails, images, controls, or options) can be a voice operation, a gesture operation (such as clicking, long-pressing, or swiping), or a motion-sensing operation (such as device shaking or air gestures), etc., and this embodiment does not limit this.

[0082] like Figure 1As shown in (d) above, in some embodiments, when the electronic device displays interface 24, it can automatically display the input box 241 of the intelligent voice assistant and multiple AI editing controls on interface 24. In other embodiments, the user can wake up the intelligent voice assistant while the electronic device displays interface 24. After the intelligent voice assistant is woken up, the electronic device can display the input box 241 of the intelligent voice assistant and multiple AI editing controls on interface 24. For example, the user can wake up the intelligent voice assistant using any of the following methods: voice wake-up, breath wake-up, button wake-up, navigation bar wake-up, etc. For example, the multiple AI editing controls may include a pedestrian removal control 242, a one-click skin beautification control 243, and an intelligent removal control 244. The pedestrian removal control 242 is used to trigger the removal of pedestrians in the image. The one-click skin beautification control 243 is used to trigger skin beautification of people in the image. The intelligent removal control 244 is used to trigger the removal of specific targets in the image. Specific targets may include one or more of people, animals, plants, objects, text, watermarks, etc. In this application's embodiments, "passersby" can refer to other figures appearing in the image besides the main subject, such as pedestrians in the background, people passing by, or other individuals in the scene who do not directly interact with the main subject. The main subject typically refers to the central figure in the image, usually located in the center or a prominent position within the image.

[0083] In this scenario, if a user wants to remove passersby from image a, they can operate the "Remove Passersby" control 242. Upon receiving the operation on the "Remove Passersby" control 242, the electronic device can, in response to this operation, perform passersby removal processing on image a, and can display the chat window 26 of the intelligent voice assistant. For example, as... Figure 1 As shown in (e), the chat window 26 can be displayed on the interface 24 in a floating window or similar manner, and the interface 24 and the chat window 26 together constitute the interface 25. When the chat window 26 is displayed on the interface 24, the chat window 26 does not completely cover the interface 24. The chat window 26 may include an input box 261 and a dialogue area 262. Optionally, during the process of removing pedestrians from image a, the electronic device may display the user message "Remove pedestrians" and the intelligent voice assistant message "Image generation in progress" in the dialogue area 262 to remind the user that pedestrians in image a are being removed.

[0084] Assuming the electronic device obtains image b after performing pedestrian removal processing on image a, the electronic device can display a thumbnail of image b in dialog area 262. For example, as... Figure 1As shown in (e), after completing the pedestrian removal process for image a, the electronic device may no longer display the intelligent voice assistant message "Image generation in progress," but instead display the intelligent voice assistant messages "Completed, you can save directly or perform custom editing" and "Thumbnail of image b." Optionally, as... Figure 1 As shown in (e), the electronic device can display multiple controls while displaying a thumbnail of image b, such as a sharing control 263, a save control 264, a custom editing control 265, and an AI image editing control 266. The sharing control 263 is used to share image b. The save control 264 is used to save image b locally. The custom editing control 265 is used for custom editing. The AI ​​image editing control 266 is used for AI image editing of image b; the specific functions of the AI ​​image editing control 266 can be found below. Figure 7 The relevant descriptions in the implementation method will not be elaborated here. Users can manipulate image d through these controls.

[0085] The above text Figure 1 In the first implementation method, users can decide whether image processing is needed while viewing images in the gallery application, and instantly initiate image processing functions via a smart voice assistant when image processing is required. This improves ease of operation and makes image processing smoother and more intuitive.

[0086] The following is combined Figure 2 and Figure 3 The second method is illustrated by example:

[0087] like Figure 2 As shown in (a), the electronic device displays interface 31, which can be any interface displaying an image. Here, it is assumed that interface 31 is a chat interface of an instant messaging application, where the user and / or their friend sends image c to each other. Subsequently, the electronic device receives an operation applied to image c (such as a long press), and in response to this operation, the electronic device can display a thumbnail 321 of image c (also referred to as the target element). For example, as... Figure 2 As shown in (b), thumbnail 321 can be displayed on interface 31, and interface 31 and thumbnail 321 together constitute interface 32. When thumbnail 321 is displayed on interface 31, it can obscure the content at the corresponding position in interface 31. The user can drag thumbnail 321, and thumbnail 321 can move within interface 32 as the user drags it.

[0088] After that, as Figure 2As shown in (c), when the electronic device detects that the thumbnail 321 has been dragged to near the edge of the screen, it can display interface 33. Interface 33 can display the thumbnail 321, screen 331, and service recommendation bar 332. Optionally, screen 331 can display the content of the scaled-down interface 31. For example, screen 331 is used to create the visual effect of interface 31 being pushed inward on the screen. Service recommendation bar 332 is used to display icons of one or more services, which may include the icon 333 of the intelligent voice assistant. Users can drag the thumbnail 321 to the position of an icon in service recommendation bar 332 according to their own needs and then release it to transmit image c to the service corresponding to the icon and start the corresponding service function. It should be noted that the service described in this application embodiment is a program used to implement a specific function, such as an application, applet, quick app (also known as an installation-free application), or certain functional modules of an application, etc., and this application embodiment does not limit this.

[0089] Subsequently, after detecting that thumbnail 321 has been dragged to and released from the location of the smart voice assistant icon 333, the electronic device can transmit image c to the smart voice assistant and display the smart voice assistant's chat window 35. For example, as... Figure 3 As shown in (a), the chat window 35 can be displayed on the interface 31 in a floating window or similar manner, and the interface 31 and the chat window 35 together constitute the interface 34. When the chat window 35 is displayed on the interface 31, the chat window 35 does not completely cover the interface 31. The chat window 35 may include an input box 351, multiple AI editing controls, and a dialogue area 352. The multiple AI editing controls may include a passerby removal control 353, a one-click skin beautification control 354, a smart removal control 355, etc. The functions of the multiple AI editing controls have been described above. Figure 1 The implementation method will be described in detail here. The user message "thumbnail of image c" can be displayed in the dialog area 352.

[0090] In this scenario, if a user wants to remove pedestrians from image c, they can operate the pedestrian removal control 353. Upon receiving the operation on the pedestrian removal control 353, the electronic device can perform pedestrian removal processing on image c in response to the operation. Optionally, during the pedestrian removal processing of image c, the electronic device can display the user message "Remove Pedestrians" and the intelligent voice assistant message "Image Generating" in the dialog area 352 to remind the user that pedestrians are being removed from image c.

[0091] Assuming the electronic device obtains image d after performing pedestrian removal processing on image c, the electronic device can display a thumbnail of image d in dialog area 352. For example, as... Figure 3As shown in (b), after completing the pedestrian removal process for image c, the electronic device may no longer display the intelligent voice assistant message "Image generation in progress," but instead display the intelligent voice assistant message "Completed, you can save directly or perform custom editing" and "Thumbnail of image d." Optionally, as... Figure 3 As shown in (b) above, the electronic device can display multiple controls simultaneously with the thumbnail of image d, such as a share control 361, a save control 362, a custom edit control 363, and an AI image editing control 364. The functions of these multiple controls have been described above. Figure 1 The implementation method will be described in detail here. Users can manipulate image d through these controls.

[0092] The above text Figures 2 to 3 In the second implementation method, users can quickly transmit images from any interface to the intelligent voice assistant for processing. This improves operational flexibility and convenience.

[0093] The following is combined Figure 4 The third method is illustrated by example:

[0094] The electronic device can display interface 41, which can be any interface. The user can wake up the intelligent voice assistant while the electronic device displays interface 41. After the intelligent voice assistant is woken up, the electronic device can display the input box 421 of the intelligent voice assistant. For example, such as... Figure 4 As shown in (a), the input box 421 can be displayed on the interface 41 in a floating window or similar manner. The interface 41 and the input box 421 can together constitute the interface 42. When the input box 421 is displayed on the interface 41, the input box 421 does not completely cover the interface 41. The input box 421 may include an add control 422 and an input mode switching control 423. The add control 422 is used to add images or files. The input mode switching control 423 is used to switch the input mode between voice input and text input.

[0095] In this scenario, if a user wants to perform image processing, they can either add the image by operating the add control 422, or by entering the user command "add image" in the input box 421 via voice or text. This will trigger the electronic device to open the interface for adding images. Here, it is assumed that the electronic device receives the operation on the add control 422 and responds to the operation as follows: Figure 4As shown in (b), the electronic device can display multiple controls in the input box 421. These controls may include an image control 424, a camera control 425, a file control 426, etc. Users can select a local image for processing using the image control 424, or capture an image for processing using the camera control 425. The following explanation uses the image control 424 as an example to illustrate how to input an image to the intelligent voice assistant. For the method of inputting an image to the intelligent voice assistant using the camera control 425, please refer to the following text. Figure 5 The relevant descriptions in the implementation methods will not be repeated here.

[0096] Suppose the electronic device receives an operation applied to the image control 424, and responds to the operation as follows: Figure 4 As shown in (c), the electronic device can display an interface 43 of a gallery application. Interface 43 can display thumbnails of each of multiple images, assuming it includes a thumbnail 431 of image e. Subsequently, the electronic device receives an operation applied to thumbnail 431 of image e, and in response to this operation, as shown in (c). Figure 4 As shown in (d), the electronic device can display interface 44. Interface 44 can display image e, as well as a return control 441 and a completion control 442. Upon receiving an operation on the completion control 442, the electronic device can, in response to this operation, display the chat window 46 of the intelligent voice assistant. For example, as... Figure 4 As shown in (e), the chat window 46 can be displayed on the interface 41 in a floating window or similar manner. The interface 41 and the chat window 46 together constitute the interface 45. When the chat window 46 is displayed on the interface 41, the chat window 46 does not completely cover the interface 41. The chat window 46 may include an input box 461, multiple AI editing controls, and a dialogue area 462. The multiple AI editing controls may include a passerby removal control 463, a one-click skin beautification control 464, and a smart removal control 465, etc. The functions of the multiple AI editing controls have been described above. Figure 1 The implementation method will be described in detail here. The user message "thumbnail of image e" can be displayed in the dialog area 462.

[0097] After that, as Figure 4 As shown in (e) to (f), the user can use the intelligent voice assistant to remove pedestrians in image e to obtain image f. Figure 4 The processes shown in (e) to (f) are similar to Figure 3 The processes shown in (a) to (b) are similar; please refer to the above text for details. Figure 3 The relevant descriptions in the implementation methods will not be repeated here.

[0098] The above text Figure 4In the third implementation method, users can activate the intelligent voice assistant from any interface. The intelligent voice assistant serves as a centralized operation window, allowing users to easily add and process images using its provided controls. This simple process improves user efficiency.

[0099] The following is combined Figure 5 The fourth method is illustrated by example:

[0100] The electronic device can display interface 51, which can be any interface. The user can wake up the intelligent voice assistant while the electronic device displays interface 51. After the intelligent voice assistant is woken up, the electronic device can display the input box 521 of the intelligent voice assistant. For example, such as... Figure 5 As shown in (a), the input box 521 can be displayed on the interface 51 in the form of a floating window or the like, and the interface 51 and the input box 521 can together constitute the interface 52. When the input box 521 is displayed on the interface 51, the input box 521 does not completely cover the interface 51.

[0101] The electronic device receives an operation applied to the input field 521 (such as a swipe-up operation starting from the input field 521), and responds to the operation as follows: Figure 5 As shown in (b), the electronic device can display an interface 53 for a smart voice assistant. Interface 53 can display an input box 531 and multiple options, including a YOYO Assistant option 532, recent options 533, and an AI agent option 534. The YOYO Assistant option 532 displays the dialogue page of the smart voice assistant. The recent options 533 display the AI ​​agents (artificial intelligence agents) recently used by the user. The AI ​​agent option 534 displays one or more AI agents that the user can use. An AI agent is a program with autonomous perception, decision-making, and execution capabilities, which can perform specific tasks through intent understanding, task decomposition, and resource scheduling.

[0102] Suppose the electronic device receives an operation on agent option 534, and in response to that operation, such as Figure 5 As shown in (c), the electronic device can display interface 54. Interface 54 can display multiple agent icons, which may include an icon 541 for an image editing agent, an icon 542 for a video editing agent, etc. The image editing agent is used to implement functions related to image editing. The video editing agent is used to implement functions related to video editing. The user can select the desired agent from the agents provided in interface 54. Optionally, as... Figure 5 As shown in (c), a search box 543 can also be displayed in the interface 54, through which the user can search for the desired intelligent agent.

[0103] Suppose the electronic device receives an operation on icon 541 applied to the image editing agent, and in response to that operation, such as Figure 5 As shown in (d), the electronic device can display the interface 55 of the image editing intelligent agent. Interface 55 can be a chat window for an intelligent voice assistant, displaying an input box 551, multiple AI editing controls, and a dialogue area 552. These multiple AI editing controls may include a pedestrian removal control 553, a smart removal control 554, a one-click skin beautification control 555, a smart background removal control 556, and a photo ID generation control 557. The smart background removal control 555 is used to trigger the separation of the subject (such as a person, animal, or object) in the image. The photo ID generation control 557 is used to trigger the generation of a photo ID for the person in the image. The functions of the pedestrian removal control 553, the smart removal control 554, and the one-click skin beautification control 555 have been described above. Figure 1 The implementation methods will be described in detail here.

[0104] In this situation, if the user wants to perform image processing, they can operate the pedestrian removal control 553. The electronic device receives the operation applied to the pedestrian removal control 553 and responds to the operation as follows: Figure 5 As shown in (e), the electronic device can display the user message "Remove passersby," the intelligent voice assistant message "Please upload a picture containing people," a camera control 558, and an image control 559 in the dialogue area 552. The user can take a picture using the camera control 558 and process it, or select a local image using the image control 559 and process it. The following explanation uses the method of inputting an image to the intelligent voice assistant via the camera control 558 as an example. For the method of inputting an image to the intelligent voice assistant via the image control 559, please refer to the above text. Figure 4 The relevant descriptions in the implementation methods will not be repeated here.

[0105] Assuming the electronic device receives an operation applied to the camera control 558, it responds to the operation as follows: Figure 5 As shown in (f), the electronic device can display the camera application interface 56. Interface 56 is a shooting interface, which may include a shooting preview window 561 and a shooting control 562. After the user determines that the shooting screen in the shooting preview window 561 meets their needs, they can operate the shooting control 562. Upon receiving the operation applied to the shooting control 562, the electronic device, in response to the operation, can take a picture, obtaining image g, and, as shown... Figure 5As shown in (g), the electronic device can display interface 57. Interface 57 can display image g, a return control 571, and a completion control 572. Upon receiving an operation on the completion control 572, the electronic device can perform pedestrian removal processing on image g in response to this operation. Optionally, during pedestrian removal processing on image g, the electronic device can display the user messages "thumbnail of image g" and "remove pedestrians" in the dialogue area 552 of interface 55, and no longer display the photo control 558 and image control 559. Optionally, the electronic device can also display the intelligent voice assistant message "Image generation in progress" in the dialogue area 552 to remind the user that pedestrians in image g are being removed.

[0106] Assuming that the electronic device obtains image h after performing pedestrian removal processing on image g, the electronic device can display a thumbnail of image h in the dialog area 552. For example, as... Figure 5 As shown in (h), after the electronic device completes the pedestrian removal process for image g, it may no longer display the intelligent voice assistant message "Image generation in progress," but instead display the intelligent voice assistant message "Completed, you can save directly or perform custom editing" and "Thumbnail of image h." Optionally, as... Figure 5 As shown in (h), the electronic device can display multiple controls simultaneously with a thumbnail of image h, such as a share control, a save control, a custom edit control, and an AI image editing control. The functions of these multiple controls have been described above. Figure 1 The implementation method is described below and will not be repeated here. Users can manipulate image h through these controls.

[0107] The above text Figure 5 In the fourth implementation method, by providing different intelligent agents, users can select the required intelligent agent for corresponding professional processing according to their own needs. For example, users can use an image editing intelligent agent to concentrate on image processing and avoid being disturbed by other functions or applications.

[0108] One point to note is that the above text Figures 1 to 5 This embodiment uses the example of triggering pedestrian removal processing on an image via a pedestrian removal control provided by a smart voice assistant. In practical applications, other controls provided by the smart voice assistant, such as smart removal controls, one-click skin beautification controls, smart background removal controls, and ID photo generation controls, can also be used to trigger other image processing. This embodiment does not limit this approach. Furthermore, image processing can be triggered not only by controls provided by the smart voice assistant but also by entering user commands in the smart voice assistant's input box. For example, a user can enter the command "remove pedestrians" in the smart voice assistant's input box using voice or text to trigger pedestrian removal processing on the image.

[0109] Another point to note is that the above text Figures 1 to 5 The implementation scheme only uses the functions provided by the pedestrian removal control, intelligent removal control, one-click skin beautification control, intelligent background cutout control, and ID photo generation control as examples for illustrative purposes. In practical applications, intelligent voice assistants can also provide other image processing functions. For example, they can add special effects or borders to images; add specific elements to images; and provide some simple image editing functions, such as cropping, filters, mosaic, and background blurring.

[0110] The embodiments described in this application are only as stated above. Figures 1 to 5 The process described in the implementation method is used as an example to illustrate the process of triggering the image processing function of the intelligent voice assistant. Figures 1 to 5 The process described in the embodiments does not constitute a specific limitation on the implementation of the embodiments of this application. In practical applications, the image processing function of the intelligent voice assistant can also be triggered in other ways different from the four methods described above, and the embodiments of this application do not limit this.

[0111] The above provides a brief explanation of the triggering process for the image processing function of intelligent voice assistants. Next, we will combine... Figures 6 to 13 Examples of various scenarios in the image processing function of intelligent voice assistants are provided below:

[0112] The electronic device can display interface 61. Interface 61 can display images, an input box for a smart voice assistant, and multiple AI editing controls. For example, interface 61 can be as described above. Figure 1 Interface 24, shown in (d) above, can be the interface described above. Figure 3 Interface 34 shown in (a) can be the one described above. Figure 4 Interface 45 shown in (e) can be the one described above. Figure 5 The interface 55 shown in this application is not limited in this embodiment.

[0113] In one possible approach, when the electronic device displays interface 61 and the intelligent voice assistant has been activated, it can determine and display multiple AI editing controls that can be used to process the image based on the content of the image in interface 61.

[0114] For example, in the above text Figure 1In this implementation, after displaying a large image page, the electronic device can automatically activate the intelligent voice assistant and display its input box on that page, or it can display the intelligent voice assistant's input box on the large image page when the assistant is activated. Furthermore, the electronic device can recommend multiple AI editing controls to the user based on the image's content. For example, if the electronic device detects the presence of people in the image, it can display AI editing controls such as "Remove People," "One-Click Skin Beautification," and "Intelligent Removal" on the large image page. In this way, users can conveniently and quickly process the image using these AI editing controls.

[0115] For example, in the above text Figure 3 or Figure 4 In this implementation, after displaying the chat window of the intelligent voice assistant, the electronic device can recommend multiple AI editing controls to the user based on the content of the image entered by the user in the chat window.

[0116] The following uses interface 61 as an example. Figure 3 The interface 34 shown in (a) is used as an example for illustration. The operation process of other interfaces is similar and will not be described again.

[0117] It should be noted that the above text Figures 1 to 5 This embodiment uses the example of an electronic device displaying a simple intelligent voice assistant message "Image being generated" during image processing as an example for illustration. In other embodiments, the electronic device can display various intelligent voice assistant messages during image processing to remind the user of the current processing and its progress. These will be described in detail below.

[0118] In some cases, the embodiments of this application support image processing through multi-turn dialogue. In multi-turn dialogue scenarios, processing based on either the original image or the process diagram can be supported. The original image refers to the image initially provided by the user, while the process diagram refers to the image obtained after processing the original image. The following describes... Figures 6 to 8 An example is provided to illustrate a multi-turn dialogue scenario:

[0119] In some implementations, such as Figure 6 As shown in (a), the electronic device displays interface 61. Interface 61 can display a chat window 611, which may include an input box 612, multiple AI editing controls, and a dialogue area 613. The dialogue area 613 can display the user message "Thumbnail of image A".

[0120] To remove pedestrians from image A, a user can either use the pedestrian removal control provided by the smart voice assistant or enter a user command in the smart voice assistant's input box. The following example illustrates how entering the command "remove pedestrians" in the smart voice assistant's input box, triggering the pedestrian removal process in image A, will demonstrate this:

[0121] like Figure 6 As shown in (a), the electronic device receives the user command "Remove Pedestrian" entered in input box 612. In this case, the processing of the electronic device can include three stages: command parsing, pedestrian identification, and pedestrian removal. These three stages are explained below:

[0122] The electronic device first parses the user's command to identify the user's intent. This command parsing process takes a short time (e.g., about 1 second). To alleviate the user's waiting anxiety, such as... Figure 6 As shown in (b) above, the electronic device can display a smart voice assistant message 621 in the dialogue area 613 to remind the user to wait. For example, the smart voice assistant message 621 could be a star icon, as shown in the figure. Optionally, after receiving a user command, the electronic device can expand the chat window 611 to facilitate subsequent dialogue and result display. For example, Figure 6 Chat window 611 shown in (b) is compared to Figure 6 The chat window 611 shown in (a) has been enlarged vertically.

[0123] After recognizing the user's intent to remove pedestrians, the electronic device can identify pedestrians in image A. This pedestrian identification process also takes some time. To alleviate the user's waiting anxiety, such as... Figure 6 As shown in (c), the electronic device may no longer display the intelligent voice assistant message 621, but instead display the intelligent voice assistant message 622 to remind the user that it is recognizing passersby. For example, the intelligent voice assistant message 622 could be something like "Intelligently recognizing passersby" as shown in the figure. Optionally, during the process of recognizing passersby, the electronic device may, for example... Figure 6 As shown in (c), the intelligent voice assistant message 623 can also be displayed to remind the user of the current pedestrian removal processing progress. It should be noted that the pedestrian removal processing process can include both pedestrian recognition and pedestrian removal processes. The pedestrian removal processing progress displayed here represents the overall progress of the pedestrian removal processing process.

[0124] After identifying a pedestrian in image A, the electronic device can remove that pedestrian from the image. This pedestrian removal process also takes some time. To alleviate the user's anxiety while waiting, such as... Figure 6As shown in (d), the electronic device may stop displaying the intelligent voice assistant message 622 and instead display the intelligent voice assistant message 624 to remind the user that pedestrians are being eliminated. Furthermore, the electronic device may continue to display the intelligent voice assistant message 623 to remind the user of the current pedestrian elimination progress. For example, the intelligent voice assistant message 624 could be something like "Intelligently eliminating pedestrians" as shown in the figure.

[0125] After successfully removing the passerby from image A and obtaining image B, the electronic device, as follows: Figure 6 As shown in (e), the electronic device may no longer display intelligent voice assistant messages 624 and 623, but instead display intelligent voice assistant message 625 to remind the user that the pedestrian removal process has been completed. Additionally, the electronic device may display the intelligent voice assistant message "Thumbnail of Image B" and AI image editing controls 626. For example, intelligent voice assistant message 625 could be something like "Completed. You can save the image directly or continue editing via 'AI Image Editing,' or you can perform custom editing," as shown in the figure. Optionally, such as... Figure 6 As shown in (e), the intelligent voice assistant message "thumbnail of image B" can replace intelligent voice assistant message 623 displayed in its message area. AI image editing control 626 has been described above. Figure 1 The implementation method will be described in detail here.

[0126] In some cases, as stated above Figure 6 In this implementation, due to the absence of pedestrians in image A or other reasons, the electronic device may fail to recognize pedestrians in image A, i.e., pedestrian recognition fails. In this case, Figure 6 After interface 61 shown in (c), the electronic device may no longer display the intelligent voice assistant messages 622 and 623, but instead display an intelligent voice assistant message to remind the user of the failure to recognize a passerby. For example, this intelligent voice assistant message could be "No passerby was detected in the current image; please try again with a different image." Optionally, the electronic device may also display a camera control and an image control. The user can input an image using the camera control or the image control for reprocessing. For details on this image input process, please refer to the above. Figure 4 Implementation methods and Figure 5 The relevant descriptions in the implementation methods will not be repeated here.

[0127] The above text Figure 6 In some implementations, the electronic device displays the pedestrian removal progress via a separate intelligent voice assistant message 623. In other implementations, the electronic device may not display the pedestrian removal progress via a separate intelligent voice assistant message 623, but instead display the progress within other intelligent voice assistant messages. For example, in... Figure 6 In interface 61 shown in (c), the intelligent voice assistant message 623 may not be displayed. Instead, the pedestrian removal processing progress may be added to the message content of the intelligent voice assistant message 622. In this case, the intelligent voice assistant message 622 may be something like "Intelligently identifying pedestrians...10%", where "10%" represents the pedestrian removal processing progress. For example, in... Figure 6 In the interface 61 shown in (d), the intelligent voice assistant message 623 may not be displayed. Instead, the pedestrian removal processing progress may be added to the message content of the intelligent voice assistant message 624. In this case, the intelligent voice assistant message 624 may be something like "Intelligently removing pedestrians...15%", where "15%" represents the pedestrian removal processing progress.

[0128] The above text Figure 6 In this implementation, the pedestrian removal process includes a pedestrian identification process and a pedestrian removal process. The above description uses the display of the overall progress of the pedestrian removal process (i.e., the pedestrian removal process progress mentioned above) as an example. In other implementations, the electronic device may not display the overall progress of the pedestrian removal process, but instead display the progress of the pedestrian identification process (hereinafter referred to as pedestrian identification progress) and the progress of the pedestrian removal process (hereinafter referred to as pedestrian removal progress) separately. For example, in... Figure 6 In interface 61 shown in (c), the intelligent voice assistant message 623 may not be displayed; instead, the pedestrian recognition progress may be displayed. When displaying the pedestrian recognition progress, a separate intelligent voice assistant message reminding the user of the progress can be shown, or the pedestrian recognition progress can be added to the message content of intelligent voice assistant message 622. For example, in... Figure 6 In interface 61 shown in (d), the intelligent voice assistant message 623 may not be displayed; instead, the pedestrian elimination progress may be displayed. When displaying the pedestrian elimination progress, a separate intelligent voice assistant message can be shown to remind users of the progress, or the pedestrian elimination progress can be added to the message content of intelligent voice assistant message 624 for display.

[0129] It should be noted that in the above text Figure 6In one implementation, the user inputs the command "Remove Pedestrian" in the input box 612 via voice or text to trigger the electronic device to remove pedestrians. In other implementations, the user can also trigger the electronic device to remove pedestrians using the "Remove Pedestrian" control among the multiple AI editing controls displayed in the chat window 611. In this case, the electronic device can directly recognize the user's intent without parsing the user command. That is, the electronic device's processing includes two stages: pedestrian recognition and pedestrian removal. Therefore, after receiving the operation on the "Remove Pedestrian" control, the electronic device does not need to display the aforementioned intelligent voice assistant message 621, but can directly display the intelligent voice assistant message 622. Subsequent processes are the same as described above. Figure 6 The processes shown in (c) to (e) are similar; please refer to the above text for details. Figure 6 The relevant descriptions in the implementation methods will not be repeated here.

[0130] In some implementations, after the electronic device performs pedestrian removal processing on image A to obtain image B and displays it, the user can continue to process image B. For example, the user can trigger the processing of image B through the AI ​​image editing control 626, or by entering the user command "XX" in the input box 612. Here, "XX" refers to the processing the user wants to perform on the image; for example, the user command here could be "intelligent background removal". These two cases are explained below:

[0131] In one case, such as Figure 7 As shown in (a), the electronic device receives an operation applied to the AI ​​image editing control 626, and responds to the operation as follows: Figure 7 As shown in (b) above, the electronic device can display pop-up window 63. Pop-up window 63 may include multiple AI editing controls, such as controls for removing passersby, intelligent removal, intelligent background removal, generating ID photos, and one-click skin beautification. These multiple AI editing controls have already been described above. Figures 1 to 5 The implementation method will be described in detail here. Assume the electronic device receives an operation on the intelligent matting control. In response to this operation, the electronic device can perform intelligent matting processing on image B to obtain image C. Furthermore, as... Figure 7 As shown in (c), the electronic device can display a thumbnail of image C in dialog area 613.

[0132] In this situation, users can quickly determine what processing can be performed on image B using the AI ​​image editing control 626. This improves ease of use and increases user efficiency.

[0133] In another scenario, the electronic device receives a user command, "intelligent background removal," entered in input box 612. In response to this command, the electronic device can perform intelligent background removal on image B to obtain image C. The electronic device can then display a thumbnail of image C in dialog area 613.

[0134] In this case, the user can freely input the corresponding user instructions in the input box 612 according to their own image processing needs to continue processing image B, which provides high operational flexibility.

[0135] In some implementations, after the electronic device performs pedestrian removal processing on image A to obtain image B and displays it, the user can also reprocess image A. For example, the user can enter the editing interface of image A and trigger processing of image A within that interface; or, the user can enter the user command "process the original image A" in input box 612. Figure X The processing of image A is triggered by the "X" command. Here, "XX" refers to the processing the user wants to perform on the image; for example, the user command here could be "process the original image A...". Figure 1 "Key Skin Beautification". The following explains these two situations separately:

[0136] In one case, such as Figure 8 As shown in (a), the electronic device receives an operation applied to a thumbnail of image A, and in response to the operation, as follows: Figure 8 As shown in (b) above, the electronic device can display interface 64. Interface 64 is the editing interface for image A, and it can display image A and multiple controls. These multiple controls may include a sharing control 641, a saving control 642, a custom editing control 643, and an AI image editing control 644. These multiple controls have been described above. Figures 1 to 5 The implementation method will be described in detail here. Subsequently, the electronic device receives an operation applied to the AI ​​image editing control 644 and responds to the operation as follows: Figure 8 As shown in (c) above, the electronic device can display pop-up window 645. Pop-up window 645 may include multiple AI editing controls, such as controls for removing passersby, intelligent removal, intelligent background removal, generating ID photos, and one-click skin beautification. These multiple AI editing controls have already been described above. Figures 1 to 5 The implementation method will be described in detail here. Assume the electronic device receives an operation on the one-click skin-smoothing control; in response to this operation, the electronic device can perform skin-smoothing processing on image A. And, as... Figure 8 As shown in (d), the electronic device can display interface 61, and can display the user messages "thumbnail of image A" and "one-click skin beautification" in the dialog area 613 therein. Optionally, during the skin beautification process of image A, the electronic device can alleviate the user's waiting anxiety, such as... Figure 8 As shown in (d), the electronic device can also display the intelligent voice assistant message "One-click skin beautification for you" and the skin beautification process progress in the dialogue area 613.

[0137] In this scenario, users can access the editing interface for image A by manipulating the thumbnail of image A displayed in chat window 611. Within this editing interface, users can quickly learn about the various operations that can be performed on image A and trigger processing of image A according to their needs. This improves ease of use and enhances the user experience.

[0138] In another scenario, the electronic device receives a user instruction entered in input box 612: "to the original..." Figure 1 "Skin smoothing," in response to the user's command, the electronic device can perform skin smoothing on image A. Similar to the above, during the skin smoothing process for image A, the electronic device can also display various intelligent voice assistant messages to alleviate the user's waiting anxiety; these will not be elaborated further.

[0139] In this case, the user can freely input the corresponding user instructions in the input box 612 according to their own image processing needs to reprocess image A, which provides high operational flexibility.

[0140] In some cases, the embodiments of this application support multi-command dialogue. In multi-command dialogue scenarios, images can be processed in various ways according to user commands. The following describes... Figures 9 to 12 An example is provided to illustrate a multi-command dialogue scenario:

[0141] In some implementations, such as Figure 9 As shown in (a), the electronic device displays interface 61. Interface 61 can display a chat window 611, which may include an input box 612, multiple AI editing controls, and a dialogue area 613. The dialogue area 613 can display the user message "Thumbnail of image A".

[0142] If a user wants to remove passersby from image A and beautify the main subject, then... Figure 9 As shown in (a), a user can input the command "Remove passersby and beautify my image" via voice or text into input box 612. Upon receiving this user command in input box 612, the electronic device can sequentially perform passerby removal and skin beautification on image A. It should be noted that although this is a single user command, it contains two intentions (i.e., passerby removal and skin beautification), thus effectively issuing two commands. In this embodiment, such a user command containing multiple intentions is referred to as a multi-command.

[0143] In this scenario, the electronic device's processing can include four stages: instruction parsing, pedestrian recognition, pedestrian removal, and beautification. These four stages are explained below:

[0144] Electronic devices first parse user commands to identify user intent. During this command parsing process, to alleviate user anxiety while waiting, such as... Figure 9 As shown in (b), the electronic device can display a smart voice assistant message 651 in the dialogue area 613 to remind the user to wait.

[0145] After recognizing the user's intent to remove unwanted pedestrians and smooth skin, the electronic device can first identify pedestrians in image A. During this pedestrian identification process, to alleviate the user's waiting anxiety, such as... Figure 9 As shown in (c), the electronic device may no longer display the intelligent voice assistant message 651, but instead display the intelligent voice assistant message 652 to remind the user that it is recognizing a passerby. Optionally, during the process of recognizing a passerby, the electronic device may, as... Figure 9 As shown in (c), the intelligent voice assistant message 653 can also be displayed to remind the user of the current progress of the pedestrian removal process.

[0146] After identifying a pedestrian in image A, the electronic device can remove that pedestrian from the image. During this pedestrian removal process, to alleviate the user's waiting anxiety, such as... Figure 9 As shown in (d), the electronic device may stop displaying the intelligent voice assistant message 652 and instead display the intelligent voice assistant message 654 to remind the user that pedestrian removal is in progress. Furthermore, the electronic device may continue to display the intelligent voice assistant message 653 to remind the user of the current pedestrian removal progress.

[0147] After successfully removing passersby from image A and obtaining image B, the electronic device can perform skin-smoothing processing on the people in image B. During this skin-smoothing process, to alleviate the user's waiting anxiety, such as... Figure 9 As shown in (e), the electronic device may no longer display intelligent voice assistant messages 654 and 653, but instead display intelligent voice assistant message 655 to remind the user that a skin-beautifying treatment is in progress. Optionally, during the skin-beautifying treatment process, the electronic device may, as... Figure 9 As shown in (e), the intelligent voice assistant message 656 can also be displayed to remind the user of the current skin care treatment progress.

[0148] After the electronic device completes the skin smoothing process on the person in image B and obtains image D, as shown... Figure 9As shown in (f), the electronic device may no longer display intelligent voice assistant messages 655 and 656, but instead display intelligent voice assistant message 657 to remind the user that the passerby removal and skin beautification processes have been completed. Additionally, the electronic device may display the intelligent voice assistant message "Thumbnail of image D" and AI image editing controls. Optionally, as... Figure 9 As shown in (f), the intelligent voice assistant message "thumbnail of image D" can replace the intelligent voice assistant message 656 displayed in its message area.

[0149] It should be noted that the above text Figure 9 The implementation method is illustrated using the example of displaying the overall progress of the pedestrian removal process. In other implementations, the electronic device may not display the overall progress of the pedestrian removal process, but instead display the progress of the pedestrian identification process and the progress of the pedestrian removal process separately, as detailed above. Figure 6 The relevant descriptions in the implementation methods will not be repeated here.

[0150] The above text Figure 9 The implementation method provides an exemplary description of the process by which an electronic device performs various processes based on user instructions. When all these processes are completed, such as... Figure 9 As shown in (f), the electronic device can display a thumbnail of the final image D.

[0151] In some cases, only part of the multiple processes may be completed. For example, the electronic device may only remove passersby but not perform the subsequent skin-beautifying process. In such cases, Figure 9 After interface 61 shown in (e), the electronic device may no longer display intelligent voice assistant messages 655 and 656, but instead display an intelligent voice assistant message indicating that the pedestrian removal process was successful but the beautification process failed. For example, this intelligent voice assistant message could be something like, "Pedestrians have been intelligently removed for you, but beautification is not possible because there are no people in the image or the people are too small. You can save it directly or perform custom editing," etc. Furthermore, the electronic device can also display a thumbnail of the image B obtained after the pedestrian removal process.

[0152] In other cases, all of these processing steps may fail to complete. For example, if the electronic device fails to identify any pedestrians when performing pedestrian removal processing on image A, it can suspend the pedestrian removal process and then attempt to perform skin smoothing on the figures in image A, but this skin smoothing process also fails. In this situation, Figure 9After interface 61 shown in (c), the electronic device may no longer display the intelligent voice assistant messages 652 and 653, but instead display an intelligent voice assistant message indicating that both the pedestrian removal and beautification processes have failed. For example, this intelligent voice assistant message could be something like, "Sorry, the operation could not be completed. The image cannot be removed because there are no pedestrians, or the image is too small to beautify. Please try again with a different image." Optionally, the electronic device may also display a camera control and an image control. The user can input an image using the camera control or image control to reprocess it. For details on this image input process, please refer to the above text. Figure 4 Implementation methods and Figure 5 The relevant descriptions in the implementation methods will not be repeated here.

[0153] It should be noted that in the above text Figure 9 In this implementation, during the pedestrian removal and skin beautification processes, the electronic device reminds the user of the current processing through text messages in the intelligent voice assistant. For example, intelligent voice assistant message 652 reminds the user that pedestrian recognition is in progress; intelligent voice assistant message 654 reminds the user that pedestrian removal is in progress; and intelligent voice assistant message 655 reminds the user that skin beautification is in progress. In other implementations, the electronic device may also remind the user of the current processing through status controls, or through a combination of status controls and text. The following is a detailed explanation... Figures 10 to 12 This is illustrated by example:

[0154] exist Figure 9 After interface 61 shown in (b) of the diagram, the electronic device recognizes the user's intent to remove passersby and beautify the skin.

[0155] Next, the electronic device first identifies the pedestrians in image A. During this pedestrian identification process, to alleviate the user's waiting anxiety, the electronic device can stop displaying the intelligent voice assistant message 651 and instead display something like... Figure 10 The intelligent voice assistant message 661 shown in (a) reminds the user that image processing is in progress. Intelligent voice assistant 661 is related to the above... Figure 9 The intelligent voice assistant message 652 in the implementation is different. For example, the intelligent voice assistant message 661 may include a first state control 662 and a second state control 663. The first state control 662 is used to indicate the status of the passerby removal process, which can change between "not started," "in progress," "success," and "failure." The second state control 663 is used to indicate the skin beautification process status, which can change between "not started," "in progress," "success," and "failure." For example, Figure 10 The state of the first state control 662 shown in (a) is "in progress" to indicate that pedestrian removal processing is in progress. Figure 10 The second state control 663 shown in (a) is in the state of "Not Started" to indicate that the skin beautification process has not yet started. Optionally, during the pedestrian recognition process, the electronic device may also display a smart voice assistant message to remind pedestrians of the overall progress of the removal process, or a smart voice assistant message to remind pedestrians of the progress of the pedestrian recognition process.

[0156] After recognizing a pedestrian in image A, the electronic device can remove the pedestrian. During this pedestrian removal process, to alleviate the user's waiting anxiety, the electronic device can continue to display intelligent voice assistant message 661. Optionally, during the pedestrian removal process, the electronic device can also display intelligent voice assistant messages to remind users of the overall progress of the pedestrian removal process, or display intelligent voice assistant messages to remind users of the progress of the pedestrian removal process.

[0157] After successfully removing passersby from image A and obtaining image B, the electronic device can perform skin-smoothing processing on the people in image B. During this skin-smoothing process, to alleviate the user's waiting anxiety, the electronic device can continue to display the intelligent voice assistant message 661. In this case, such as... Figure 10 As shown in (b), the state of the first state control 662 in the intelligent voice assistant message 661 changes from "in progress" to "success" to indicate that the pedestrian removal process has been completed; the state of the second state control 663 changes from "not started" to "in progress" to indicate that the skin beautification process is in progress.

[0158] After the electronic device completes the skin smoothing process on the person in image B, resulting in image D, it can continue displaying the intelligent voice assistant message 661. In this case, such as... Figure 10 As shown in (c), the state of the first status control 662 in the intelligent voice assistant message 661 remains "successful"; the state of the second status control 663 changes from "in progress" to "success" to indicate that the skin beautification treatment has been completed. Furthermore, the electronic device can also display a thumbnail of image D.

[0159] As an example, a user can view image B obtained after the "passerby removal" process via the first state control 662, and image D obtained after the "skin smoothing" process via the second state control 663. That is, through the first state control 662 and the second state control 663, the user can selectively view the process images obtained after various processes. This improves the user experience. For example, for... Figure 10In the intelligent voice assistant message 661 shown in (c), assuming the electronic device receives an operation on the first state control 662 of the intelligent voice assistant message 661, in response to this operation, the electronic device no longer displays a thumbnail of image D, but instead displays a thumbnail of image B. Subsequently, assuming the electronic device receives an operation on the second state control 663 of the intelligent voice assistant message 661, in response to this operation, the electronic device no longer displays a thumbnail of image B, but instead displays a thumbnail of image D.

[0160] The above text Figure 10 The implementation method exemplifies the process by which an electronic device performs various processes based on user instructions. Once all these processes are completed, the electronic device can display a thumbnail of the final image D.

[0161] In some cases, only part of the multiple processes may be completed. For example, the electronic device may only remove passersby but not complete the subsequent skin-beautifying process. In this situation, the electronic device will display... Figure 10 Following the intelligent voice assistant message 661 shown in (b) above, it can be displayed Figure 11 The intelligent voice assistant message 661 is shown in (b) of the diagram. Figure 11 In the intelligent voice assistant message 661 shown in (b), the first status control 662 remains in the "success" state, while the second status control 663 changes from "in progress" to "failure" to indicate that the skin beautification process has failed. Furthermore, the electronic device can also display a thumbnail of the image B obtained after the passerby removal process. Optionally, as... Figure 11 As shown in (b), the electronic device can also display a reminder message "The image does not contain any human figures or the human figures are too small to beautify", to remind the user of the reason for the skin beautification failure.

[0162] In other cases, all of these processes may fail to complete. For example, if the electronic device fails to identify any pedestrians when performing pedestrian removal processing on image A, it can stop the pedestrian removal process and then attempt to perform skin smoothing on the people in image A, but this skin smoothing process also fails. In this situation, the electronic device will display... Figure 10 Following the intelligent voice assistant message 661 shown in (a), it can be displayed Figure 12 The intelligent voice assistant message 661 is shown in (b) of the diagram. Figure 12In the intelligent voice assistant message 661 shown in (b), the state of the first state control 662 changes from "in progress" to "failed" to indicate that the pedestrian removal process has failed; the state of the second state control 663 changes from "not started" to "failed" to indicate that the skin beautification process has failed. The electronic device can also display an intelligent voice assistant message to remind the pedestrian that both the removal and skin beautification processes have failed. For example, this intelligent voice assistant message could be something like, "Sorry, the operation could not be completed. Please try again with a different image." Optionally, the electronic device can also display a camera control and an image control. The user can input an image using the camera control or the image control to reprocess it. For details on this image input process, please refer to the above. Figure 4 Implementation methods and Figure 5 The relevant descriptions in the implementation methods will not be repeated here.

[0163] In some implementations, the user command entered by the user in the input box 612 on interface 61 may have an unclear target. For example, if the user enters the command "Remove the person next to Li Si and beautify the image," the electronic device may have difficulty determining which person Li Si is in the image and therefore cannot process the command. In such cases, the electronic device can allow the user to clarify the command, as described below. Figure 13 This is illustrated by example:

[0164] After receiving the user instruction "Remove the person next to Li Si and beautify the image" entered in input box 612, the electronic device first parses the user instruction to identify the user's intent. After recognizing the user's intent as removing the person next to Li Si and beautifying the image, the electronic device, unable to determine which person in image A Li Si refers to, performs the following... Figure 13 As shown in (a), the electronic device can display intelligent voice assistant message 671 to ask follow-up questions about Li Si's characteristics. For example, the electronic device can ask follow-up questions based on the characteristics of the main figure in image A. For instance, intelligent voice assistant message 671 could be something like, "Is Li Si the person wearing red in the picture?" Alternatively, the electronic device can prompt the user to provide specific characteristics of the person. For instance, intelligent voice assistant message 671 could be something like, "Please tell me what Li Si's characteristics are, such as the color of his clothes, his posture, and his position in the picture."

[0165] After seeing message 671 from the smart voice assistant, the user can reply. For example, in message 671 from the smart voice assistant... Figure 13 In the case of "Is Li Si the person in the red clothes in the picture?" shown in (a), if Figure 13 As shown in (b), the user can reply "yes." Based on this, the electronic device can determine that the user's intention is to remove unwanted people and smooth skin. The subsequent steps can then be performed as described above. Figures 9 to 12The implementation details include pedestrian recognition, pedestrian removal, and skin smoothing processes, which will not be elaborated upon here.

[0166] In some implementations, if a user wants to remove passersby from an image, they can do so not only with a full command, such as "remove passersby" or "remove the person next to YY" as described in the above implementations, but also with a short command. As an example, after the electronic device receives the user's command "remove" entered into the input box of the intelligent voice assistant, if passersby are present in the image, no user clarification is needed; the user's intention is directly determined to remove passersby. If no passersby are present in the image, user clarification is still required. The following explains the situations where user clarification is required:

[0167] Suppose there are no pedestrians in the image, but there is a deer and a penguin. After receiving the user command "eliminate" entered into the input box of the smart voice assistant, the electronic device can display a smart voice assistant message in the chat window asking what to eliminate. For example, the smart voice assistant message could be, "What do you want to eliminate? Try saying 'eliminate the deer in the picture' or 'eliminate the penguin in the picture'." The user can reply with the desired elimination object via voice or text. The electronic device can determine the user's intent based on the user's response. In some implementations, while displaying the smart voice assistant message asking for the elimination object in the chat window, the electronic device can also display one or more options in the chat window based on the objects present in the image, allowing the user to quickly select the object to eliminate. For example, the electronic device can display options A and B in the chat window, where option A instructs the user to eliminate the deer, and option B instructs the user to eliminate the penguin. The user can select the desired elimination object using option A or option B. The electronic device can determine the user's intent based on the user's action on option A or option B.

[0168] It should be noted that the embodiments of this application not only provide a function to remove passersby, but also a smart removal function. The smart removal function allows the user to specify the objects to be removed. For example, in the embodiments described above, the electronic device can automatically recommend smart removal controls to the user based on the content of the image in the large image page or the content of the image in the chat window of the intelligent voice assistant; or, the electronic device can display the smart removal controls in its corresponding pop-up window after receiving an operation applied to the AI ​​image editing controls.

[0169] Suppose an electronic device receives an action applied to the smart removal control. In response, the device can automatically enter the user's command "Remove the object from the image" into the input box of the smart voice assistant. The word "object" is presented as an editable field. For example, "object" could be displayed with a dotted border and a highlighted background to highlight the editable area. In this case, the user can quickly input the command by changing "object" to the object they want to remove. Suppose the user changes "object" to "wall." After modification, the user can operate the send control in the input box. The electronic device receives the action applied to the send control and, in response, receives the user's command "Remove the wall from the image." The electronic device can then remove the wall from the image.

[0170] It should be noted that the embodiments described in this application are only those described above. Figures 1 to 13 The AI ​​editing function is illustrated by taking the process described in the implementation method as an example. Figures 1 to 13 The process described in the embodiments does not constitute a specific limitation on the implementation of the embodiments of this application. In practical applications, AI editing functions can also be implemented in other ways different from the above methods, and the embodiments of this application do not limit this.

[0171] After understanding the relevant interface of the AI ​​editing function provided in the embodiments of this application, the specific implementation process of the AI ​​editing function will be described in detail below.

[0172] First, the electronic devices involved in the embodiments of this application will be described. For example, the electronic device may be a mobile phone, tablet computer, wearable device, digital camera, television, in-vehicle device, augmented reality (AR) device, virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), laptop computer, terminal in industrial control, terminal in self-driving, terminal in remote medical surgery, terminal in smart grid, terminal in transportation safety, terminal in smart city, terminal in smart home, etc., and the embodiments of this application do not limit this.

[0173] Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. See also... Figure 14 The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0174] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0175] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, memory, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). Different processing units may be independent devices or integrated into one or more processors. The controller may serve as the central nervous system and command center of electronic device 100. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0176] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves system efficiency.

[0177] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions, such as saving music, video, and other files on the external memory card.

[0178] Internal memory 121 can be used to store computer-executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created by electronic device 100 during use (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0179] The embodiments of this application do not particularly limit the specific structure of the executing entity of the image processing method. As long as the image processing method provided by the embodiments of this application can be executed by running the code that records the image processing method provided by the embodiments of this application, the image processing can be performed according to the image processing method provided by the embodiments of this application. For example, the executing entity of the image processing method provided by the embodiments of this application can be a functional module in an electronic device that can call and execute a program, or it can be a processing device applied in an electronic device, such as a chip.

[0180] The software system of an electronic device is described below. The software system of an electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application uses the layered architecture Android system as an example to illustrate the software system of an electronic device. It should be noted that although this application uses the Android system as an example, its basic principles are equally applicable to electronic devices based on operating systems such as iOS or Windows.

[0181] Figure 15 This is a block diagram of a software system for an electronic device according to an embodiment of this application. See also... Figure 15 A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some implementations, the Android system is divided into an application layer, an application framework layer, the Android runtime, a system layer, and a kernel layer.

[0182] The application layer can include a series of applications. For example... Figure 15 As shown, the application layer can include applications such as calendar, map, video, music, SMS, call, Bluetooth, camera, gallery, and smart voice assistant.

[0183] Intelligent voice assistants can interact with users, understand user commands, and provide corresponding functions based on those commands.

[0184] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0185] like Figure 15 As shown, the application framework layer may include a voice engine, a smart brain, etc.

[0186] A speech engine is used to parse user commands to identify intelligent agents capable of processing them. An intelligent agent is a program that performs a specific task. Different intelligent agents can handle different types of tasks.

[0187] The intelligent brain is used for managing intelligent agents. For example, such as... Figure 16As shown, the intelligent brain can include an agent management module (also known as an agent lifecycle management framework), one or more agents, and an artificial general intelligence agent framework (AGI agent framework). The agent management module can manage the lifecycle of each agent and can invoke each agent. The agent framework is used to realize the interaction between edge-side agents and cloud-side agent services.

[0188] For example, the one or more intelligent agents may include image editing agents, video editing agents, etc. An intelligent agent may include a task engine and one or more task tools. Task tools can be used to perform tasks. For example, an image editing agent may include skin smoothing tools, removal tools, etc. The skin smoothing tool can perform skin smoothing tasks; the removal tool can perform removal tasks. During task execution, the task tools can call the software development kit (SDK) interface of the visual platform to request the visual platform to perform related operations.

[0189] Optionally, the intelligent brain may also include a learning and memory platform. This platform can learn about the relationships between people in images managed by the image library application, thereby enabling person clarification during image processing.

[0190] The Android Runtime comprises the core libraries and the virtual machine. The Android Runtime is responsible for the scheduling and management of the Android system. The core libraries consist of two parts: one part contains the functionalities that Java calls, and the other part is the core Android library itself. The application layer and application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0191] The system layer can include multiple functional modules. For example... Figure 15 As shown, the system layer may include a visual middleware platform. The visual middleware platform is the central hub for scheduling visual capabilities, encapsulating complex cloud-side visual capabilities (such as object detection, instance segmentation, and elimination) into visual services that can be invoked by edge-side agents. Edge-side agents can call the SDK interface of the visual middleware platform to perform related operations through the corresponding visual services.

[0192] The kernel layer is the layer between hardware and software. The kernel layer can include display drivers, camera drivers, audio drivers, sensor drivers, etc.

[0193] Understandably, the embodiments described in this application are only for... Figure 15 The software system shown is used as an example for illustration. Figure 15 The software system shown does not constitute a specific limitation on the electronic device. In practical applications, the internal structure of the electronic device can be divided into different layers and / or different functional modules as needed to complete some or all of the functions described above. Furthermore, the names of each layer and each functional module are only for easy distinction and are not intended to limit the scope of protection of the embodiments of this application.

[0194] The implementation method of this application can achieve image processing through edge-cloud collaboration. The following will combine... Figure 16 An example is provided to illustrate the architecture of edge-cloud collaboration.

[0195] Figure 16 This is a schematic diagram of an edge-cloud collaborative architecture provided in an embodiment of this application. See also... Figure 16 Electronic devices may include the above. Figure 15 The implementation plan includes intelligent voice assistants, voice engines, intelligent brains, and visual platforms. Cloud servers may include first cloud services, second cloud services, third cloud services, and fourth cloud services.

[0196] The first cloud service is a cloud-side image storage module used to store images sent from the end-user side. Optionally, the first cloud service can also be called a smart storage cloud. For example, the first cloud service can be an object storage service (OSS or OBS) cloud, etc.

[0197] The second cloud service is used to identify the intelligent agent that will process the user's command. Optionally, the second cloud service may also be referred to as a smart voice cloud.

[0198] The third cloud service is used for task orchestration and response generation. Optionally, the third cloud service may also be called a smart voice cloud. A cloud server may include one or more third cloud services. Each of these third cloud services corresponds one-to-one with one or more intelligent agents. Each intelligent agent can interact with its corresponding third cloud service. Each third cloud service can provide its corresponding intelligent agent with functions such as task orchestration and response generation. By setting up corresponding third cloud services for each intelligent agent, task orchestration and response generation can be better tailored to the different needs of each agent, thereby greatly improving the accuracy and efficiency of user command execution.

[0199] For example, the third-party cloud service may include an intelligent agent microservice, a task orchestration model, and a response generation model. The intelligent agent microservice can invoke the task orchestration model to parse user instructions and obtain a task list based on client-side requests; alternatively, the intelligent agent microservice can invoke the response generation model to generate dialogue responses based on client-side requests. For example, both the task orchestration model and the response generation model can be large language models (LLMs), etc.

[0200] The fourth cloud service provides visual capabilities such as object detection, instance segmentation, and removal. Object detection refers to detecting specific objects in an image. Instance segmentation refers to separating specific objects from an image. Removal refers to removing specific objects from an image.

[0201] Understandably, Figure 16 The functional modules shown in the cloud server diagram do not constitute a specific limitation on the cloud server. The division and naming of functional modules in the cloud server can be adjusted according to actual usage needs.

[0202] One possible interaction process between the edge and the cloud can be referenced. Figure 16 The numbers, arrows, and interactive content are shown below. The following is a brief explanation of this interactive process. This interactive process may include the following steps (1) to (16).

[0203] (1) The intelligent voice assistant obtains user commands and images. For specific operations, please refer to the relevant descriptions in steps 1701 to 1702 below.

[0204] (2) The intelligent voice assistant stores the image to the first cloud service and obtains the cloud storage address of the image. For specific operations, please refer to the relevant descriptions in steps 1703 to 1705 below.

[0205] (3) The intelligent voice assistant sends user commands to the voice engine. For specific instructions, please refer to the relevant description in step 1706 below.

[0206] (4) The voice engine returns the agent identifier to the intelligent voice assistant. For specific instructions, please refer to the relevant description in step 1706 below.

[0207] (5) The intelligent voice assistant sends user commands, intelligent agent identifiers, and image storage addresses (including local and cloud storage addresses) to the intelligent agent management module. For specific instructions, please refer to the relevant description in step 1707 below.

[0208] (6) The agent management module calls the corresponding agent based on the agent identifier. For specific operations, please refer to the relevant description in step 1708 below.

[0209] (7) The task engine in the agent sends a task orchestration request to the agent framework according to the user's instructions. For specific operations, please refer to the relevant description in step 1709 below.

[0210] (8) The agent framework returns a task list to the task engine. For specific operations, please refer to the relevant descriptions in steps 1709 to 1711 below.

[0211] (9) The task engine calls the corresponding task tool based on the task list and image storage address. For specific instructions, please refer to the relevant description in step 1712 below.

[0212] (10) The task tool calls the visual services provided by the visual platform to perform image processing based on the task to be executed and the image storage address. For specific operations, please refer to the relevant description in step 1712 below.

[0213] (11) The visual platform returns the image processing results to the task tool. For specific operations, please refer to the relevant description in step 1712 below.

[0214] (12) The task tool returns the task execution result to the task engine based on the image processing result. For specific operation details, please refer to the relevant description in step 1712 below.

[0215] (13) The task engine sends a response generation request to the agent framework based on the task execution result. For specific operations, please refer to the relevant description in step 1713 below.

[0216] (14) The agent framework returns a response to the task engine. For specific details, please refer to the relevant descriptions in steps 1713 to 1715 below.

[0217] (15) The task engine returns a response and a result image to the agent management module. For specific instructions, please refer to the relevant description in step 1716 below.

[0218] (16) The intelligent agent management module returns a response and a result image to the intelligent voice assistant. For specific operations, please refer to the relevant description in step 1717 below.

[0219] The following describes a timing diagram of the image processing method provided in the embodiments of this application.

[0220] Figure 17 This is a flowchart of an image processing method provided in an embodiment of this application. See also... Figure 17 The method may include the following steps:

[0221] Step 1701: The intelligent voice assistant obtains user commands.

[0222] This user instruction can be a user-triggered instruction. In this embodiment, the user instruction is used to instruct image processing. This user instruction can also be called an image editing instruction.

[0223] For example, the user command can be entered by the user through voice or text into the input box of the smart voice assistant. For instance, the user can enter the command "remove pedestrians" through voice or text into the input box of the smart voice assistant.

[0224] Alternatively, the user command can be generated after the user clicks on an AI editing control displayed by the smart voice assistant. The user commands corresponding to each AI editing control can be pre-set, and different AI editing controls can have different user commands. For example, clicking the "Remove Pedestrians" control displayed by the smart voice assistant will issue the user command "Remove Pedestrians".

[0225] Of course, this is not the only possibility. The user instruction can also be triggered by the user in other ways, such as by gesture operation or motion sensing operation. This application does not limit this.

[0226] Step 1702: In response to the user's instruction, the intelligent voice assistant obtains the image to be processed (i.e., the first image).

[0227] In one possible approach, when an electronic device displays a large image page, and the user has not input an image before triggering the user command, the intelligent voice assistant can use the image displayed on the large image page as Image 1, and then retrieve Image 1 based on its storage address 1 (also known as the local storage address). Storage address 1 of Image 1 is the storage address of Image 1 within the electronic device. It should be noted that the intelligent voice assistant can obtain the storage address 1 of Image 1 displayed on the large image page immediately after the large image page is activated.

[0228] For example, such as Figure 1 As shown in (d), the electronic device displays a large image page of image a (i.e., interface 24) and automatically launches the intelligent voice assistant. Then, the user clicks the "Remove Pedestrian" control 242 to trigger the user command "Remove Pedestrian". In this case, the intelligent voice assistant can use image a displayed in the large image page as image 1 and acquire image 1.

[0229] In another possible approach, the intelligent voice assistant can use the most recently input image by the user before triggering the user's command as image 1, and then retrieve image 1 based on its storage address 1. It should be noted that after the user inputs image 1 into the intelligent voice assistant, the assistant can obtain image 1's storage address 1.

[0230] For example, such as Figure 2(c) to Figure 3 As shown in (a), the user passes image c from another application to the intelligent voice assistant. The user then taps the remove pedestrian control 353 to trigger the user command "remove pedestrian". In this case, the intelligent voice assistant can use the most recently entered image c before triggering the user command as image 1 and obtain image 1.

[0231] For example, such as Figure 4 As shown in (a) to (e), the user inputs image e into the intelligent voice assistant by adding a picture. Then, the user clicks the "Remove Pedestrian" control 463 to trigger the user command "Remove Pedestrian". In this case, the intelligent voice assistant can use the most recently input image e before triggering the user command as image 1 and obtain image 1.

[0232] In another possible approach, if the electronic device is not displaying a large image page and the user has not entered an image before triggering the user command, the intelligent voice assistant can display a reminder message to prompt the user to enter an image. After the user enters an image, the intelligent voice assistant can use that image as Image 1 and then retrieve Image 1 from its storage address 1. It should be noted that the intelligent voice assistant can obtain the storage address 1 of Image 1 immediately after the user enters Image 1.

[0233] For example, such as Figure 5 As shown in (d) to (h), the user clicks the "Remove Pedestrian" control 553 to trigger the user command "Remove Pedestrian". However, since the user did not input an image, the intelligent voice assistant prompts the user to upload an image. Afterwards, the user inputs image g into the intelligent voice assistant by taking a photo. The intelligent voice assistant then uses image g as image 1 and acquires image 1.

[0234] Of course, it is not limited to the above methods. Intelligent voice assistants can also obtain the image to be processed 1 through other methods. The implementation method of this application does not limit this.

[0235] Step 1703: The intelligent voice assistant sends image 1 to the first cloud service.

[0236] Optionally, the intelligent voice assistant can compress image 1 to obtain a compressed image 1, and then send the compressed image 1 to the first cloud service for storage. This can save transmission and storage resources.

[0237] Step 1704: After receiving image 1, the first cloud service stores image 1 and obtains the storage address 2 of image 1.

[0238] The storage address 2 of image 1 (also known as the cloud storage address) is the storage address of image 1 in the first cloud service. For example, the storage address 2 of image 1 can be the uniform resource locator (URL) address of image 1, etc.

[0239] Step 1705: The first cloud service sends the storage address 2 of image 1 to the intelligent voice assistant.

[0240] After receiving the storage address 2 of image 1, the intelligent voice assistant can save the storage address 2 of image 1.

[0241] It should be noted that, in this embodiment of the application, steps 1703 to 1705 can be executed after step 1702 to store image 1 to the first cloud service. Simultaneously, step 1706 can also be executed to determine the target intelligent agent for processing user requests.

[0242] Step 1706: The intelligent voice assistant identifies the target intelligent agent.

[0243] In one possible approach, step 1706 can be: the intelligent voice assistant determines the target agent through a voice engine based on the user's instruction. In this approach, the intelligent voice assistant can parse the user's instruction through the voice engine to determine the target agent for processing the user's instruction.

[0244] In another possible approach, step 1706 can be performed as follows: if it is determined that the user instruction is triggered by an AI editing control, the agent corresponding to the AI ​​editing control is identified as the target agent; if it is determined that the user instruction is not triggered by an AI editing control and the electronic device is displaying an agent interface, the agent to which the agent interface belongs is identified as the target agent; if it is determined that the user instruction is not triggered by an AI editing control and the electronic device is not displaying an agent interface, the target agent is identified by the voice engine based on the user instruction.

[0245] Each AI editing control can represent a corresponding intent. For example, the "Remove Passersby" control represents the intent to remove passersby, while the "One-Click Skin Beautification" control represents the intent to beautify skin. Each intelligent agent can implement a corresponding intent. For example, an image editing intelligent agent can be used to implement intents related to image processing. Therefore, the intelligent agents corresponding to each AI editing control can be pre-defined. The intelligent agent corresponding to an AI editing control is used to implement the intent represented by that AI editing control; that is, the intelligent agent is used to execute the task of that intent. The intelligent agents corresponding to different AI editing controls can be the same or different. For example, the "Remove Passersby" control, the "Intelligent Removal" control, the "Intelligent Background Removal" control, the "Generate ID Photo" control, and the "One-Click Skin Beautification" control can all correspond to image editing intelligent agents.

[0246] If an electronic device is displaying an agent interface, it means the user is using the agent associated with that interface. Therefore, this agent can be directly identified as the target agent. For example, ... Figure 5 As shown, if the user triggers the user command "remove passersby" in the interface 55 of the image editing agent, the image editing agent can be directly used as the target agent.

[0247] In this embodiment, when a user triggers a user command through an AI editing control, there is no need to parse the command; the target agent can be directly determined based on the AI ​​editing control. Similarly, when a user triggers a user command on the agent interface, there is also no need to parse the command; the target agent can be directly determined based on the agent interface. This reduces user waiting time. However, when a user triggers a command neither through an AI editing control nor on the agent interface, the command needs to be parsed using a voice engine to determine the target agent.

[0248] As an example, in the first and second possible methods described above, the intelligent voice assistant can determine the operation of the target intelligent agent through the voice engine based on the user's instruction in either method 1 or method 2.

[0249] Method 1: The intelligent voice assistant sends the user command to the voice engine. Upon receiving the user command, the voice engine sends it to the second cloud service. The second cloud service, upon receiving the user command, parses it to obtain the identifier of the target intelligent agent used to process the command, and sends the identifier to the voice engine. Upon receiving the identifier, the voice engine sends processing instruction 1 to the intelligent voice assistant, which includes the identifier of the target intelligent agent. Upon receiving processing instruction 1, the intelligent voice assistant can then identify the target intelligent agent.

[0250] In this way, the second cloud service can accurately identify the target intelligent agent required to process user instructions, ensuring that user instructions can be correctly assigned to the appropriate intelligent agent, thereby improving the accuracy of user instruction processing.

[0251] Optionally, the second cloud service can parse the user command using the first model. The first model can be a pre-set cloud-side large model. The first model can determine the intelligent agent capable of processing the user command based on the user command. For example, the first model can be an LLM, etc.

[0252] Optionally, the voice engine may include an intent controller. The intent controller can interact with a second cloud service to determine, with the help of the second cloud service, the target agent for processing the user command.

[0253] Method 2: The intelligent voice assistant sends the user command to the voice engine. Upon receiving the user command, the voice engine sends it to the second cloud service. The second cloud service, upon receiving the user command, parses it to obtain the user intent and sends this intent to the voice engine. Upon receiving the user intent, the voice engine sends processing instruction 1 to the intelligent voice assistant, which includes the user intent. Upon receiving processing instruction 1, the intelligent voice assistant identifies the agent corresponding to the user intent as the target agent.

[0254] The agent corresponding to this user intent is the agent used to implement that user intent. The agents corresponding to various user intents can be pre-defined. For example, the user intent for the command "Remove passersby and beautify my image" can be an image editing intent. The agent corresponding to the image editing intent can be an image editing agent. Similarly, the user intent for the command "Please generate a dynamic video" can be a video editing intent. The agent corresponding to the video editing intent can be a video editing agent.

[0255] In this way, the user intent corresponding to the user command can be accurately identified through the second cloud service, and the user command can be correctly assigned to the corresponding intelligent agent, thereby improving the accuracy of user command processing.

[0256] Optionally, the second cloud service can parse the user command using an intent recognition model. The intent recognition model can be a pre-set cloud-side large model. The intent recognition model can determine the corresponding user intent based on the user command. For example, the intent recognition model could be an LLM (Local Level Model), etc.

[0257] Optionally, the voice engine may include an intent controller. The intent controller can interact with a second cloud service to determine the user intent corresponding to the user command.

[0258] As an example, in the first and second possible approaches described above, while the intelligent voice assistant is identifying the target agent through its voice engine based on the user's instruction, it can also display a reminder message to alleviate the user's anxiety while waiting. For example, this reminder message could be... Figure 6 The intelligent voice assistant message 621 shown in (b) is displayed with a star graphic.

[0259] Step 1707: The intelligent voice assistant sends processing instruction 2 to the intelligent agent management module. Processing instruction 2 includes the user instruction, the identifier of the target intelligent agent, and the storage address 1 and storage address 2 of image 1.

[0260] Step 1708: After receiving the processing instruction 2, the agent management module sends the processing instruction 3 to the target agent. The processing instruction 3 includes the user instruction, the storage address 1 of image 1, and the storage address 2.

[0261] After receiving processing instruction 2, the agent management module can invoke the target agent so that the target agent can process image 1 according to the user instruction.

[0262] Step 1709: After receiving the processing instruction 3, the target intelligent agent sends a task orchestration request to the third cloud service. The task orchestration request includes the user instruction and the storage address 2 of image 1.

[0263] Optionally, the target agent can send the task orchestration request to the agent framework. After receiving the task orchestration request, the agent framework can send the task orchestration request to the third-party cloud service.

[0264] Step 1710: After receiving the task orchestration request, the third cloud service retrieves image 1 from the first cloud service according to the storage address 2 of image 1, and retrieves the task list according to the user instruction and image 1.

[0265] This task list includes one or more tasks. The tasks in this task list are those that need to be performed to fulfill this user instruction. This task list may include one or more of the following: removal tasks, skin smoothing tasks, background removal tasks, ID photo generation tasks, element addition tasks, basic editing tasks, etc.

[0266] Removal tasks can include pedestrian removal tasks, intelligent removal tasks, etc. Pedestrian removal tasks are used to remove pedestrians from an image. Intelligent removal tasks are used to remove specific targets from an image. Skin smoothing tasks are used to smooth the skin of people in an image. Background removal tasks are used to separate the subject from an image. ID photo generation tasks are used to generate ID photos of people in an image. Element addition tasks are used to add specific elements to an image. Basic editing tasks are used to perform common image editing, such as cropping, applying filters, mosaic effects, and background blurring.

[0267] The tasks in this task list have an execution order. The execution order of any two tasks in this task list can be either sequential or parallel.

[0268] It should be noted that the sequential execution order of the two tasks described in this application means that there is a strict order in which the two tasks are executed. For example, assuming that task a and task b are executed sequentially, it means that task a must be executed first and then task b.

[0269] The parallel execution order of the two tasks described in this application means that there is no strict order in which the two tasks are executed. For example, assuming that task c and task d are executed in parallel, it means that task c can be executed first and then task d, or task d can be executed first and then task c.

[0270] For example, each task in the task list may include data such as task number (taskId), task tool identifier (tool), and task parameters, but this application implementation does not limit this.

[0271] Task numbers can indicate the order in which tasks are executed. For example, if there are 3 tasks with task numbers 1, 2, and 3, these 3 tasks can be executed sequentially. Alternatively, if there are 3 tasks, all with task number 1, these 3 tasks can be executed in parallel.

[0272] Task tool identifiers are used to identify the task tool used to perform the task. For example, the task of removing passersby corresponds to the removal tool (photo_erase), the task of beautifying skin corresponds to the beautification tool (photo_beauty), and the task of generating ID photos corresponds to the ID photo tool (photo_id_generator).

[0273] Each task tool corresponds to one task parameter. For example, the task parameters for an eraser tool may include the following object position information (object_avatar_positions): the task object's tag, position, etc. The task object's position may include its coordinates (e.g., x and y coordinates) and the size of its prediction bounding box (e.g., width and height). Similarly, the task parameters for a skin smoothing tool may include object position information. Furthermore, the task parameters for an ID photo tool may include the following specification parameters (parm): background color, ID photo size (e.g., photo_width and photo_height), etc.

[0274] Optionally, the task list can be a list in the form of a domain-specific language (DSL). Of course, the task list can also be in other forms, and this application does not limit this. For example, a task list in DSL form can be as follows: [

[0276] {"taskId": "1", "tool": "photo_erase", "object_avatar_positions": [{"tag": "passerby", "position": {"x": 1.0f, "y": 1.0f, "width": 1.0f, "height": 1.0f}}]},

[0277] {"taskId": "2", "tool": "photo_beauty", "object_avatar_positions": [{"tag": "Beauty", "position": {"x": 3.0f, "y": 3.0f, "width": 1.0f, "height": 1.0f}}]},

[0278] {"taskId": "3", "tool": "photo_id_generator", "parm": {"background_color": "blue", "photo_width": 1.0f, "photo_heigh": 1.0f}} ]

[0280] The DSL-style task list includes three tasks: Passerby Removal Task, Skin Beautification Task, and ID Photo Generation Task. These three tasks are executed sequentially.

[0281] In one possible approach, for multiple tasks executed sequentially, their order in the task list can be set according to the order in which they appear in the user's instruction. That is, the order of the sequentially executed tasks in the task list corresponds to the order in which the user indicates the tasks. For example, the user instruction is "Remove passersby and beautify my skin." Based on this instruction and image 1, the tasks of removing passersby and beautifying the skin are identified, and both have a task number of 1. In this case, the task of removing passersby can be placed before the task of beautifying the skin in the task list, based on the order in the user's instruction. This provides a reference for subsequent task execution.

[0282] Optionally, the third cloud service may include an intelligent agent microservice. The intelligent agent microservice can interact with the electronic device to receive the task orchestration request sent by the electronic device. Based on the user instruction and image 1, the intelligent agent microservice can orchestrate tasks using a second model to obtain a task list.

[0283] The second model can be a pre-set large cloud-side model. The second model can also be called a task orchestration model. Based on the user instruction and image 1, the second model determines the various tasks that can implement the user instruction on image 1. The second model can perform one or more operations such as task decomposition, task filtering, task merging, and task sorting. For example, the second model can be an LLM (Limited Least Mechanism).

[0284] Step 1711: The third cloud service sends the task list to the target intelligent agent.

[0285] Optionally, the third-party cloud service can send the task list to the agent framework. After receiving the task list, the agent framework can send the task list to the target agent.

[0286] Step 1712: After receiving the task list, the target agent executes the tasks in the task list according to the storage address 1 and storage address 2 of image 1, and obtains the task execution results and the result image.

[0287] In some implementations, after receiving the task list, the target agent can send the task identifier of each task in one or more tasks in the task list to the intelligent voice assistant through the agent management module. Furthermore, during the execution of tasks in the task list, the target agent can send indication messages to the intelligent voice assistant through the agent management module to indicate task nodes (such as task start, task success, task failure, etc.). The intelligent voice assistant can display corresponding reminder messages based on the task identifier and task node of each task to remind the user of the current task execution status, such as reminding the user of the currently executing task and the task processing progress. For example, the reminder message could be... Figure 9 The intelligent voice assistant messages 652, 653, 654, 655, and 656 are shown in (c) to (e) in the diagram. Alternatively, the reminder message can be... Figures 10 to 12 The intelligent voice assistant message 661 is shown.

[0288] Optionally, the operation for the intelligent voice assistant to determine the task processing progress of a certain task can be as follows: After receiving an instruction message indicating that the task should begin execution, the intelligent voice assistant periodically determines the current time elapsed for the task; the current time elapsed is divided by a preset duration to obtain the task processing progress. The preset duration is the estimated time required to execute the task. The preset duration can be set in advance.

[0289] In some cases, if the intelligent voice assistant receives a success message within a preset time after receiving the instruction message indicating the start of the task, it can quickly jump the displayed task progress to 100% to indicate to the user that the task is complete. In other cases, if the intelligent voice assistant receives neither a success message nor a failure message within the preset time after receiving the instruction message, it can maintain the displayed task progress at 99% to await task completion.

[0290] The task execution result may include information indicating whether each task in the task list was executed successfully or failed. The result image may include images obtained from each successfully executed task. In this embodiment, the image obtained after all tasks in the task list have been successfully executed is referred to as Image 2.

[0291] It should be noted that if the task list contains only one task, and the task is executed successfully, the task execution result may include information indicating that the task was executed successfully, and the result image may include image 2 obtained after the task was executed successfully; if the task fails, the task execution result may include information indicating that the task failed, and in this case there is no result image.

[0292] If the task list includes multiple tasks, and all tasks are executed successfully, the task execution result may include information indicating that all tasks were executed successfully, and the result image may include images obtained from each successfully executed task. If some tasks are executed successfully and others fail, the task execution result may include information indicating whether each task was executed successfully or failed, and the result image may include images obtained from each successfully executed task. If all tasks fail, the task execution result may include information indicating that all tasks failed, and in this case, no result image exists.

[0293] In some implementations, step 1712 can be implemented in either of the following two ways: the first way is a single-task execution process; the second way is a multi-task execution process.

[0294] The first method: After receiving the task list, if the task list contains only one task, the target agent calls the task tool corresponding to that task to execute the task and obtains image 2.

[0295] Specifically, the target agent can call the task tool identified by the task tool identifier in the task according to the task parameters in the task, storage address 1 and storage address 2 of image 1, so that the task tool can execute the task and obtain image 2.

[0296] It should be noted that the target intelligent agent may include a task engine, which can interact with the intelligent agent management module and the intelligent agent framework. The target intelligent agent may also include one or more task tools.

[0297] In this case, after the task engine receives the task list, if the task list contains only one task, it can send the task parameters, storage address 1 and storage address 2 of image 1 to the task tool identified by the task tool identifier in that task.

[0298] After receiving the task parameters, storage address 1, and storage address 2 of image 1 from the task, if the task can be executed locally, the task tool retrieves image 1 according to storage address 1, processes image 1 according to the task parameters to obtain image 2, and sends image 2 to the task engine. If the task requires cloud-side processing, the task tool sends a task request to the visual middleware platform, which includes the task parameters and storage address 2 of image 1. Upon receiving the task request, the visual middleware platform sends the task request to the fourth cloud service. Upon receiving the task request, the fourth cloud service retrieves image 1 from the first cloud service according to storage address 2 of image 1, processes image 1 according to the task parameters to obtain image 2, and sends image 2 to the visual middleware platform. Upon receiving image 2, the visual middleware platform sends image 2 to the task tool. Upon receiving image 2, the task tool sends image 2 to the task engine.

[0299] The second method: After receiving the task list, if the task list includes multiple tasks, the target agent sequentially calls the task tools corresponding to each task in the multiple tasks according to the task number in the multiple tasks to execute each task, and obtains image 2.

[0300] The target agent can determine the actual execution order of the multiple tasks based on the task number of each task. Following this execution order, the corresponding task tools are invoked sequentially to execute each task, resulting in Figure 2.

[0301] When two tasks have different task numbers, the actual execution order of the two tasks can be determined based on the order of their task numbers. For example, if task number 1 is in task a and task number 2 is in task b, then the actual execution order of task a and task b can be: task a → task b.

[0302] When two tasks have the same task number, their actual execution order can be determined randomly, or it can be determined based on their order in the task list. For example, if both task c and task d have a task number of 3, their actual execution order could be: task c → task d, or task d → task c. Alternatively, if both task c and task d have a task number of 3, and task c is listed before task d in the task list, their actual execution order could be: task c → task d.

[0303] It should be noted that, for any one of these multiple tasks, calling the corresponding task tool to execute that task is similar to the single-task execution operation in the first method described above. The difference from the first method is that, for the first task in these multiple tasks, its input image is image 1; for any other task in these multiple tasks, its input image is usually the image obtained after the previous task was successfully executed. If all multiple tasks are executed successfully, the image obtained after the last task in these multiple tasks is image 2.

[0304] Another point to note is that, for any one of these tasks, if the task fails, and the next task has the same task number, then the input image of the failed task can be used as the input image for the next task, and execution can continue with that task. If the next task has a different task number, then subsequent tasks will not be executed, and all subsequent tasks will be considered failed; alternatively, the execution of the next task can be determined according to pre-set task rules; or, the input image of the failed task can be used as the input image for the next task, and execution can continue with that task.

[0305] For example, these multiple tasks include a passerby removal task and a skin beautification task. The passerby removal task and the skin beautification task have the same task number. If the passerby removal task fails, the input image of the passerby removal task can be used as the input image of the skin beautification task to continue the skin beautification task.

[0306] For example, these multiple tasks include a skin beautification task and an ID photo generation task. The skin beautification task and the ID photo generation task have different task numbers. If the skin beautification task fails, the ID photo generation task can be skipped, and the ID photo generation task is confirmed as failed. Alternatively, if the task rule indicates that the ID photo generation task can continue even if the skin beautification task fails, then the input image of the skin beautification task can be used as the input image for the ID photo generation task, and the task can continue. However, if the task rule indicates that the ID photo generation task cannot continue even if the skin beautification task fails, then the ID photo generation task can be skipped, and the task is confirmed as failed. Alternatively, if the skin beautification task fails, the input image of the skin beautification task can be used as the input image for the ID photo generation task, and the task can continue.

[0307] For any one of the multiple tasks, if the task is executed successfully, the next task can be executed directly. Alternatively, if the task is executed successfully, and the next task has the same task number as the current task, then the next task can be executed. If the next task has a different task number, and there is no task with the same task number among the multiple tasks, then the next task can be executed. If there is at least one task with the same task number as the current task, and all of these at least one task has executed successfully, then the next task can be executed. If there is at least one task with the same task number as the current task, and one of these at least one task has failed, then subsequent tasks can be discontinued and all subsequent tasks can be determined to have failed, or the execution of the next task can be determined according to pre-set task rules.

[0308] For example, these multiple tasks include a passerby removal task, a skin beautification task, and an ID photo generation task. The passerby removal task and the skin beautification task have the same task number. The skin beautification task and the ID photo generation task have different task numbers. If the passerby removal task fails, the input image of the passerby removal task can be used as the input image of the skin beautification task, and the skin beautification task can continue. If the skin beautification task succeeds, the ID photo generation task can be executed directly; or, because the passerby removal task with the same task number as the skin beautification task failed, the ID photo generation task is not executed, and the ID photo generation task is determined to have failed; or, if the task rule indicates that the ID photo generation task can continue even if the passerby removal task fails, then the input image of the skin beautification task is used as the input image of the ID photo generation task, and the ID photo generation task continues; if the task rule indicates that the ID photo generation task cannot continue even if the passerby removal task fails, then the ID photo generation task is not executed, and the ID photo generation task is determined to have failed.

[0309] After obtaining the task execution result and result image through step 1712 above, the target agent can continue to execute step 1713 to obtain the corresponding response based on the task execution result.

[0310] Step 1713: The target agent sends a dialogue response request 1 to the third cloud service. The dialogue response request 1 includes the result of the task execution.

[0311] For example, dialogue response request 1 may also include dialogue information and the storage address 2 of image 1. This dialogue information may include all user messages and intelligent voice assistant messages during this dialogue, reflecting one or more rounds of dialogue between the user and the intelligent voice assistant. For example, the target agent can obtain this dialogue information from the intelligent voice assistant through the agent management module.

[0312] Optionally, the target agent can send a dialogue response request 1 to the agent framework. After receiving the dialogue response request 1, the agent framework can send the dialogue response request 1 to the third-party cloud service.

[0313] Step 1714: After receiving the dialogue response request 1, the third cloud service generates response 1 based on the task execution result.

[0314] In one possible approach, the third cloud service can generate response 1 solely based on the task execution result. In another possible approach, the third cloud service can retrieve image 1 from the first cloud service based on image 1's storage address 2, and then generate response 1 based on the task execution result, the dialogue information, and image 1.

[0315] Response Message 1 is used to inform the user about the image processing status. Optionally, if the task execution result indicates that the task has failed, Response Message 1 may include the reason for the failure. For example, if the pedestrian removal task fails, Response Message 1 could be "No pedestrians were detected in the current image. Please try again with a different image"; if the skin beautification task fails, Response Message 1 could be "No human figures or the human figures are too small in the current image. Please try again with a different image."

[0316] Optionally, the third cloud service may include an intelligent agent microservice. The intelligent agent microservice can interact with electronic devices to receive dialogue response requests sent by the electronic devices. Based on the task execution result, the intelligent agent microservice can generate a response message through a fifth model. The fifth model can generate the response message based on input data (such as task execution results, or task execution results, dialogue information, and images), enabling natural language dialogue and improving user experience.

[0317] Step 1715: The third cloud service sends reply 1 to the target intelligent agent.

[0318] Optionally, the third cloud service can send response 1 to the agent framework. After receiving response 1, the agent framework can send response 1 back to the target agent.

[0319] Step 1716: After receiving response 1, the target agent sends response 1 and the result image to the agent management module.

[0320] Optionally, the result image sent by the target agent to the agent management module may include images obtained from each successfully executed task, or it may include only the image obtained from the last successfully executed task.

[0321] In some cases, if all tasks in the task list fail, the target agent will only send response 1 to the agent management module.

[0322] Step 1717: After receiving the response message 1 and the result image, the intelligent agent management module sends the response message 1 and the result image to the intelligent voice assistant.

[0323] Step 1718: After receiving the reply message 1 and the result image, the intelligent voice assistant displays the reply message 1 and the result image.

[0324] In this case, the smart voice assistant can display a message with the content "Reply 1".

[0325] For example, if the user command is "Remove passersby and beautify my image," the corresponding task list includes a passerby removal task and a skin beautification task. If both the passerby removal task and the skin beautification task are successfully executed, the resulting image is image D obtained after the skin beautification task is successfully executed. Figure 9 As shown in (f), the intelligent voice assistant can display intelligent voice assistant message 657, the content of which is reply 1, and can also display a thumbnail of image D. If the passerby removal task is successfully executed but the skin beautification task fails, the resulting image is image B obtained after the passerby removal task is successfully executed. In this case, the intelligent voice assistant can display intelligent voice assistant message with reply 1 as its content, and can also display a thumbnail of image B. If both the passerby removal task and the skin beautification task fail, the intelligent voice assistant can display intelligent voice assistant message with reply 1 as its content.

[0326] In some implementations, if the result image includes images from each of multiple successfully executed tasks, the intelligent voice assistant, upon receiving the result image, can display the image from the last successfully executed task. In this case, the intelligent voice assistant can also display images from other successfully executed tasks based on user input.

[0327] For example, these multiple tasks include a pedestrian removal task and a skin beautification task. The target agent first performs the pedestrian removal task to process image 1 to obtain the target image, and then performs the skin beautification task to process the target image to obtain image 2. The target image and image 2 are then sent to the intelligent voice assistant through the agent management module. In this case, the user can selectively view the images obtained after each task is successfully completed. For example, ... Figure 10 As shown in (c), the user can click the first status control 662 displayed by the intelligent voice assistant to trigger the intelligent voice assistant to display the target image; or, the user can click the second status control 663 displayed by the intelligent voice assistant to trigger the intelligent voice assistant to display image 2.

[0328] In this embodiment, after the intelligent voice assistant receives the user command and image 1, it determines the target intelligent agent to process the user command. The target intelligent agent can request the cloud to perform task orchestration based on the user command and image 1, obtaining a task list. The target intelligent agent can execute the tasks in the task list to process image 1, obtaining task execution results and a result image. Then, the target intelligent agent can request the cloud to generate a response message 1 based on the task execution results. Finally, the intelligent voice assistant can display the response message 1 and the result image. This achieves automated processing of image 1 based on user commands, improving image processing efficiency. Furthermore, this end-to-cloud collaborative approach enhances the overall system's response speed and processing capabilities, enabling the target intelligent agent to maintain flexibility, accuracy, and efficiency when processing complex user commands. Additionally, for the user, after issuing a user command to the intelligent voice assistant, they not only receive a text reply but also see a result image related to the user command, making the interaction more natural and personalized. Based on this, the embodiments of this application can reduce the operational complexity for users when processing images, while meeting the needs for fast and accurate image processing in various complex scenarios, thereby greatly improving the user experience.

[0329] To facilitate understanding of the task execution process in step 1712 above, the following will combine... Figures 18 to 22 The above step 1712 will be described by way of example. Wherein, Figures 18 to 20 This is used to illustrate the single-task execution process involved in the first method in step 1712 above; Figures 21 to 22 This is used to illustrate the multi-task execution process involved in the second method in step 1712 above. Specifically, Figure 18 This describes the process of eliminating passersby. In some cases, the task may involve naming people. Figure 19 and Figure 20 This is used to explain the execution process of tasks involving the naming of people. Among them, Figure 19 This is used to illustrate the process of how the names of the people involved can be automatically clarified through a learning memory platform and image library application during task execution. Figure 20 The task execution process is used to clarify the names of the people involved. Figure 21 and Figure 22 This is used to explain the execution process of the "Pedestrian Elimination" and "Skin Beautification" tasks. Among them, Figure 21 This is used to explain the execution process of the skin beautification task when the passerby elimination task is successfully completed. Figure 22 This is used to explain the execution process of the skin beautification task when the task of eliminating passersby fails.

[0330] The following example uses the "Passerby Elimination" task in the task list as an example, which requires cloud-side processing. Figure 18The execution process of the tasks in this task list is illustrated by the following example:

[0331] Figure 18 This is a flowchart illustrating a task execution process provided in an embodiment of this application. See also... Figure 18 The process may include steps 1801 to 1814.

[0332] Step 1801: The task engine sends the task parameters, storage address 1 and storage address 2 of image 1 from the pedestrian removal task to the removal tool.

[0333] Step 1802: After receiving the task parameters, storage address 1 and storage address 2 of image 1, the removal tool sends a pedestrian recognition request to the visual platform. The pedestrian recognition request includes the task parameters and storage address 2 of image 1.

[0334] Optionally, the removal tool can send an instruction message to the task engine to indicate the start of the pedestrian recognition operation in the pedestrian removal task. Upon receiving this instruction message, the task engine can send it to the intelligent voice assistant through the agent management module. Upon receiving the instruction message, the intelligent voice assistant can display a reminder message to inform the user that pedestrian recognition is in progress, and to inform the user of the current pedestrian removal processing progress (i.e., the overall processing progress of the pedestrian removal task) or the current pedestrian recognition progress (i.e., the processing progress of the pedestrian recognition operation). For example, this reminder message could be... Figure 6 The intelligent voice assistant messages 622 and 623 are shown in (c) in the figure.

[0335] Step 1803: After receiving the pedestrian recognition request, the visual platform sends the pedestrian recognition request to the fourth cloud service. The pedestrian recognition request includes the task parameters and the storage address 2 of image 1.

[0336] Step 1804: After receiving the pedestrian recognition request, the fourth cloud service retrieves image 1 from the first cloud service according to the storage address 2 of image 1, performs pedestrian recognition on image 1 according to the task parameters, and obtains the mask of the pedestrian in image 1.

[0337] This mask image can indicate the pixel position of a passerby in image 1.

[0338] As an example, the operation of the fourth cloud service in recognizing pedestrians in image 1 based on the task parameters can be as follows: The fourth cloud service, based on the task parameters, performs pedestrian detection in image 1 using the third model, obtaining a bounding box in image 1. This bounding box includes the coordinate information of the pedestrian in image 1, indicating the region where the pedestrian is located in image 1. Based on this bounding box, the fourth cloud service segments the target instance from image 1 using the fourth model, obtaining a mask image of the pedestrian in image 1.

[0339] The third model is a pre-set cloud-based large model. This third model is used for object detection. Specifically, it can locate the corresponding target in the input image based on the input text description (such as the task parameters). For example, the third model can be a multimodal large language model (MLLM), etc.

[0340] The fourth model is a pre-set large cloud-side model. This model is used for instance segmentation. A target instance refers to an independent target in image 1. For example, assuming image 1 contains two pedestrians, then the target instances in image 1 include two independent pedestrians, such as instance 1 and instance 2. After instance segmentation, two mask images are obtained, used to label the pixel positions of these two pedestrians respectively.

[0341] It should be noted that the above methods implement object detection and instance segmentation using different models. In some cases, object detection and instance segmentation can also be achieved simultaneously using a single model. In this case, after the fourth cloud service inputs image 1 and the task parameters into the model, the model can output mask images of each passerby in image 1.

[0342] Step 1805: The fourth cloud service sends the mask image to the visual middleware platform.

[0343] Step 1806: After receiving the mask image, the visual platform sends the mask image to the removal tool.

[0344] In some cases, the locally stored image 1 may differ slightly from the cloud-stored image 1. For example, the locally stored image may be the original (e.g., 4000×3000 pixels), while the cloud-stored image may be a compressed image (e.g., 800×600 pixels), resulting in different resolutions.

[0345] In this scenario, the visual platform can store resolution scaling rules between local images and cloud-based images. After receiving the mask image from the fourth cloud service, the visual platform can scale the mask image according to these scaling rules and then send the scaled mask image to the removal tool.

[0346] Step 1807: After receiving the mask image, the removal tool sends a pedestrian removal request to the visual platform. The pedestrian removal request includes the storage address 1 of image 1 and the mask image.

[0347] Optionally, the removal tool can send an instruction message to the task engine to indicate the start of the pedestrian removal operation in the pedestrian removal task. Upon receiving the instruction message, the task engine can send it to the intelligent voice assistant through the intelligent agent management module. Upon receiving the instruction message, the intelligent voice assistant can display a reminder message to inform the user that pedestrian removal is in progress, and to inform the user of the current pedestrian removal processing progress (i.e., the overall processing progress of the pedestrian removal task) or the current pedestrian removal progress (i.e., the processing progress of the pedestrian removal operation). For example, the reminder message could be... Figure 6 The intelligent voice assistant messages 624 and 623 are shown in (d) in the figure.

[0348] Step 1808: After receiving the pedestrian removal request, the visual platform obtains image 1 according to the storage address 1 of image 1, and obtains the pedestrian image from image 1 according to the mask image.

[0349] The image of the passerby is the area in Image 1 that needs to be de-identified. This passerby image can be a portion of Image 1, including the pixel positions indicated by the mask. For example, the image area corresponding to the passerby image in Image 1 can be a rectangular region. The image area corresponding to the passerby image can be slightly larger than the image area indicated by the mask. For instance, the size of the image area corresponding to the passerby image can be 1.1 times the size of the image area indicated by the mask.

[0350] Step 1809: The visual middleware sends a pedestrian removal request to the fourth cloud service. The pedestrian removal request includes the pedestrian's image.

[0351] In this embodiment, when requesting pedestrian removal from the cloud side, it is not necessary to send the entire image 1 to the cloud side; instead, only a portion of the image 1 containing the pedestrian (i.e., the pedestrian image) needs to be sent. This not only saves transmission resources but also reduces the processing load on the cloud side and increases processing speed.

[0352] Step 1810: After receiving the pedestrian removal request, the fourth cloud service performs removal processing on the pedestrian image to obtain image 3.

[0353] As an example, the operation of the fourth cloud service to remove the pedestrian image can be as follows: the fourth cloud service removes the pedestrian image through the sixth model to obtain image 3.

[0354] The sixth model is a pre-set large cloud-based model. This model can remove people from the pedestrian image and fill the removed areas with image content consistent with the background. For example, the sixth model could be an artificial intelligence generated content (AIGC) model, etc.

[0355] Step 1811: The fourth cloud service sends image 3 to the visual middleware platform.

[0356] Step 1812: After receiving image 3, the vision platform fuses image 3 with image 1 according to the mask image to obtain image 2.

[0357] Fusing Image 3 with Image 1 based on the mask image means stitching the image region in Image 3 indicated by the mask image with other image regions in Image 1 other than the image region indicated by the mask image, to obtain Image 2. In this case, the image content of the image region in Image 2 indicated by the mask image is the same as the image content of the corresponding image region in Image 3, and the image content of other image regions in Image 2 other than the image region indicated by the mask image is the same as the image content of the corresponding image regions in Image 1.

[0358] Step 1813: The visual platform sends image 2 to the elimination tool.

[0359] Step 1814: After receiving image 2, the removal tool sends image 2 to the task engine.

[0360] Once the task engine receives image 2, it can determine that the task of eliminating passersby has been successfully completed.

[0361] In this embodiment, for the pedestrian removal task, the removal tool in the target agent can perform pedestrian recognition on image 1 via the cloud to obtain a mask image of the pedestrian. Then, the visual platform can obtain the pedestrian image from image 1 based on the mask image and perform removal processing on the pedestrian image via the cloud to obtain image 3. Afterwards, the visual platform can fuse image 3 and image 1 to obtain image 2. Thus, during pedestrian recognition, the powerful processing capabilities of the cloud can efficiently and accurately identify pedestrians in image 1. Furthermore, during pedestrian removal, only a portion of the image region containing the pedestrian (i.e., the pedestrian image) in image 1 is sent to the cloud for processing, thereby saving transmission resources, reducing the processing pressure on the cloud, and improving processing speed. Through this end-to-cloud collaborative approach, the pedestrian removal task can be effectively completed, improving the quality and efficiency of image processing.

[0362] In some cases, the tasks in this task list may involve names of people. In such cases, it's necessary to identify the specific person corresponding to that name in Image 1 to complete the task. The following example uses the task "Eliminate passersby while keeping Li Si," which requires cloud-based processing, as an example. Figure 19 An example is provided to illustrate this situation:

[0363] Figure 19 This is a flowchart illustrating a task execution process provided in an embodiment of this application. See also... Figure 19 The process may include steps 1901 to 1925.

[0364] Step 1901: The task engine sends the task parameters, storage address 1 and storage address 2 of image 1 to the elimination tool in the task "Eliminate passersby and keep Li Si".

[0365] Step 1902: After receiving the task parameters, storage address 1 and storage address 2 of image 1, the elimination tool sends a query request 1 to the learning and memory platform if the task parameters contain a person's name. The query request 1 includes the person's name "Li Si".

[0366] The form of address can include one or more of the following: name, title, and appellation. For example, a name can be a person's full name, such as Zhang San or Li Si, or a nickname, such as "baby." A title can be a kinship term, such as "Mom," "Dad," or "Daughter," or a professional title, such as "Dr. Li" or "Professor Wang." An appellation can be a title of respect earned for a specific reason, such as "award winner" or "artist."

[0367] Step 1903: After receiving query request 1, the learning and memory platform queries the character identifier corresponding to the character name "Li Si".

[0368] This person identifier can also be called a person tag, such as Tag ID.

[0369] Optionally, the learning and memory platform can query the character identifier corresponding to the name "Li Si" from a preset database.

[0370] The pre-defined database can store the correspondence between character titles and character identifiers. In this correspondence, one character identifier can correspond to one or more character titles. For example, the titles "daughter" and "baby" can correspond to the same character identifier, and the titles "mother" and "Zhang San" can correspond to the same character identifier.

[0371] Step 1904: When the learning and memory platform finds the character identifier corresponding to the character name "Li Si", it sends the character identifier to the elimination tool.

[0372] Step 1905: After receiving the person's identifier, the removal tool sends a query request 2 to the image library application. The query request 2 includes the person's identifier and the storage address 1 of image 1.

[0373] Step 1906: After receiving query request 2, the image library application obtains the target face bounding box corresponding to the person's identifier in image 1 based on the storage address 1 of image 1.

[0374] The target face bounding box includes the coordinate information of the face corresponding to the person's name "Li Si" in the locally stored image 1, which is used to indicate the area where the face is located in the locally stored image 1.

[0375] The gallery application manages an image set associated with a person's identifier, where each image in the set contains the face identified by that identifier. A corresponding face bounding box exists for that person's identifier in each image set. The gallery application can determine whether the image set includes image 1 based on its storage address 1. If image 1 is included in the image set, the gallery application can retrieve the face bounding box corresponding to that person's identifier in image 1 as the target face bounding box.

[0376] Step 1907: The gallery application sends the target face frame to the removal tool.

[0377] Step 1908: After receiving the target face bounding box, the removal tool sends a person recognition request to the visual platform. The person recognition request includes the storage address 2 of image 1 and the target face bounding box.

[0378] Step 1909: After receiving the person recognition request, the visual platform sends the person recognition request to the fourth cloud service. The person recognition request includes the storage address 2 of image 1 and the target face bounding box.

[0379] In some cases, the locally stored image 1 may differ slightly from the cloud-stored image 1. In such cases, the visual platform can store coordinate transformation rules between the local and cloud-stored images. After receiving the target face bounding box from the removal tool, the visual platform can transform the target face bounding box according to these coordinate transformation rules and send the transformed target face bounding box to the fourth cloud service. The transformed target face bounding box includes the coordinate information of the face corresponding to the person named "Li Si" in the cloud-stored image 1, indicating the area where the face is located in the cloud-stored image 1.

[0380] Step 1910: After receiving the person recognition request, the fourth cloud service retrieves image 1 from the first cloud service according to the storage address 2 of image 1, performs human detection on image 1 according to the target face bounding box, and obtains the target human bounding box in image 1.

[0381] The target human body bounding box includes the coordinate information of the human body to which the face corresponding to the target face bounding box belongs in image 1, which is used to indicate the area where the human body is located in image 1.

[0382] Step 1911: The fourth cloud service sends the target human body bounding box to the visual middleware platform.

[0383] Step 1912: After receiving the target human body bounding box, the visual platform sends the target human body bounding box to the elimination tool.

[0384] Step 1913: After receiving the target human bounding box, the elimination tool sends a pedestrian recognition request to the vision platform. The pedestrian recognition request includes the task parameters, the storage address 2 of image 1, and the target human bounding box.

[0385] Optionally, the removal tool can send an instruction message to the task engine to indicate the start of the pedestrian recognition operation in the pedestrian removal task. Upon receiving this instruction message, the task engine can send it to the intelligent voice assistant through the agent management module. Upon receiving the instruction message, the intelligent voice assistant can display a reminder message to inform the user that pedestrian recognition is in progress, and to inform the user of the current pedestrian removal processing progress (i.e., the overall processing progress of the pedestrian removal task) or the current pedestrian recognition progress (i.e., the processing progress of the pedestrian recognition operation).

[0386] Step 1914: After receiving the pedestrian recognition request, the visual platform sends the pedestrian recognition request to the fourth cloud service. The pedestrian recognition request includes the task parameters, the storage address 2 of image 1, and the target human bounding box.

[0387] Step 1915: After receiving the pedestrian recognition request, the fourth cloud service retrieves image 1 from the first cloud service according to the storage address 2 of image 1, performs pedestrian recognition on image 1 according to the task parameters and the target human bounding box, and obtains the mask image of the pedestrian in image 1.

[0388] As an example, the operation of the fourth cloud service in recognizing pedestrians in image 1 based on the task parameters and the target human bounding box can be as follows: The fourth cloud service uses the third model to detect pedestrians in image 1 based on the task parameters and the target human bounding box, obtaining the target bounding box in image 1. This target bounding box includes the coordinate information of the pedestrian in image 1, indicating the region where the pedestrian is located in image 1. Based on the target bounding box, the fourth cloud service uses the fourth model to segment the target instance from image 1, obtaining the mask image of the pedestrian in image 1.

[0389] The third model can output the target bounding boxes of all passersby in Image 1, excluding the person corresponding to the target bounding box, based on the input task parameters and the target human bounding box.

[0390] It should be noted that the above methods implement object detection and instance segmentation using different models. In some cases, object detection and instance segmentation can also be achieved simultaneously using a single model. In this case, after the fourth cloud service inputs image 1, the task parameters, and the target human bounding box into the model, the model can output mask images of all passersby in image 1 except for the person corresponding to the target human bounding box.

[0391] Step 1916: The fourth cloud service sends the mask image to the visual middleware platform.

[0392] Step 1917: After receiving the mask image, the visual platform sends the mask image to the removal tool.

[0393] Step 1918: After receiving the mask image, the removal tool sends a pedestrian removal request to the visual platform. The pedestrian removal request includes the storage address 1 of image 1 and the mask image.

[0394] Step 1919: After receiving the pedestrian removal request, the visual platform obtains image 1 according to the storage address 1 of image 1, and obtains the pedestrian image from image 1 according to the mask image.

[0395] Step 1920: The visual middleware sends a pedestrian removal request to the fourth cloud service. The pedestrian removal request includes the pedestrian's image.

[0396] Step 1921: After receiving the pedestrian removal request, the fourth cloud service performs removal processing on the pedestrian image to obtain image 4.

[0397] Step 1922: The fourth cloud service sends image 4 to the visual middleware platform.

[0398] Step 1923: After receiving image 4, the vision platform fuses image 4 with image 1 according to the mask image to obtain image 2.

[0399] Step 1924: The visual platform sends image 2 to the elimination tool.

[0400] Step 1925: After receiving image 2, the removal tool sends image 2 to the task engine.

[0401] The operations described in steps 1916 to 1925 are similar to those described in steps 1805 to 1814. Please refer to the above text for details. Figure 18 The relevant descriptions in the implementation methods will not be repeated here.

[0402] In this embodiment, for the task of eliminating passersby involving person names, the elimination tool in the target agent can first query the person identifier corresponding to the person name in the learning and memory platform, and then query the target face bounding box corresponding to the person identifier in image 1 from the image library application. Afterwards, the elimination tool can obtain the target body bounding box corresponding to the target face bounding box in image 1 via cloud-based processing, and then perform passerby recognition on image 1 based on the target body bounding box via cloud-based processing to obtain a passerby mask image. Then, the visual platform can obtain the passerby image from image 1 based on the mask image, and perform elimination processing on the passerby image via cloud-based processing to obtain image 4. Afterwards, the visual platform can merge image 4 and image 1 to obtain image 2. Thus, by associating person names with person identifiers, the specific person corresponding to the person name in image 1 can be quickly and accurately identified without user clarification, thereby accurately and automatically completing the passerby elimination task, reducing the complexity of user operations, and improving image processing efficiency.

[0403] It should be noted that in step 1903 above, if the learning and memory platform does not find a character identifier corresponding to the name "Li Si," then character clarification needs to be performed via cloud-based methods. The following section will explain... Figure 20 To clarify:

[0404] Figure 20 This is a flowchart illustrating a task execution process provided in an embodiment of this application. See also... Figure 20 The process may include steps 2001 to 2016.

[0405] Step 2001: The task engine sends the task parameters, storage address 1 and storage address 2 of image 1 to the elimination tool in the task "Eliminate passersby and keep Li Si".

[0406] Step 2002: After receiving the task parameters, storage address 1 and storage address 2 of image 1, if the task parameters contain a person's name, the elimination tool sends a query request 1 to the learning and memory platform. The query request 1 includes the person's name "Li Si".

[0407] Step 2003: After receiving query request 1, the learning and memory platform queries the character identifier corresponding to the character name "Li Si".

[0408] Step 2004: If the learning and memory platform cannot find the character identifier corresponding to the name "Li Si", it sends a query failure message to the elimination tool.

[0409] This query failure message indicates that the query for the character name "Li Si" in the task "Eliminate passersby while retaining Li Si" failed.

[0410] Step 2005: After receiving the query failure information, the elimination tool sends the query failure information to the task engine.

[0411] Step 2006: After receiving the query failure information, the task engine sends a dialogue response request 2 to the third cloud service. The dialogue response request 2 includes the query failure information, dialogue information, and the storage address 2 of image 1.

[0412] This dialogue information includes all user messages and intelligent voice assistant messages during the current conversation, reflecting one or more rounds of dialogue between the user and the intelligent voice assistant. For example, the task engine can obtain this dialogue information from the intelligent voice assistant through the agent management module.

[0413] Optionally, the task engine can send dialogue response request 2 to the agent framework. After receiving dialogue response request 2, the agent framework can send dialogue response request 2 to the third-party cloud service.

[0414] Step 2007: After receiving the dialogue reply request 2, the third cloud service retrieves image 1 from the first cloud service according to the storage address 2 of image 1, and generates reply 2 based on the query failure information, the dialogue information and image 1.

[0415] Reply 2 is used to ask the user which specific person in image 1 corresponds to the name "Li Si".

[0416] Optionally, the third cloud service may include an intelligent agent microservice. The intelligent agent microservice can interact with electronic devices to receive dialogue response requests 2 sent by the electronic devices. Based on the query failure information, the dialogue information, and image 1, the intelligent agent microservice can generate a response 2 using a fifth model.

[0417] The fifth model can be a pre-set large cloud-side model. It can also be called a response generation model. The fifth model can generate responses based on input query failure information, dialogue information, images, etc., to achieve dialogue clarification and reply functions. For example, the fifth model can be an LLM (Local Level Model), etc.

[0418] For example, if the dialogue information includes the user message "Eliminate the person next to Li Si," the fifth model can generate a response 2 for clarifying the person based on the query failure information, the dialogue information, and the image content of Image 1. For instance, response 2 could be "Please tell me what Li Si's characteristics are, such as the color of his clothes, his posture, and his position in the picture," prompting the user to provide specific characteristics of Li Si; or, response 2 could be related to the characteristics of the main person in Image 1, such as "Is Li Si the person wearing red in the picture?" so that the user can answer quickly.

[0419] Step 2008: The third cloud service sends response 2 to the task engine.

[0420] Optionally, the third cloud service can send response 2 to the intelligent agent framework. After receiving response 2, the intelligent agent framework can send response 2 to the task engine.

[0421] Step 2009: After receiving reply 2, the task engine sends reply 2 to the intelligent voice assistant through the intelligent agent management module.

[0422] Step 2010: After receiving reply 2, the intelligent voice assistant displays reply 2.

[0423] In this case, the smart voice assistant can display a message with the content of reply 2.

[0424] Step 2011: After receiving the user's input statement, the intelligent voice assistant sends the user's statement to the task engine through the intelligent agent management module.

[0425] After seeing response 2, the user can enter a user statement to respond. This user statement can be entered by the user via voice or text into the input box of the intelligent voice assistant.

[0426] Step 2012: After receiving the user's statement, the task engine generates character description information based on reply 2 and the user's statement.

[0427] The character description information describes the characteristics of the person referred to as "Li Si" in Image 1. For example, if the response 2 is "Please tell me what Li Si's characteristics are, such as the color of his clothes, his posture, and his position in the picture," and the user's statement is "Li Si wears red clothes," the task engine can generate the character description information: Li Si is the person wearing red clothes. Or, if the response 2 is "Is Li Si the person wearing red clothes in the picture?" and the user's statement is "Yes," the task engine can generate the character description information: Li Si is the person wearing red clothes.

[0428] Step 2013: The task engine sends the character description information to the elimination tool.

[0429] Step 2014: After receiving the person description information, the removal tool sends a person recognition request to the visual platform. The person recognition request includes the storage address 2 of image 1 and the person description information.

[0430] Step 2015: After receiving the person recognition request, the visual platform sends the person recognition request to the fourth cloud service. The person recognition request includes the storage address 2 of image 1 and the person description information.

[0431] Step 2016: After receiving the person recognition request, the fourth cloud service retrieves image 1 from the first cloud service according to the storage address 2 of image 1, performs human detection on image 1 according to the person description information, and obtains the target human bounding box in image 1.

[0432] The target human bounding box includes the coordinate information of the human body corresponding to the description information of the person in image 1, which is used to indicate the area where the human body is located in image 1.

[0433] The subsequent steps are the same as steps 1911 to 1925 above, and will not be repeated here.

[0434] In this embodiment, for the task of eliminating passersby involving names of people, the elimination tool in the target agent can first query the person identifier corresponding to the name in the learning and memory platform. If the target agent does not find the person identifier corresponding to the name, it can generate a response 2 via the cloud to ask the user which specific person the name "Li Si" corresponds to in image 1. Then, the target agent can generate person description information based on the response 2 and the user's response to the response 2, and then obtain the target human bounding box corresponding to the person description information in image 1 via the cloud. Based on the target human bounding box, it can then perform passerby recognition on image 1 via the cloud to obtain a mask image of the passerby. The visual platform can obtain the passerby image from image 1 based on the mask image, and perform elimination processing on the passerby image via the cloud to obtain image 4. Then, image 4 and image 1 are merged to obtain image 2. In this way, by actively interacting with users through reply 2 and inquiring about the specific characteristics of a particular person (such as "Li Si"), the interaction between users and the system can be enhanced, and the system can also help the system accurately understand user needs, thereby ensuring the correct execution of the passerby elimination task to a certain extent.

[0435] The following examples use the tasks in the task list, namely the "Passerby Elimination Task" and the "Skin Beautification Task," as an example. The "Passerby Elimination Task" requires cloud-based processing, while the "Skin Beautification Task" can be processed locally. Figure 21 The execution process of the tasks in this task list is illustrated by the following example:

[0436] Figure 21 This is a flowchart illustrating a task execution process provided in an embodiment of this application. See also... Figure 21 The process may include the following steps 2101 to 2104.

[0437] Step 2101: The task engine calls the elimination tool to perform the passerby elimination task.

[0438] The operation of step 2101 can be referred to steps 1801 to 1814 above, and will not be repeated here.

[0439] The image obtained after the successful completion of the pedestrian elimination task will be referred to as image 5.

[0440] Step 2102: If the task of eliminating passersby is successfully executed, the task engine sends the task parameters and image 5 from the skin beautification task to the skin beautification tool.

[0441] Step 2103: After receiving the task parameters and image 5, the skin beautification tool processes image 5 according to the task parameters to obtain image 2.

[0442] Step 2104: The skin beautification tool sends image 2 to the task engine.

[0443] Once the task engine receives image 2, it can determine that the skin beautification task has been successfully executed.

[0444] In some implementations, the task engine can continue executing the skin beautification task even if the pedestrian removal task fails. The following section will combine... Figure 22 Please explain this situation:

[0445] Figure 22 This is a flowchart illustrating a task execution process provided in an embodiment of this application. See also... Figure 22 The process may include steps 2201 to 2204.

[0446] Step 2201: The task engine calls the elimination tool to perform the passerby elimination task.

[0447] Step 2202: If the task of removing passersby fails, the task engine sends the task parameters, storage address 1 and storage address 2 of image 1 from the skin beautification task to the skin beautification tool.

[0448] Step 2203: After receiving the task parameters, storage address 1 and storage address 2 of image 1, the skin beautification tool obtains image 1 according to storage address 1 of image 1, processes image 1 according to the task parameters, and obtains image 6.

[0449] Step 2204: The skin beautification tool sends image 6 to the task engine.

[0450] Once the task engine receives image 6, it can determine that the skin beautification task has been successfully executed.

[0451] In this embodiment, the task engine can sequentially call the task tools corresponding to each of the multiple tasks, so that each task tool can execute its corresponding task. This multi-task execution method can meet the image processing needs of complex user commands. Furthermore, if a task fails, subsequent tasks can continue to execute, thus avoiding the entire process from stalling due to the failure of a single task and improving system stability.

[0452] The above text Figures 17 to 22 The implementation method describes the image processing procedure when the agent processing the user command is a single agent. In some cases, there may be multiple agents for processing the user command; the image processing procedure in this case will be described below.

[0453] As described in steps 1701 to 1705 above, the intelligent voice assistant obtains the user command and image 1, and stores image 1 in the first cloud service, obtaining the storage address 2 of image 1. Afterwards, the intelligent voice assistant can send the user command to the second cloud service through its voice engine.

[0454] After receiving the user command, the second cloud service can parse it to obtain the identifiers of multiple intelligent agents, the calling order of these agents, and a portion of the instructions that each agent needs to process within the user command (hereinafter referred to as the target instruction). The second cloud service can then send agent information to the intelligent voice assistant via its voice engine. This agent information may include the identifiers of the multiple intelligent agents, the calling order of these agents, and the target instruction corresponding to each agent.

[0455] For example, the user command is "Remove passersby and beautify the image, then generate an animated GIF." After parsing the user command, the second cloud service can obtain the identifiers of the image editing agent and the video editing agent, and the calling order is from the image editing agent to the video editing agent. The image editing agent is used to process the target command "Remove passersby and beautify the image," and the video editing agent is used to process the target command "Then generate an animated GIF."

[0456] After receiving the agent information, the intelligent voice assistant can sequentially invoke each of the multiple agents to execute the corresponding target instruction, according to the invocation order. The process of the intelligent voice assistant invoking an agent to execute a corresponding target instruction is similar to that described above. Figures 17 to 22 The process of the intelligent voice assistant calling the target intelligent agent to execute user commands as described in the embodiments is similar, and will not be repeated in the embodiments of this application.

[0457] After executing their respective target instructions, each intelligent agent can send response 1 and a result image to the intelligent voice assistant. Upon receiving response 1 and the result image from each of the multiple intelligent agents, the intelligent voice assistant can integrate all the response 1s sent by the multiple agents to obtain response 3. The intelligent voice assistant can then display response 3 and the result image.

[0458] In this embodiment, a multi-agent collaboration approach enables the intelligent voice assistant to execute complex user commands more intelligently and efficiently. This improves the system's processing power and flexibility, allowing it to meet diverse image processing needs and significantly enhancing the user experience.

[0459] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0460] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0461] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the various method embodiments described above.

[0462] This application also provides a computer program product that, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.

[0463] This application also provides a chip system including a processor coupled to a memory. The processor executes a computer program stored in the memory to implement the steps of any method embodiment of this application. The chip system can be a single chip or a chip module composed of multiple chips.

[0464] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave) means. The computer-readable storage medium can be any available medium accessible to a computer, or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes, etc.), optical media (such as digital versatile discs (DVDs), etc.) or semiconductor media (such as solid state disks (SSDs), etc.).

[0465] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0466] The above description is an optional implementation provided by this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the technical scope disclosed in this application should be included within the protection scope of this application.

Claims

1. An image processing method, characterized in that, The method includes: When the first interface of the intelligent voice assistant is displayed, image editing instructions are obtained, and the image editing instructions correspond to at least two tasks; In response to the image editing instruction, acquire the first image; The second interface of the intelligent voice assistant is displayed. During the execution of each of the at least two tasks, a first reminder message is displayed on the second interface. The first reminder message is used to remind users of the identifier and progress of the task being executed in the at least two tasks. After the execution of at least two tasks is completed, a first response and a second image are displayed on the second interface. The first response indicates the processing result of the first image, and the second image is the image obtained after processing the first image through at least one of the at least two tasks.

2. The method as described in claim 1, characterized in that, The at least two tasks include at least two of the following: elimination task, skin beautification task, background removal task, ID photo generation task, element addition task, and basic editing task.

3. The method as described in claim 1 or 2, characterized in that, The method further includes: Display a large image page of the gallery application, in which the first image is displayed; The input box of the intelligent voice assistant is displayed in the large image page, and the first interface is the interface that displays the input box; The step of acquiring the first image in response to the image editing instruction includes: In response to the image editing instruction, the first image is retrieved according to its local storage address.

4. The method as described in claim 1 or 2, characterized in that, The method further includes: A third interface is displayed, in which a first thumbnail of the first image is displayed; In response to receiving an operation applied to a first thumbnail of the first image, a target element is displayed, the target element displaying the content of the first image; In response to detecting a drag operation on the target element, the identifier of the intelligent voice assistant is displayed; In response to receiving an operation where the target element is dragged to the location of the identifier of the intelligent voice assistant and released, the chat window of the intelligent voice assistant is displayed, and a second thumbnail of the first image is displayed in the chat window. The first interface is the interface that displays the chat window. The step of acquiring the first image in response to the image editing instruction includes: In response to the image editing instruction, the first image is retrieved according to its local storage address.

5. The method as described in any one of claims 1 to 4, characterized in that, The first interface displays one or more AI editing controls. Before obtaining image editing instructions, the interface further includes: Receive operations applied to any one of the one or more AI editing controls; The image editing instructions include: The image editing instructions are obtained based on the received operation applied to the AI ​​editing control.

6. The method according to any one of claims 1 to 5, characterized in that, The first reminder message includes at least two status controls, each of which corresponds to one of the at least two tasks. Each status control is used to indicate the task status of the corresponding task, and the task status includes one or more of the following: not started, in progress, successful, and failed.

7. The method as described in claim 6, characterized in that, After displaying the first reply and the second image on the second interface, the method further includes: For any one of the at least two status controls, if the task status indicated by the status control is successful, in response to receiving an operation performed on the status control, the image obtained through the task processing corresponding to the status control is displayed in the second interface.

8. The method as described in any one of claims 1 to 7, characterized in that, The step of displaying the first reply and the second image on the second interface includes: The first reply, the second image, and AI image editing controls are displayed on the second interface. The method further includes: In response to receiving an operation applied to the AI ​​image editing control, multiple AI editing controls are displayed on the second interface; For any one of the plurality of AI editing controls, in response to receiving an operation applied to the AI ​​editing control, a second response and the third image are displayed in the second interface. The second response is used to indicate the processing result of the second image, and the third image is the image obtained after processing the second image according to the user instruction corresponding to the AI ​​editing control.

9. The method according to any one of claims 1 to 7, characterized in that, After displaying the first reply and the second image on the second interface, the method further includes: In response to receiving an operation applied to the first image, an editing interface for the first image is displayed, the editing interface including AI image editing controls; In response to receiving an operation applied to the AI ​​editing control, multiple AI editing controls are displayed in the editing interface; For any one of the plurality of AI editing controls, in response to receiving an operation applied to the AI ​​editing control, the second interface is displayed, in which a third response and the fourth image are displayed. The third response is used to indicate the processing result of the first image, and the fourth image is an image obtained after processing the first image according to the user instruction corresponding to the AI ​​editing control.

10. The method according to any one of claims 1 to 9, characterized in that, After acquiring the first image, the process further includes: The intelligent voice assistant sends the image editing command and the storage address of the first image to the target intelligent agent; After receiving the image editing instruction and the storage address of the first image, the target intelligent agent obtains the at least two tasks according to the image editing instruction; The target agent executes the at least two tasks according to the storage address of the first image, and obtains the task execution result and the second image; The target agent obtains the first response based on the task execution result; The target intelligent agent sends the first reply and the second image to the intelligent voice assistant.

11. The method as described in claim 10, characterized in that, The method further includes: During the execution of the at least two tasks, the target intelligent agent sends an instruction message to the intelligent voice assistant to indicate task nodes, wherein the task nodes are used to indicate whether the task has started, the task has been successfully executed, or the task has failed. Displaying the first reminder message on the second interface includes: After receiving the instruction message, the intelligent voice assistant displays the first reminder message according to the task node.

12. The method as described in claim 10 or 11, characterized in that, The storage address of the first image includes a local storage address and a cloud storage address. Before the intelligent voice assistant sends the image editing command and the storage address of the first image to the target intelligent agent, it also includes: The intelligent voice assistant sends the first image to the first cloud service; The intelligent voice assistant receives a cloud storage address sent by the first cloud service, which is the storage address of the first image in the first cloud service.

13. The method as described in any one of claims 10 to 12, characterized in that, Before the intelligent voice assistant sends the image editing command and the storage address of the first image to the target intelligent agent, it also includes: When the image editing command is triggered by an AI editing control, the intelligent voice assistant obtains the intelligent agent identifier corresponding to the AI ​​editing control, and the intelligent agent identifier corresponding to the AI ​​editing control is the identifier of the target intelligent agent; or... The intelligent voice assistant sends the image editing command to the second cloud service; and receives the identifier of the target intelligent agent sent by the second cloud service.

14. The method according to any one of claims 10 to 13, characterized in that, The target agent obtains the at least two tasks according to the image editing instructions, including: The target intelligent agent sends a task orchestration request to the third cloud service, the task orchestration request including the image editing instructions; The target intelligent agent receives the at least two tasks sent by the third cloud service.

15. The method as described in any one of claims 10 to 14, characterized in that, The target agent includes a task engine and one or more task tools. The target agent executes at least two tasks based on the storage address of the first image, including: For the first of the at least two tasks, the task engine calls the task tool corresponding to the first task to execute the first task based on the storage address of the first image; For any one of the at least two tasks other than the first task, the task engine calls the task tool corresponding to the task to execute the task based on the image obtained from the task executed before the task.

16. The method as described in claim 15, characterized in that, The storage address of the first image includes a local storage address and a cloud storage address. The first task is a pedestrian removal task, and the task tool corresponding to the pedestrian removal task is a removal tool. The task engine calls the task tool corresponding to the first task to execute the first task based on the storage address of the first image, including: The task engine sends the pedestrian removal task and the storage address of the first image to the removal tool; After receiving the pedestrian removal task and the storage address of the first image, the removal tool sends a pedestrian recognition request to the visual platform. The pedestrian recognition request includes the pedestrian removal task and the cloud storage address. After receiving the pedestrian recognition request, the visual platform sends the pedestrian recognition request to the fourth cloud service; after receiving the mask image of the pedestrian in the first image sent by the fourth cloud service, it sends the mask image to the removal tool. After receiving the mask image, the removal tool sends a pedestrian removal request to the visual platform. The pedestrian removal request includes the local storage address and the mask image. After receiving the pedestrian removal request, the visual platform obtains the first image according to the local storage address, extracts the pedestrian image from the first image according to the mask image, and sends the pedestrian removal request including the pedestrian image to the fourth cloud service; after receiving the fifth image sent by the fourth cloud service, it merges the fifth image with the first image according to the mask image to obtain the sixth image, and sends the sixth image to the removal tool. After receiving the sixth image, the elimination tool sends the sixth image to the task engine.

17. The method as described in claim 15, characterized in that, The task engine invokes the task tool corresponding to the first task based on the storage address of the first image to execute the first task, including: The task engine sends the first task to the task tool corresponding to the first task. After receiving the first task, the task tool corresponding to the first task sends a first query request to the learning and memory platform if a person's name exists in the first task. The first query request includes the person's name. If it receives a person's identifier from the learning and memory platform, it sends a second query request to the image library application. The second query request includes the person's identifier. If it receives a target face bounding box from the image library application, it sends a person recognition request to the visual platform. The person recognition request includes the cloud storage address of the first image and the target face bounding box. After receiving the person recognition request, the visual platform sends the person recognition request to the fourth cloud service; after receiving the target human body bounding box sent by the fourth cloud service, it sends the target human body bounding box to the task tool corresponding to the first task. The task tool corresponding to the first task executes the first task based on the target human body frame.

18. The method as described in claim 17, characterized in that, After the task tool corresponding to the first task sends the first query request to the learning and memory platform, it also includes: When the task tool corresponding to the first task receives the query failure information sent by the learning and memory platform, it sends the query failure information to the task engine. After receiving the query failure information, the task engine obtains a fourth response based on the query failure information. The fourth response is used to ask which person's name corresponds to the person in the first image; and sends the fourth response to the intelligent voice assistant. After receiving the fourth response, the intelligent voice assistant displays the fourth response on the second interface; after receiving a user statement entered on the second interface, it sends the user statement to the task engine. After receiving the user's statement, the task engine generates character description information based on the fourth reply and the user's statement, and sends the character description information to the task tool corresponding to the first task. After receiving the person description information, the task tool corresponding to the first task sends a person recognition request to the visual platform. The person recognition request includes the cloud storage address of the first image and the person description information. After receiving the person recognition request, the visual platform sends the person recognition request to the fourth cloud service; after receiving the target human body bounding box sent by the fourth cloud service, it sends the target human body bounding box to the task tool corresponding to the first task. The task tool corresponding to the first task executes the first task based on the target human body frame.

19. An electronic device, characterized in that, The electronic device includes: one or more processors, and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 18.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 18.

21. A computer program product, characterized in that, When the computer program product is run on an electronic device, it causes the electronic device to perform the method as described in any one of claims 1 to 18.

Images