Image retouching method and electronic device

Abstract

The application provides a photo retouching method and an electronic device, and relates to the technical field of terminals. The application can flexibly generate a plurality of tools corresponding to the editing instructions of a user, thereby meeting the personalized needs of the user. The method comprises the following steps: in response to a first operation of a user, an electronic device displays a first interaction window, and the first operation indicates processing a first picture. The electronic device receives a first editing instruction of the user on the first picture in the first interaction window. In response to the first editing instruction, the electronic device acquires a plurality of tools for processing the first picture. The electronic device displays a plurality of controls for adjusting the use parameters of the plurality of tools in the first interaction window.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a photo editing method and an electronic device. Background Technology

[0002] With the development of terminal technology, mobile phones and other electronic devices can support users' image editing needs. For example, electronic devices support one-click image editing functions. Responding to user operations on the one-click editing controls, electronic devices can adjust the colors of images by adding filters, thereby optimizing the image display effect, meeting users' image editing needs, and reducing the difficulty of image editing.

[0003] However, the current photo editing functions offer only one type of editing method, which is insufficient to meet users' personalized needs. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides an image editing method and an electronic device. The technical solution provided by this application allows the electronic device to flexibly generate multiple corresponding tools based on the user's editing instructions, thereby meeting the user's personalized needs.

[0005] To achieve the above-mentioned technical objectives, this application provides the following technical solution:

[0006] In a first aspect, a photo editing method is provided, applied to an electronic device. The method includes: displaying a first interactive window in response to a first user operation, the first operation instructing the processing of a first image; receiving a first editing instruction on the first image input by the user in the first interactive window; acquiring multiple tools in response to the first editing instruction, the multiple tools being used to process the first image; and displaying multiple controls in the first interactive window for adjusting the usage parameters of the multiple tools.

[0007] In this way, the electronic device can flexibly output multiple tools according to the user's input editing instructions, meeting the user's personalized needs. Furthermore, the electronic device can also provide users with an entry point to adjust the usage parameters of multiple tools, satisfying the user's need to adjust the usage parameters of multiple tools.

[0008] According to the first aspect, the multiple controls include general controls and multiple special controls. The general controls are used to adjust the usage parameters of the multiple tools, and each of the multiple special controls is used to adjust the usage parameters of the corresponding tool.

[0009] In this way, by displaying general and special controls, users are provided with a variety of parameter adjustment methods to meet their flexible adjustment needs.

[0010] Furthermore, compared to users manually operating each dedicated control and adjusting the parameters of each tool, displaying general controls simplifies user operations and reduces the difficulty of operation.

[0011] According to the first aspect, or any implementation of the first aspect above, after obtaining multiple tools in response to the first editing instruction, the method further includes: displaying a second image, the second image being an image obtained after processing the first image through the multiple tools.

[0012] In this way, by automatically retouching and displaying the image, users can quickly confirm the retouching effect and reduce user operations.

[0013] According to the first aspect, or any implementation of the first aspect above, the multiple tools obtained in response to the first editing instruction belong to the first toolchain, and the method further includes: receiving a second editing instruction on the second image input by the user in a first interactive window. In response to the second editing instruction, a second toolchain is obtained. Wherein, at least some of the tools included in the second toolchain are different from the multiple tools included in the first toolchain.

[0014] In this way, electronic devices can adaptively provide various tool combinations according to photo editing needs, simplifying user operations while providing users with a more flexible photo editing experience.

[0015] According to the first aspect, or any implementation of the first aspect above, the first interactive window is displayed floating above the first image or the second image, and the method further includes: in response to a second operation by the user, moving the display position of the first interactive window.

[0016] Thus, by displaying the first interactive window in a floating manner, users can avoid conveniently viewing the content displayed below the first interactive window by moving the first interactive window.

[0017] According to the first aspect, or any implementation of the first aspect above, the method further includes: displaying a first control in a first interactive window for invoking a first tool among multiple tools; displaying an image editing interface corresponding to the first tool in response to a third user action on the first control; and invoking the first tool for image processing based on a fourth user action in the image editing interface.

[0018] In this way, electronic devices can be adapted to appropriate tools according to different types of user intentions, and can flexibly call up tools according to different usage conditions to ensure the effect of image editing.

[0019] According to the first aspect, or any implementation of the first aspect above, in response to the first editing instruction, multiple tools are acquired, including: in response to the first editing instruction, sending first input information to the first model, the first input information including a first image and the first editing instruction; and acquiring first output information of the first model, the first output information including relevant information of the multiple tools.

[0020] According to the first aspect, or any implementation of the first aspect above, the first input information further includes one or more of the following: database information, first inference instruction, and historical processing results; wherein, the database information includes relevant information about the tools in the electronic device, the first inference instruction is used to guide the first model to output the first output information, and the historical processing results include the first n rounds of output information of the first model corresponding to the first image, where n is a positive integer.

[0021] In this way, electronic devices can achieve intelligent image editing functions through the first model, and by enhancing the understanding of the first model, they can meet the diverse image editing needs of users.

[0022] Furthermore, by performing text analysis on the input editing commands through the model, it can adaptively match multiple more suitable tools. Compared to existing slot matching methods, by enhancing the model's understanding capabilities, it can more intelligently acquire image editing tools and is no longer limited to outputting only one tool.

[0023] According to the first aspect, or any implementation of the first aspect above, in response to the first editing instruction, multiple tools are acquired, including: in response to the first editing instruction, sending second input information to the second model, the second input information including the first image; acquiring second output information of the second model, the second output information including the evaluation result of the first image; sending third input information to the third model, the third input information including the second output information and the first editing instruction; and acquiring third output information of the third model, the third output information including relevant information of the multiple tools.

[0024] According to the first aspect, or any implementation of the first aspect above, the third input information further includes: database information, second inference instructions, historical processing results, and one or more of the first images, wherein the database information includes relevant information about the tools in the electronic device, the second inference instructions are used to guide the third model to output the third output information, and the historical processing results include the first m rounds of output information of the third model corresponding to the first image, where m is a positive integer.

[0025] In this way, by configuring two models, the model requirements can be reduced and the adaptability of the solution to various types of electronic devices can be improved.

[0026] Secondly, an electronic device is provided. The electronic device includes a processor, a memory, and a display screen. The memory and display screen are coupled to the processor. The memory stores computer program code, including computer instructions. When the processor reads the computer instructions from the memory, the electronic device performs the following actions: In response to a first user operation, a first interactive window is displayed, the first operation instructing the processing of a first image. A first editing instruction for the first image is received from the user in the first interactive window. In response to the first editing instruction, multiple tools are acquired, the multiple tools being used to process the first image. Multiple controls for adjusting the usage parameters of the multiple tools are displayed in the first interactive window.

[0027] According to the second aspect, the multiple controls include general controls and multiple dedicated controls. The general controls are used to adjust the usage parameters of multiple tools, and each dedicated control in the multiple dedicated controls is used to adjust the usage parameters of the corresponding tool.

[0028] According to the second aspect, or any implementation of the second aspect above, when the processor reads computer instructions from memory, it also causes the electronic device to execute: display a second image, which is an image obtained after processing the first image through multiple tools.

[0029] According to the second aspect, or any implementation of the second aspect above, the multiple tools acquired in response to the first editing instruction belong to the first toolchain. When the processor reads computer instructions from memory, the electronic device further executes: receiving a second editing instruction on the second image input by the user in the first interactive window; and acquiring a second toolchain in response to the second editing instruction. The multiple tools included in the second toolchain are at least partially different from the multiple tools included in the first toolchain.

[0030] According to the second aspect, or any implementation of the second aspect above, the first interactive window is displayed floating above the first or second image. When the processor reads computer instructions from the memory, it also causes the electronic device to perform: in response to the user's second operation, move the display position of the first interactive window.

[0031] According to the second aspect, or any implementation of the second aspect above, when the processor reads computer instructions from memory, it also causes the electronic device to perform the following: Displaying a first control in a first interactive window for invoking a first tool among a plurality of tools. Responding to a third user action on the first control, displaying an image editing interface corresponding to the first tool. Based on a fourth user action in the image editing interface, invoking the first tool for image processing.

[0032] According to the second aspect, or any implementation thereof, in response to the first editing instruction, multiple tools are acquired, including: in response to the first editing instruction, sending first input information to the first model, the first input information including a first image and the first editing instruction; and acquiring first output information of the first model, the first output information including relevant information of the multiple tools.

[0033] According to the second aspect, or any implementation of the second aspect above, the first input information further includes one or more of the following: database information, first inference instruction, and historical processing results; wherein, the database information includes relevant information about the tools in the electronic device, the first inference instruction is used to guide the first model to output the first output information, and the historical processing results include the first n rounds of output information of the first model corresponding to the first image, where n is a positive integer.

[0034] According to the second aspect, or any implementation thereof, in response to the first editing instruction, multiple tools are acquired, including: in response to the first editing instruction, sending second input information to the second model, the second input information including the first image; acquiring second output information of the second model, the second output information including the evaluation result of the first image; sending third input information to the third model, the third input information including the second output information and the first editing instruction; and acquiring third output information of the third model, the third output information including relevant information of the multiple tools.

[0035] According to the second aspect, or any implementation of the second aspect above, the third input information further includes: database information, second inference instructions, historical processing results, and one or more items from the first image, wherein the database information includes relevant information about the tools in the electronic device, the second inference instructions are used to guide the third model to output the third output information, and the historical processing results include the first m rounds of output information of the third model corresponding to the first image, where m is a positive integer.

[0036] Thirdly, an electronic device is provided that has the function of implementing the method described in the first aspect and any of its possible implementations. This function can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the described function.

[0037] Fourthly, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program (also referred to as instructions or code) that, when executed by an electronic device, causes the electronic device to perform the method of the first aspect or any embodiment of the first aspect.

[0038] Fifthly, a computer program product is provided that, when run on an electronic device, causes the electronic device to perform the method of the first aspect or any one of the embodiments of the first aspect.

[0039] In a sixth aspect, a circuit system is provided, the circuit system including processing circuitry configured to perform the method of the first aspect or any embodiment of the first aspect.

[0040] In a seventh aspect, a chip system is provided, including at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions and send instructions to the at least one processor, and when the at least one processor executes the instructions, the at least one processor performs the method of the first aspect or any embodiment of the first aspect.

[0041] The technical effects of the aforementioned aspects can be referenced from each other, and will not be elaborated further here. Attached Figure Description

[0042] Figure 1 Illustration of an image retouching scenario provided in this application embodiment Figure 1 ;

[0043] Figure 2 Illustration of an image retouching scenario provided in this application embodiment Figure 2 ;

[0044] Figure 3A A schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application;

[0045] Figure 3B A schematic diagram of the hardware structure of the chip provided in the embodiments of this application;

[0046] Figure 4 This is a schematic diagram of the image retouching method provided in the embodiments of this application;

[0047] Figure 5 Schematic diagram three of the image retouching scenarios provided in this application embodiment;

[0048] Figure 6 Module interaction diagram provided for embodiments of this application Figure 1 ;

[0049] Figure 7 Module interaction diagram provided for embodiments of this application Figure 2 ;

[0050] Figure 8 Illustration of an image retouching scenario provided in this application embodiment Figure 4 ;

[0051] Figure 9 Schematic diagram three illustrating module interaction provided in this application embodiment;

[0052] Figure 10 Module interaction diagram provided for embodiments of this application Figure 4 ;

[0053] Figure 11 Module interaction diagram provided for embodiments of this application Figure 5 ;

[0054] Figure 12 Module interaction diagram provided for embodiments of this application Figure 6 ;

[0055] Figure 13 Illustration of an image retouching scenario provided in this application embodiment Figure 5 ;

[0056] Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0057] The technical solutions of the embodiments of this application are described below with reference to the accompanying drawings. In the description of the embodiments of this application, the terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to be a limitation of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one or more (including two).

[0058] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes direct connections and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0059] In the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0060] In some embodiments, electronic devices support image editing functions to meet users' image editing needs. However, the image editing process generally requires some professional knowledge. For example, users need to know the image editing goal and which parameters to adjust to achieve that goal, making the image editing process relatively difficult. Therefore, electronic devices can reduce the difficulty of the image editing process by configuring a one-click image editing function or artificial intelligence (AI) functions. It should be understood that in different types of electronic devices, functions that can achieve the same effect as a one-click image editing function may have other names, such as automatic optimization functions, etc., and the embodiments of this application do not limit this.

[0061] For example, such as Figure 1 As shown in (a), the electronic device displays images through a gallery application, which is configured with an automatic optimization function. In response to user operation of the automatic optimization function control 11, the electronic device can automatically retouch the currently displayed image and, after retouching, display the image as shown in (a). Figure 1 The interface shown in (b) displays the edited image. During the automatic optimization process, the electronic device can adjust the color of the image by adding filters and other methods, thereby optimizing the image display effect and meeting the user's image editing needs.

[0062] However, current photo editing functions offer only basic adjustments, focusing solely on color and lacking support for other aspects such as composition, super-resolution, and lighting, thus failing to meet users' personalized needs. Furthermore, the editing process is completed with a single click, offering only two states: before and after editing, without allowing users to adjust the level of optimization.

[0063] For example, electronic devices equipped with AI functions can perform image editing through natural language interaction. For instance, as... Figure 2As shown, an electronic device receives a user's input image and editing instruction, "Make this picture look more delicious," through an AI application with AI capabilities. The AI ​​functionality relies on pre-configured instruction slots and corresponding tools. In natural language understanding, a slot refers to the key information that needs to be filled in for a specific intent. For example, instruction slots include filters, saturation, and color temperature. Upon receiving the image editing instruction, the AI ​​application uses slot matching to determine if the editing instruction contains text identical to that in the instruction slot, thus recognizing the user's intent. For example, as... Figure 2 As shown, if the editing command received by the AI ​​application cannot be matched with the command slot, the AI ​​application cannot recognize the user intent corresponding to the editing command and cannot perform image editing. For example, as... Figure 2 As shown, the AI ​​application receives the user's editing instruction again: "Add a food filter to this image." If the pre-configured instruction slot includes "food filter," then the AI ​​application's received editing instruction matches the instruction slot successfully, allowing it to obtain the corresponding food filter tool and add it to the image, completing the retouching. Alternatively, the AI ​​application needs to receive an explicit editing instruction, such as "Adjust the color temperature to XX value," to adjust the image's color temperature.

[0064] It's evident that the AI ​​functionality requires pre-configured command slots and only supports explicit invocation of a single tool, failing to address the need for multiple tools and thus unable to meet users' personalized requirements. Furthermore, users need to memorize each command slot, placing a burden on their memory and increasing the operational complexity. Additionally, the image editing process only presents two states: before and after editing, without allowing users to adjust the optimization level.

[0065] In response, this application provides an image editing method in which an electronic device can flexibly generate multiple corresponding tools according to the user's editing instructions, thereby meeting the user's personalized needs.

[0066] Optionally, the image editing method provided in this application embodiment can be applied to electronic device 100. Optionally, electronic device 100 can be, for example, a mobile phone, tablet computer, personal computer (PC), ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), wearable device, artificial intelligence (AI) device, or other terminal device. The operating system installed on electronic device 100 includes, but is not limited to, […]. Alternatively, other operating systems may be used. This application does not limit the specific type of electronic device 100 or the operating system installed on it.

[0067] For example, Figure 3A A schematic diagram of an electronic device 100 is shown.

[0068] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc.

[0069] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0070] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0071] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0072] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the electronic device 100 to capture images. The processor 110 and the display screen 194 communicate via the DSI interface to enable the electronic device 100 to display images.

[0073] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0074] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be manufactured using a liquid crystal display (LCD), such as an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini-LED, a micro-LED, a micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0075] The sensor module 180 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc.

[0076] A touch sensor, also known as a "touch device," can be located on the display screen 194. The touch sensor and the display screen 194 together form a touchscreen, also known as a "touchscreen." The touch sensor detects touch operations applied to or near it. The touch sensor can then transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In some embodiments, the touch sensor may also be located on the surface of the electronic device 100, in a different position than the display screen 194.

[0077] Figure 3B This application provides a hardware structure for a chip, which includes a neural network processing unit (NPU) 300. This chip can be configured as follows: Figure 3A The processor 110 of the electronic device 100 shown is used to complete... Figure 3A The processor 110 performs some or all of its tasks, such as image editing.

[0078] The neural network processor NPU300 is mounted as a coprocessor on the main central processing unit (CPU) (host CPU), and tasks are assigned by the main CPU. The core of the NPU 300 is the arithmetic circuit 303, and the controller 304 controls the arithmetic circuit 303 to retrieve data from the memory (weight memory or input memory) and perform calculations.

[0079] In some implementations, the arithmetic circuit 303 internally includes multiple process engines (PEs). In some implementations, the arithmetic circuit 303 is a two-dimensional pulsating array. The arithmetic circuit 303 can also be a one-dimensional pulsating array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 303 is a general-purpose matrix processor.

[0080] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit 303 retrieves the corresponding data for matrix B from the weight memory 302 and caches it in each PE (Engineer Component) of the arithmetic circuit 303. The arithmetic circuit 303 retrieves the data for matrix A from the input memory 301 and performs matrix operations with matrix B. The partial or final result of the obtained matrix is ​​stored in the accumulator 308.

[0081] The vector calculation unit 307 can further process the output of the arithmetic circuit 303, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc.

[0082] In some implementations, the vector computation unit 307 can store the processed output vector into the unified memory 306. For example, the vector computation unit 307 can apply a nonlinear function to the output of the arithmetic circuit 303, such as a vector of accumulated values, to generate activation values.

[0083] In some implementations, vector computation unit 307 generates normalized values, merged values, or both.

[0084] In some implementations, the vector of processed output can be used as an activation input to the computation circuit 303, for example, for use in subsequent layers of the neural network.

[0085] Unified memory 306 is used to store input data and output data. Weight data is directly stored in input memory 301 and / or unified memory 306, weight data in external memory 302, and data in unified memory 306 in external memory through direct memory access controller 305 (DMAC).

[0086] The bus interface unit 310 (BIU) is used to enable interaction between the main CPU, DMAC and instruction fetch memory 309 via the bus.

[0087] The instruction fetch buffer 309, connected to the controller 304, stores the instructions used by the controller 304. The controller 304 uses the instructions cached in the instruction fetch buffer 309 to control the operation of the arithmetic accelerator.

[0088] Generally, the unified memory 306, input memory 301, weight memory 302, and instruction fetch memory 309 are all on-chip memories, while the external memory is memory outside the NPU. This external memory can be double data rate synchronous dynamic random access memory (DDR SDRAM), high bandwidth memory (HBM), or other readable and writable memory.

[0089] Based on the hardware structure of the electronic device 100 described above, the image retouching method provided in the embodiments of this application will be described in detail below.

[0090] In some embodiments, the electronic device 100 is equipped with an image editing function, which can be implemented through an image editing application. Optionally, the image editing application can be a local application or a third-party application. Optionally, the local application can be, for example, a gallery application, a Xiaoyi application, etc. The third-party application can be, for example, other AI applications.

[0091] In some embodiments, users can instruct electronic devices 100 to edit images in various ways, triggering the image editing process of electronic devices 100. For example, when electronic devices 100 detect user operation on preset controls while displaying an image, the image editing process can be triggered. For instance, in response to user operation on preset controls, electronic devices 100 can display an interactive window to receive user editing instructions, clarify the user's image editing intent, and trigger subsequent image editing processes. Alternatively, in response to user operation on preset controls, electronic devices 100 can also edit images according to the default user intent, such as without needing to display an interactive window. The default image editing intent could be, for example, beautifying the image. Optionally, preset controls could be, for example, a control in the application currently displaying the image, an identifier corresponding to other AI applications (such as the Xiaoyi application), or a hardware button on electronic devices 100. For another example, when electronic devices 100 detect user-inputted images and image editing instructions while displaying AI applications, the image editing process can be triggered.

[0092] In some embodiments, during image editing, the electronic device 100 can acquire the image and the user's intent, and then, based on the user's intent, obtain the toolchain required to edit the current image. The toolchain is a collection of tools that work together in a certain order and manner to complete a specific task or achieve a goal, such as image editing. The tools, for example, are used to perform image editing, and their related information may include, for example, the tool name (e.g., color temperature, saturation, sharpness), function, and usage parameters. Optionally, different usage parameters of the tools can achieve different image editing effects. Therefore, the toolchain acquired by the electronic device 100 may include, for example, at least one tool, as well as the usage parameters, effects, and reasons for using each tool.

[0093] Optionally, the electronic device 100 may also process images using the default method. That is, the electronic device 100 may not need to obtain the user's intent, but may directly process the image using the default method after obtaining it. Optionally, the default method may be, for example, processing the image according to a default intent.

[0094] In this way, the electronic device 100 can adaptively provide various tool combinations according to the photo editing needs, simplifying user operations while providing users with a more flexible photo editing experience.

[0095] The following text provides a detailed introduction to the photo retouching process.

[0096] Figure 4 This is a schematic flowchart illustrating an image retouching method provided in an embodiment of this application. It should be noted that this method does not rely on... Figure 4The specific order described below is a limitation. It should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0097] S401. In response to a user's first operation, the electronic device 100 displays a first interactive window, the first operation instructing the processing of a first image.

[0098] Optionally, the first interactive window may be, for example, a natural language interactive window, used to receive user editing instructions for the image. Optionally, the editing instructions are used to instruct image editing.

[0099] Optionally, the first operation may be, for example, an operation on a preset control. Optionally, the preset control may be, for example, a control in the application currently displaying the image, an identifier corresponding to another AI application (such as the Xiaoyi application), a hardware button on the electronic device 100, etc. Optionally, the first operation may also be an operation of inputting an image, etc.

[0100] For example, electronic device 100 can receive the user's first operation while displaying the first image, confirming that the user instructs to edit the currently displayed first image. Alternatively, electronic device 100 can receive the user's image editing instruction while displaying an AI application, and then receive the first image input by the user, confirming that the user instructs to edit the currently input first image. The first image input by the user can be a locally selected image or an image acquired by the electronic device through receiving, capturing, or other means.

[0101] For example, such as Figure 5 As shown in (a), during the process of displaying the first image through the gallery application, the electronic device 100 detects the user's operation on control 51. Control 51, for example, is the identifier corresponding to an AI application (such as the Xiaoyi application). Therefore, as... Figure 5 As shown in (b), the electronic device 100 can launch the AI ​​application and display the first interactive window 52 corresponding to the AI ​​application to receive editing instructions input by the user through the first interactive window.

[0102] S402, Electronic device 100 receives a first editing instruction for the first image input by the user in the first interactive window.

[0103] Alternatively, users can input the first editing command to the electronic device 100 through various means such as touch operation on the electronic device 100, voice, input via an external physical or virtual keyboard.

[0104] For example, such as Figure 5As shown in (b), in response to the user's operation on the sending control 53, the electronic device 100 can obtain the user's first editing instruction as "Can you make me look better in the photo?"

[0105] S403, In response to the first editing instruction, the electronic device 100 acquires a plurality of tools for processing the first image.

[0106] In some embodiments, after receiving the first editing instruction, the electronic device 100 can obtain the currently required toolchain through the toolchain analysis module.

[0107] Optionally, the toolchain analysis module can be a preset model, configured in the electronic device 100 or the image editing application, thereby outputting the toolchain required for image editing based on the input image. The preset model is a pre-trained model with image editing capabilities downloaded to or pre-installed by the electronic device 100. For example, the preset model can be one or more of the following: multilingual large language models (LLM), large language models (LLM), and visual question answering (VQA).

[0108] For example, such as Figure 6 As shown, the input module can receive images and editing instructions from the user. Then, the input module can send the received images and editing instructions to the toolchain analysis module. Correspondingly, the toolchain analysis module can receive the images and editing instructions sent by the input module. Then, based on information about relevant tools available in the database, the toolchain analysis module can obtain the toolchain-related information required for this image retouching and send this information to the processing module to trigger the corresponding image retouching process.

[0109] In this way, by analyzing the text of the input editing instructions through the model, multiple more suitable tools can be adaptively matched. Compared with the slot matching method in the existing technology, by increasing the model's understanding, it is possible to more intelligently obtain image editing tools, and is no longer limited to outputting only one tool.

[0110] The specific implementation of the model output toolchain for electronic device 100 is detailed in the following description and will not be repeated here.

[0111] In some embodiments, during the image editing process, the toolchain analysis module can not only refer to the information on relevant tools available in the database to obtain the required toolchain, but also obtain other image editing-related information included in the database to facilitate obtaining a more suitable toolchain. Optionally, other image editing-related information may include, for example, user image editing preference information, tool usage frequency information, and image editing knowledge information. User image editing preference information may include, for example, tools preferred by the user during historical image editing processes, such as soft light filters. Thus, based on user image editing preference information, a toolchain that better meets the user's preferences can be obtained. Tool usage frequency information is used to indicate the usage frequency of different tools in the electronic device 100. Optionally, the database may be one or more databases. For example, the database may be a single database including one or more of the following: information on relevant tools, user image editing preference information, tool usage frequency information, and image editing knowledge information. Alternatively, the database may be a collection of one or more databases, such as a tool database, a user preference database, and an image editing knowledge base, where the tool database includes information on relevant tools, the user preference database includes user image editing preference information and tool usage frequency information, and the image editing knowledge base includes image editing knowledge information. It should be understood that the embodiments of this application do not limit the specific form of the database.

[0112] S404, Electronic device 100 displays multiple controls in the first interactive window for adjusting the usage parameters of multiple tools.

[0113] In some embodiments, after acquiring multiple tools, the electronic device 100 may display controls corresponding to those tools. These controls can be used to adjust the usage parameters of the respective tools.

[0114] For example, such as Figure 5 As shown in (c), when the user inputs the first editing command, "Can you make me look better in the photo?", the electronic device 100 can obtain the first toolchain required to implement the first editing command. For example, the first toolchain includes a contrast tool, a brightness tool, and a smoothing tool. Then, as shown by reference numeral 54, the electronic device 100 can display the controls corresponding to each tool in the first interactive window.

[0115] Optionally, users can adjust the usage parameters of the corresponding tools through various controls. For example... Figure 5 As shown in (c), for example, in response to the user's operation of the control corresponding to the contrast tool, the electronic device 100 can adjust the usage parameters of the contrast tool, thereby increasing or decreasing the contrast of the photo.

[0116] In this way, users can flexibly adjust the parameters according to their own needs. Furthermore, the electronic device 100, through the display of multiple controls, meets the user's needs for adjusting the degree of image optimization under the current image editing requirements.

[0117] It should be understood that Figure 5 The display effect of the control bars (or sliders) corresponding to the various tools shown in (c) is merely illustrative. The electronic device 100 can also display these controls in other ways and prompt the user to adjust the display effect. For example, the electronic device 100 can also directly display the usage parameters of each tool. In response to the user's operation on the control, the electronic device 100 can receive new usage parameters input by the user. This application embodiment does not limit the display method of the controls corresponding to the tools.

[0118] In some embodiments, in response to a first editing instruction, after acquiring multiple tools, the electronic device 100 may also display a second image, which is an image obtained after processing the first image using multiple tools.

[0119] For example, such as Figure 6 As shown, after acquiring the toolchain, the toolchain analysis module can send the corresponding toolchain-related information to the processing module. Optionally, the toolchain-related information may include, for example, the name, usage parameters, usage effects, and reasons for use of each tool in the toolchain. Correspondingly, after receiving the toolchain-related information, the processing module can identify the relevant information and thus invoke the corresponding tool. For example, the processing module can obtain the tool name included in the toolchain-related information, determine the required tool and its corresponding application programming interface (API). Then, the processing module can invoke the corresponding tool through the API interface and send the acquired usage parameters to the tool, thereby triggering the tool to process the first image using these parameters. Subsequently, the processing module can acquire the processed second image. Optionally, the electronic device 100 can acquire and display the second image.

[0120] Optionally, the toolchain analysis module outputs information about different tools in the toolchain in the same format, which facilitates the subsequent parsing of the model information to call the corresponding tools.

[0121] Optionally, the toolchain information may also include, for example, the order in which the tools in the toolchain are used. Some tools in the electronic device 100 can be used in parallel, while others have a specific order of use. Therefore, after acquiring the toolchain, the electronic device 100 can also obtain the order in which the tools in the toolchain are used, and subsequently call the corresponding tools according to this order to achieve an ordered image processing process.

[0122] For example, such as Figure 5In the scenario shown, after acquiring the first toolchain, the electronic device 100 can use the tools within that toolchain to edit the photo. Then, as... Figure 5 As shown in (c), the electronic device 100 can display the edited photo. Furthermore, a first interactive window and controls corresponding to multiple tools displayed in the first interactive window are also displayed on the edited photo, thereby satisfying the user's needs to adjust the optimization level of the edited photo.

[0123] In this way, by automatically retouching and displaying the image, users can quickly confirm the retouching effect and reduce user operations.

[0124] In some embodiments, the first interactive window is displayed floating above the first or second image. Optionally, in response to a second user operation, the electronic device 100 moves the display position of the first interactive window. Optionally, the first interactive window has a certain degree of transparency, thereby reducing the occlusion effect of the first interactive window on the image. Optionally, in response to a user operation, the electronic device 100 may also change the size of the first interactive window.

[0125] Thus, by displaying the first interactive window in a floating manner, users can avoid conveniently viewing the content displayed below the first interactive window by moving the first interactive window.

[0126] In some embodiments, the toolchain acquired by the electronic device 100 includes at least one tool. When the toolchain includes multiple tools, the multiple controls displayed by the electronic device 100 include general controls and multiple dedicated controls. The general controls are used to adjust the usage parameters of the multiple tools, and each dedicated control is used to adjust the usage parameters of the corresponding tool.

[0127] Optionally, the electronic device 100 may have pre-configured control names corresponding to each tool, or the electronic device 100 may determine the control names corresponding to tools through a toolchain analysis module. Optionally, the electronic device 100 may obtain the name of a general control based on the names of the specific controls corresponding to the multiple tools included in the toolchain. Optionally, the electronic device 100 may also obtain the name of a general control based on the names of the specific controls and the editing instructions entered by the user.

[0128] For example, such as Figure 5 In the scenario shown, the toolchain analysis module can determine that the user intent corresponding to the first editing command is to beautify the image. Furthermore, the toolchain analysis module can also obtain the semantics of the names of each dedicated control. Therefore, as... Figure 5 As shown by reference numeral 55 in Figure (c), the toolchain analysis module can obtain the name of the general control as "beautification" by combining the user intent and the semantics of the name of the special control.

[0129] In this way, the electronic device 100 can generate more universal control names that better meet the user's current photo editing needs, making it easier for the user to understand.

[0130] Optionally, the electronic device 100 can integrate multiple tools into one tool and display corresponding general controls. When adjusting the parameters of the integrated tool, the usage parameters of each tool corresponding to each dedicated control are adjusted proportionally. For simplicity, the multiple tools can also be described as sub-tools, and the integrated tool can be described as a total tool.

[0131] For example, such as Figure 5 As shown in (c), in response to the user's operation of the enhancement controls, the electronic device 100 can proportionally adjust the usage parameters of some or all of the contrast, brightness, and smoothing tools, thereby meeting the user's needs for further adjustments to the image display effect. Furthermore, compared to the user manually operating each dedicated control and adjusting the parameters of each tool, the display of universal controls simplifies the user's operation and reduces the difficulty of operation.

[0132] Optionally, the electronic device 100 can expand or retract the display of dedicated controls according to user operation.

[0133] For example, such as Figure 5 As shown in (c), the electronic device 100 can expand or collapse the display of each dedicated control in response to the user's operation on the control 56.

[0134] This satisfies users' need for flexible use of dedicated controls and reduces the amount of content displayed in the interactive window, making it easier for users to find the content they need.

[0135] Optionally, the parameter adjustment range of multiple sub-tools in the toolchain obtained by the electronic device 100 is 0-100, and the parameter adjustment range of the corresponding total tool is also 0-100. Specifically, in response to the user's adjustment of the total tool parameters, if the sub-tool parameter is less than 0 after proportional adjustment, it can be set to 0; if the sub-tool parameter is greater than 100 after proportional adjustment, it can be set to 100.

[0136] Optionally, in response to user operations on dedicated controls, the electronic device 100 adjusts the parameters of the corresponding sub-tools and makes corresponding adjustments to the display effects. For example, in... Figure 5 Under the display effect shown in (c), the electronic device 100 can adjust the display effect of the corresponding control bar. Optionally, in response to changes in the sub-tool parameters, the electronic device 100 will not adjust the overall tool parameters, such as not adjusting the display effect of general controls, thereby avoiding the problem of changes in other sub-tool parameters caused by changes in the overall tool parameters, which would affect the user's adjustment effect.

[0137] Optionally, in response to adjustments to the sub-tool parameters, the proportion of the sub-tool parameters to the total tool parameters will be adjusted accordingly. Then, subsequently, in response to user adjustments to the total tool parameters, the electronic device 100 can obtain the parameters of the sub-tool according to the new proportion.

[0138] In some embodiments, the multiple tools acquired in response to the first editing instruction belong to a first toolchain. Subsequently, the electronic device 100 may also receive a second editing instruction on the second image input by the user in the first interactive window. In response to the second editing instruction, the electronic device 100 acquires a second toolchain. The multiple tools included in the second toolchain are at least partially different from the multiple tools included in the first toolchain.

[0139] Optionally, for a single image, a user may need to go through multiple rounds of retouching to achieve the desired image display effect. In each round of retouching, the user can trigger the electronic device 100 to perform the corresponding retouching process using appropriate editing commands.

[0140] For example, after the first round of image editing, the electronic device 100 can receive a second editing instruction from the user to continue retouching the image, and in response to this second editing instruction, generate another toolchain, such as a second toolchain. Different editing instructions can trigger the electronic device 100 to generate different toolchains, which may include the same or different tools. The same tools may also use the same or different usage parameters to achieve the retouching effect actually required by the user.

[0141] In some embodiments, before a new round of image retouching, the electronic device 100 can acquire at least one previous round of image processing results, thereby obtaining an image retouching effect that meets the user's needs during the current round of retouching based on these processing results. For example, the electronic device 100 can acquire the image processing results from the previous round. In this way, the electronic device 100 can further optimize the image based on the historical processing results, avoiding over-correction of the usage parameters of certain tools. Furthermore, after retouching, problems may arise in certain aspects of the processing results, and the electronic device 100 can also determine the cause of the problem based on the historical processing results. For example, if the image is overexposed, the cause may be an operational error, and the electronic device 100 can revert the image display effect to the state before the overexposure occurred based on the historical processing results.

[0142] For example, such as Figure 7As shown, after processing the image using multiple tools, the processing module can feed back the processing results to the input module and the historical data storage module. Optionally, the processing results fed back by the processing module to the input module and the historical data processing module can be the same or different. For example, the processing results fed back by the processing module to the input module include the image after this round of retouching, while the processing results fed back to the historical data storage module include toolchain-related information such as the tools used in this round of retouching and their usage parameters. In this way, in a new round of retouching, the toolchain analysis module can not only obtain the image after the previous round of retouching from the input module, but also obtain the toolchain-related information used in previous retouching processes from the historical data storage module. Then, the toolchain analysis module can output more appropriate toolchain-related information based on the user's new editing instructions in this round and with reference to the retouching history.

[0143] For example, such as Figure 8 As shown in (a), during the display of image A, electronic device 100 receives an editing command from the user: "Make the sky and lake in this image bluer." In response to this editing command and the input of image A, electronic device 100, through its toolchain analysis module, obtains the color temperature and saturation tools, along with their corresponding usage parameters. Then, electronic device 100 can use the color temperature and saturation tools to process image A using the appropriate parameters, obtaining the processed image, such as image B. Afterwards, as... Figure 8 As shown in (b), the electronic device 100 can display image B. Furthermore, the electronic device 100 can display the tools and corresponding parameters used in processing image A in the first interactive window, facilitating user understanding and adjustment. Subsequently, the electronic device 100 receives a new editing instruction from the user: "Make the water clearer." The electronic device 100 can then obtain historical processing results, including, for example, images B and the tools and parameters used to process image A. In response to the new editing instruction and the input of historical processing results, the electronic device 100, through the toolchain analysis module, can obtain the saturation tool and the sharpness tool, along with their corresponding parameters. The electronic device 100 can then use the saturation and sharpness tools with the corresponding parameters to process image B, obtaining the processed image, such as image C. Afterwards, as... Figure 8 As shown in (c), the electronic device 100 can display image C. Furthermore, the electronic device 100 can display the tools used in processing image B and their corresponding parameters in the first interactive window, facilitating user understanding and adjustment.

[0144] In this way, through natural language interaction, the electronic device 100 can flexibly output the necessary tools according to the user's photo editing needs, satisfying the user's personalized requirements. Furthermore, users do not need to memorize slots, reducing the difficulty of operation.

[0145] In some embodiments, the electronic device 100 may delete historical retouching results in response to a user's instruction to end the retouching process. For example, such as Figure 8 As shown in (c), when the electronic device 100 detects the user's operation on the close control 81, it can end the current image retouching process and delete the historical image retouching results related to image A, such as the processing results generated during multiple retouching processes.

[0146] In this way, during the retouching process of a new image, there will be no issue of the retouching effect of the previous image affecting the retouching result of the new image.

[0147] The above text introduces the overall workflow of the image retouching process. The following text introduces the specific processing procedures of the toolchain analysis module during the image retouching process.

[0148] In some embodiments, in response to a first editing instruction, the electronic device 100 acquires multiple tools, including: sending first input information to a first model in response to the first editing instruction, the first input information including a first image and the first editing instruction; and acquiring first output information of the first model, the first output information including relevant information about the multiple tools.

[0149] Optionally, the first model may be a multimodal model, capable of processing the image and the first editing command. For example, the first model may be an MLLM model. Optionally, the first model may obtain the user's intent based on the first editing command, thereby determining the user's processing needs for the image. For example, ... Figure 5 In the scenario shown, the first model obtains the user's intent, which is to beautify the photo, based on the first editing instruction.

[0150] Optionally, the first input information may further include one or more of the following: database information, a first inference instruction, and historical processing results. The database information includes information about the tools included in the electronic device 100. The first inference instruction is used to guide the first model to output first output information. The historical processing results include the first n rounds of output information of the first model corresponding to the first image, where n is a positive integer.

[0151] For example, such as Figure 9 As shown, the toolchain analysis module includes a first model. This first model can acquire images and editing instructions. The image can be the first image newly input for this retouching session, or it can be the image from the previous retouching round. The editing instructions are the newly received instructions during this retouching process. It should be understood that editing instructions are optional, and the first model can also retouch images based on default user intent.

[0152] Optionally, the first model can also refer to image editing knowledge to process the received image. For example, Figure 9 As shown, image editing knowledge can include database information. Optionally, image editing knowledge can also include other information such as user image editing preferences and tool usage frequency. For example, the first input information can include database information.

[0153] Optionally, such as Figure 9 As shown, the first model can also refer to historical processing results to process the received image, thereby achieving further optimization of the image or reverting the retouching effect. For example, the first input information can also include historical processing results.

[0154] Optionally, the first input information may also include a first inference instruction. The first inference instruction may include, for example, one or more of the following: reward and penalty information for the first model's processing, output result requirements, output format requirements, prompts for possible tools, and image editing examples. Optionally, the electronic device 100 has at least one preset image editing example, and different image editing examples include the tool usage corresponding to the user's intent. This allows the first model to obtain better processing results by referring to the image editing examples. Optionally, the formats of the first inference instructions corresponding to different models are different, and inference instructions input to different models can be understood by the corresponding models. For example, such as... Figure 9 As shown, the first inference instruction input to the first model is, for example, the first model inference instruction.

[0155] Optionally, the first input information is fed into the first model as a prompt. In natural language processing and generation tasks, a prompt refers to the text input used to guide or stimulate the model to generate a specific answer. Simply put, a prompt provides the model with context or a question that helps the model understand the user's needs and generate relevant output.

[0156] For example, such as Figure 9 As shown, after acquiring the image, editing instructions, and initial input information, the first model can output toolchain-related information, which may include information about multiple tools. Thus, after the processing module acquires this toolchain-related information, it can invoke the corresponding tools to process the input image.

[0157] In this way, the electronic device 100 realizes the intelligentization of the image editing function through the first model. By enhancing the understanding of the first model, it can meet the diverse image editing needs of users.

[0158] Optionally, information about each tool can be stored in a database. The model can directly access this database information to output the required toolchain. In this way, when tools are updated, the electronic device 100 only needs to update the database information, allowing for the addition of new tools to the database (or the tool library included in the database), simplifying the tool update process. Compared to directly configuring tool capabilities in the model, the database-based implementation eliminates the need for extensive model training, reducing tool update costs and timelines.

[0159] In other embodiments, in response to a first editing instruction, the electronic device 100 acquires multiple tools, including: sending second input information to a second model, the second input information including a first image; acquiring second output information from the second model, the second output information including an evaluation result of the first image; sending third input information to a third model, the third input information including the second output information and the first editing instruction; and acquiring third output information from the third model, the third output information including relevant information about the multiple tools.

[0160] Optionally, the second model can be, for example, an image evaluation model capable of outputting an aesthetic evaluation of the image. For instance, the second model could be a Visual Quality Assurance (VQA) model. Optionally, the image evaluation results output by the second model, based on the input image, may include descriptions and evaluations of various preset dimensions such as color, composition, and lighting. Different tools are used to adjust different dimensions of the image. Subsequently, the third model, based on the evaluation results output by the second model, can more quickly determine the tools needed for processing the image.

[0161] Optionally, the third model may be, for example, a toolchain orchestration model, which can be used to combine the user-input editing instructions and the second output information of the second model to output relevant information of multiple tools required for this image retouching.

[0162] Optionally, the third input information may also include: database information, second inference instructions, historical processing results, and one or more of the first images, wherein the database information includes relevant information about the tools included in the electronic device 100, the second inference instructions are used to guide the third model to output third output information, and the historical processing results include the first m rounds of output information of the third model corresponding to the first image, where m is a positive integer.

[0163] Optionally, the third model is an LLM model. For example, such as Figure 10As shown, the toolchain analysis module includes a second model and a third model. The second model acquires images and outputs evaluation results based on them. The third model acquires the evaluation results output by the second model and, based on the evaluation results, editing instructions, and third input information, outputs toolchain-related information to trigger the processing module to call tools for image processing based on the toolchain-related information. Optionally, the third input information may include one or more of the following: database information, third model inference instructions (such as second inference instructions), and historical processing results. Optionally, other contents of the various information included in the third input information can refer to the relevant contents of the first input information mentioned above, and will not be repeated here.

[0164] Optionally, the third model can also be a multimodal model. For example, such as... Figure 11 As shown, the third model can obtain not only the evaluation results output by the second model, but also the input images of the second model; that is, the third input information can also include images. In this way, the third model can refer to the images to output toolchain-related information, achieving the acquisition of more suitable tools and a better toolchain arrangement.

[0165] In this way, by configuring two models, the model requirements can be reduced and the adaptability of the solution to various types of electronic devices 100 can be improved.

[0166] In some embodiments, after receiving a user's editing instruction, the electronic device 100 can determine the user's intent based on the instruction. Optionally, the user intent can be categorized into image editing intent and tool invocation intent. The image editing intent instructs the electronic device 100 to edit an image, and the electronic device 100 can directly trigger image processing. For example, Figure 5 In the scenario shown, the user's intent to beautify the photo is an image editing intent. The tool invocation intent instructs the electronic device 100 to invoke tools. However, these tools may require multiple interactions between the electronic device 100 and the user to determine the final image processing method. Therefore, the electronic device 100 cannot directly invoke these tools for image editing; instead, it needs to display the corresponding image editing interface to receive user interactions. For example, if the user's intent is to crop the image, different users may have different cropping needs. Therefore, the electronic device 100 cannot directly invoke the cropping tool; it needs to obtain the actual cropping requirements based on subsequent user interactions.

[0167] In some examples, the toolchain analysis module can determine the classification of user intent based on the input editing instructions, thereby determining whether the current image editing task is an image editing task, a tool invocation task, or a combination of both. In other examples, such as Figure 12As shown, the input module sends editing instructions to the intent classification module. This module, based on the editing instructions, obtains the classification result of the user's intent and sends it to the toolchain analysis module. Subsequently, the toolchain analysis module can refer to the classification result and output appropriate toolchain-related information during the toolchain generation process.

[0168] In some examples, electronic device 100 displays a first control in a first interactive window for invoking a first tool among multiple tools. In response to a third user action on the first control, electronic device 100 displays an image editing interface corresponding to the first tool. Based on a fourth user action in the image editing interface, electronic device 100 invokes the first tool for image processing.

[0169] For example, such as Figure 13 As shown in (a), the electronic device 100 receives an editing instruction from the user to crop a photo. In response to this editing instruction, the electronic device 100 determines that the current image editing task is a tool invocation task. Furthermore, the electronic device 100 can determine that the tool instructed by the user to be invoked is a cropping tool (such as the first tool). Then, as... Figure 13 As shown in (b), the electronic device 100 can display a cropping control 131 (such as the first control) in a first interactive window. Subsequently, in response to a third user action on the cropping control 131, the electronic device 100 can display... Figure 13 The cropping interface is shown in (c). In the cropping interface, the electronic device 100 can invoke the cropping tool to crop the photo according to the user's fourth operation on the cropping frame 132.

[0170] In this way, electronic devices can also respond to image input, displaying the tool access points to the user, thereby receiving personalized image editing operations from the user.

[0171] Optionally, image editing tasks can include image editing tasks and tool application tasks. For example, such as Figure 13 In the scenario shown in (a), the user's input editing instructions could also be "Can I make myself look better in the photo and can I crop the photo?" Thus, the user intent obtained by the electronic device 100 includes both photo enhancement and photo cropping. Responding to this user intent, the electronic device 100 invokes enhancement-related tools to edit the image and displays the enhanced photo. Furthermore, the electronic device 100 can display controls corresponding to each enhancement-related tool and cropping controls in an interactive window on the enhanced photo. In this way, the user can not only intuitively see the photo enhancement effect but also use the cropping controls to invoke cropping tools to crop the enhanced photo to meet their cropping needs.

[0172] It should be understood that the above tool invocation scheme uses a cropping tool as an example. In some scenarios, the electronic device 100 can also crop images according to the default cropping method. In this case, the image editing task corresponding to the cropping tool can also be an image editing task. The default cropping method includes, for example, cropping to the default size or cropping the main displayed content. This application embodiment does not specifically limit the classification method of image editing tasks.

[0173] In this way, the electronic device 100 can adapt to the appropriate tools according to different types of user intentions, and flexibly call up the tools according to the different usage conditions to ensure the image editing effect.

[0174] In some solutions, multiple embodiments of this application can be combined, and the combined solution can be implemented. Optionally, some operations in the processes of each method embodiment may be combined, and / or the order of some operations may be changed. Furthermore, the execution order between the steps of each process is merely exemplary and does not constitute a limitation on the execution order between steps; other execution orders are also possible. It is not intended to indicate that the execution order is the only possible order in which these operations can be performed. Those skilled in the art will conceive of various ways to reorder the operations described herein. In addition, it should be noted that the process details involved in one embodiment of this document are similarly applicable to other embodiments, or different embodiments may be combined.

[0175] Furthermore, some steps in the method embodiments can be equivalently replaced with other possible steps. Alternatively, some steps in the method embodiments may be optional and can be deleted in certain use cases. Or, other possible steps may be added to the method embodiments.

[0176] Furthermore, the various method embodiments can be implemented individually or in combination.

[0177] The above combination Figures 4-13 The image retouching method provided in the embodiments of this application is described in detail below. Figure 14 This application provides a detailed description of the electronic device provided in its embodiments.

[0178] In one possible design, Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 14 As shown, the electronic device 1400 may include a transceiver unit 1401, a processing unit 1402, and a display unit 1403. The electronic device 1400 can be used to implement the functions of the electronic device 100 involved in the above method embodiments.

[0179] Optionally, the transceiver unit 1401 is used to support the electronic device 1400 in performing... Figure 4 S401 and S402 in the example.

[0180] Optionally, the processing unit 1402 is used to support the electronic device 1400 in performing operations. Figure 4 S403 in the middle.

[0181] Optionally, the display unit 1403 is used to support the electronic device 1400 in performing operations. Figure 4 S401 and S404 in the example.

[0182] Optionally, the transceiver unit 1401 may include a receiving unit and a transmitting unit, which may be implemented by a transceiver or transceiver-related circuit components, and may be a transceiver or transceiver module. The operation and / or function of each unit in the electronic device 1400 are respectively to implement the corresponding process of the image retouching method described in the above method embodiments. All relevant content of each step involved in the above method embodiments can be referred to the functional description of the corresponding functional unit, and for the sake of brevity, it will not be repeated here.

[0183] Optionally, Figure 14 The illustrated electronic device 1400 may also include a storage unit ( Figure 14 (not shown in the image), this storage unit stores a program or instruction. When the transceiver unit 1401, processing unit 1402, and display unit 1403 execute the program or instruction, it causes... Figure 14 The electronic device 1400 shown can perform the image retouching method described in the above method embodiments.

[0184] Figure 14 The technical effects of the electronic device 1400 shown can be referred to the technical effects of the image retouching method described in the above method embodiments, and will not be repeated here.

[0185] In addition to being in the form of electronic device 1400, the technical solution provided in this application can also be a functional unit or chip in an electronic device, or a device used in conjunction with an electronic device.

[0186] This application also provides a chip system, including: a processor coupled to a memory, the memory being used to store programs or instructions, wherein when the program or instructions are executed by the processor, the chip system implements the methods in any of the above method embodiments.

[0187] Optionally, the chip system may contain one or more processors. These processors can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0188] Optionally, the chip system may contain one or more memories. The memory may be integrated with the processor or disposed separately from it; this application embodiment does not limit this. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or disposed separately on different chips. This application embodiment does not specifically limit the type of memory or the arrangement of the memory and processor.

[0189] For example, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0190] It should be understood that each step in the above method embodiments can be completed by integrated logic circuits in the processor hardware or by instructions in software form. The method steps disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0191] This application also provides a computer-readable storage medium storing a computer program. When the computer program is run on a computer, it causes the computer to perform the aforementioned steps to implement the image retouching method described in the above embodiments.

[0192] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the image editing method described in the above embodiments.

[0193] In addition, this application also provides an apparatus. Specifically, the apparatus may be a component or module, and may include one or more processors and a memory connected together. The memory is used to store a computer program. When the computer program is executed by one or more processors, the apparatus performs the image retouching methods described in the above-described method embodiments.

[0194] The apparatus, computer-readable storage medium, computer program product, or chip provided in the embodiments of this application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0195] The steps of the methods or algorithms described in conjunction with the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an application-specific integrated circuit (ASIC).

[0196] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, the division of the above functional modules is only used as an example. In practical applications, the above functions can be assigned to different functional modules as needed; that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0197] In the several embodiments provided in this application, it should be understood that the disclosed methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical functional division, and there may be other division methods in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of modules or units may be electrical, mechanical or other forms.

[0198] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0199] Computer-readable storage media include, but are not limited to, any of the following: USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media capable of storing program code.

[0200] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of retouching an image, characterized in that, The method is applied to an electronic device, and the method includes: In response to the user's first action, a first interactive window is displayed, and the first action instructs the processing of a first image; The first interactive window receives the user's first editing command for the first image; In response to the first editing instruction, multiple tools are acquired, the multiple tools being used to process the first image; The first interactive window displays multiple controls for adjusting the usage parameters of the multiple tools.

2. The method of claim 1, wherein, The plurality of controls includes general controls and a plurality of special controls. The general controls are used to adjust the usage parameters of the plurality of tools, and each of the plurality of special controls is used to adjust the usage parameters of the corresponding tool.

3. The method according to claim 1 or 2, characterized in that, After acquiring multiple tools in response to the first editing instruction, the method further includes: The second image is displayed; the second image is the image obtained after processing the first image using the multiple tools.

4. The method of claim 3, wherein, In response to the first editing command, the multiple tools obtained belong to the first toolchain, and the method further includes: The first interactive window receives a second editing instruction for the second image input by the user; In response to the second editing command, obtain the second toolchain; In this process, at least some of the tools included in the second toolchain are different from the tools included in the first toolchain.

5. The method according to claim 3 or 4, characterized in that, The first interactive window is displayed floating above the first image or the second image, and the method further includes: In response to the user's second action, the display position of the first interactive window is moved.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: A first control for invoking a first tool among the plurality of tools is displayed in the first interactive window; In response to the user's third operation on the first control, the image editing interface corresponding to the first tool is displayed; Based on the user's fourth operation in the image editing interface, the first tool is invoked to process the image.

7. The method according to any one of claims 1-6, characterized in that, In response to the first editing command, multiple tools are acquired, including: In response to the first editing instruction, first input information is sent to the first model, the first input information including the first image and the first editing instruction; Obtain the first output information of the first model, wherein the first output information includes relevant information of the plurality of tools.

8. The method according to claim 7, characterized in that, The first input information further includes one or more of the following: database information, first inference instruction, and historical processing results; wherein, the database information includes relevant information about the tools in the electronic device, the first inference instruction is used to guide the first model to output the first output information, and the historical processing results include the first n rounds of output information of the first model corresponding to the first image, where n is a positive integer.

9. The method according to any one of claims 1-6, characterized in that, In response to the first editing command, multiple tools are acquired, including: In response to the first editing instruction, second input information is sent to the second model, the second input information including the first image; Obtain the second output information of the second model, wherein the second output information includes the evaluation result of the first image; Send third input information to the third model, the third input information including the second output information and the first editing instruction; Obtain the third output information of the third model, which includes relevant information of the plurality of tools.

10. The method according to claim 9, characterized in that, The third input information further includes: database information, a second inference instruction, historical processing results, and one or more items from the first image. The database information includes information related to the tools in the electronic device. The second inference instruction is used to guide the third model to output the third output information. The historical processing results include the first m rounds of output information of the third model corresponding to the first image, where m is a positive integer.

11. An electronic device, characterized in that, include: The electronic device includes a processor, a memory, and a display screen, wherein the memory and the display screen are coupled to the processor, the memory being used to store computer program code, the computer program code including computer instructions, and the electronic device performing the method as described in any one of claims 1-10 when the processor reads the computer instructions from the memory.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a computer program that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 1-10.

13. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1-10.

Images