Image processing method and device, equipment and storage medium

Abstract

The embodiment of the invention provides an image processing method and device, equipment and a storage medium. The method includes presenting a first image during interaction between a user and a digital assistant; based on an interaction operation from a user, determining a first area in the first image and adjustment information for the first area, the adjustment information indicating adjustment of a visual effect of the first area; and in response to the received adjustment instruction for the first area, presenting a second image as an adjustment result of the first image, a second area corresponding to the first area in the second image having an adjusted visual effect. In this way, the local area of the image can be precisely processed, and the image processing efficiency can be improved.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to methods, devices, apparatuses, and computer-readable storage media for image processing. Background Art

[0002] In the field of computer vision (CV), various image processing technologies have been significantly developed and have a wide range of applications. For example, in many application scenarios such as social networking, games, and image editing, it is expected to generate and use target images. For example, image processing technology based on machine learning can be used in such application scenarios to improve the user experience. Summary of the invention

[0003] In a first aspect of the present disclosure, a method for image processing is provided. The method includes: in the interaction between a user and a digital assistant, presenting a first image; based on an interactive operation from the user, determining a first region in the first image and adjustment information for the first region, the adjustment information indicating an adjustment to a visual effect of the first region; and in response to receiving an adjustment indication for the first region, presenting a second image as an adjustment result of the first image, wherein a second region corresponding to the first region in the second image has an adjusted visual effect.

[0004] In a second aspect of the present disclosure, a device for image processing is provided. The device includes: a presentation module configured to present a first image in the interaction between a user and a digital assistant; a receiving module configured to determine a first region in the first image and adjustment information for the first region based on an interactive operation from the user, the adjustment information indicating an adjustment to a visual effect of the first region; and a processing module configured to present a second image as an adjustment result of the first image in response to receiving an adjustment indication for the first region, wherein a second region in the second image corresponding to the first region has an adjusted visual effect.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.

[0007] It should be understood that the contents described in this content section are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0009] Figure 1 A schematic diagram showing an example environment in which embodiments according to the present disclosure may be implemented;

[0010] Figures 2A to 2G A schematic diagram showing an example interface for partial regeneration according to some embodiments of the present disclosure;

[0011] FIG. 3A to FIG. 3E A schematic diagram showing an example interface for local update according to some embodiments of the present disclosure;

[0012] FIG. 4A to FIG. 4C Schematic diagrams showing some other example interfaces for local update according to some embodiments of the present disclosure;

[0013] Figure 5 A flowchart showing an example process of image processing according to some embodiments of the present disclosure;

[0014] Figure 6 A schematic structural block diagram showing an example apparatus for image processing according to some embodiments of the present disclosure; and

[0015] Figure 7 A block diagram of an electronic device capable of implementing various embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0016] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0017] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0018] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0019] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0020] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0021] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0022] It should be noted that the titles of any sections / subsections provided herein are not restrictive. Various embodiments are described throughout this article, and any type of embodiment may be included under any section / subsection. In addition, the embodiments described in any section / subsection may be combined in any manner with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0023] Herein, unless explicitly stated, executing a step “in response to A” does not mean executing the step immediately after “A” but may include one or more intermediate steps.

[0024] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below. The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may be included below.

[0025] As used herein, the term "model" can learn the association between the corresponding input and output from the training data, so that after the training is completed, the corresponding output can be generated for a given input. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multiple layers of processing units. In this article, "model" may also be referred to as "machine learning model", "machine learning network" or "network", which are used interchangeably in this article. A model may also include different types of processing units or networks.

[0026] As briefly mentioned above, image processing technology has been applied to a variety of application scenarios. Traditional image processing technology has certain defects in local erasure and local re-editing, resulting in problems such as difficulty in controlling the generated results and low image processing efficiency during the image processing process.

[0027] The embodiments of the present disclosure propose an image processing scheme. According to the embodiments of the present disclosure, in the interaction between a user and a digital assistant, a first image is presented. Based on the interactive operation from the user, a first area in the first image and adjustment information for the first area are determined, and the adjustment information indicates the adjustment of the visual effect of the first area. If an adjustment indication for the first area is received, a second image is acquired and presented as the adjustment result of the first image. In the second image, the second area corresponding to the first area has the adjusted visual effect. According to various embodiments of the present disclosure, in the interaction between a user and a digital assistant, a local area of ​​the image can be adjusted based on the interactive operation of the user. In this way, the local area of ​​the image can be accurately processed and the image processing efficiency can be improved.

[0028] Various example implementations of the solution are described in detail below in conjunction with the accompanying drawings.

[0029] Example Environment

[0030] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. In the example environment 100, an application 120 is installed in a terminal device 110. A user 140 can interact with the application 120 via the terminal device 110 and / or an attached device of the terminal device 110.

[0031] In some embodiments, application 120 may be any suitable application, such as an application that can provide a query service. Figure 1 In the environment 100, if the application 120 is active, the terminal device 110 can present a page 150 of the application 120. The page 150 can include various pages that the application 120 can provide, such as an information interaction page, a query page, a search page, a search result presentation page, and the like.

[0032] In some embodiments, the terminal device 110 communicates with the server 130 to provide the service of the application 120. The terminal device 110 can be any type of mobile terminal, fixed terminal or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication systems (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio broadcast receivers, e-book devices, gaming devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the terminal device 110 can also support any type of interface for the user (such as "wearable" circuits, etc.). The server 130 can be various types of computing systems / servers that can provide computing power, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, and the like.

[0033] It should be understood that the structure and function of the various elements in the environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure.

[0034] Some example embodiments of the present disclosure will continue to be described with reference to the accompanying drawings. It should be understood that the pages shown in the accompanying drawings are merely examples, and various page designs may actually exist. The various graphic elements in the page may have different arrangements and different visual representations, one or more of which may be omitted or replaced, and one or more other elements may also exist. The embodiments of the present disclosure are not limited in this respect. In addition, hereinafter, the example embodiments will be described primarily with respect to the terminal device 110. It should be understood that the actions described with respect to the terminal device 110 may be performed by the application 120 on the terminal device 110, or may be performed by the application 120 in collaboration with its service end (e.g., server 130).

[0035] Example Interaction

[0036] In the interaction between the user 140 and the digital assistant, the user 140 can be helped to adjust the local area of ​​the image, so as to obtain the image desired by the user. In the interaction between the user 140 and the digital assistant, the terminal device 110 can present a first image. The first image can be regarded as an initial image. In some embodiments, the first image can be an image specified by the user 140 in any suitable manner, such as an image uploaded by the user 140. In some embodiments, the first image can be an image generated using a machine learning model, such as generated according to user input during the interaction between the user 140 and the digital assistant. The terminal device 110 can determine the first area in the first image and the adjustment information for the first area based on the interaction operation from the user 140. The adjustment information can indicate the adjustment of the visual effect of the first area. That is, based on the interaction operation of the user 140, the local area to be adjusted in the first image and the information about how to adjust the area can be determined. In an embodiment of the present disclosure, the visual effect may include visual elements or visual content (e.g., mountains, trees, characters, etc.) in the image, and may also include attributes of the visual elements or visual content (e.g., color, brightness, chroma, clarity, etc.).

[0037] In some embodiments, an interface element may be provided for the user 140 so that the user 140 can select an area and provide adjustment information on his own. For example, if a local adjustment request for the first image is received, the terminal device 110 may present an area selection control and a prompt input entry, as will be described below. Figures 2A to 2G As described. The terminal device 110 may receive a selection of a first area in the first image via an area selection control. The terminal device 110 may receive a user input including adjustment information via a prompt input entrance. Then, if an adjustment instruction for the first area is received, the terminal device 110 may present a second image as an adjustment result of the first image, and a second area corresponding to the first area in the second image has an adjusted visual effect. For example, the terminal device 110 may send the adjustment information, the first image, and the information of the first area to the server, and the server may update the first image to the second image using a machine learning model.

[0038] In an embodiment of the present disclosure, the adjustment of the local area may include various suitable forms of adjustment. In some embodiments, the adjustment of the visual effect of the first area may include local redrawing, and the second area may include visual content generated based on the adjustment information. In other words, the local adjustment may indicate the regeneration of the local image. Figures 2A to 2G Describe an example. Figures 2A to 2G Schematic diagrams of example interfaces 200A to 200G for partial regeneration according to some embodiments of the present disclosure are shown. The interfaces 200A to 200G may be composed of, for example, Figure 1The terminal device 110 shown is provided.

[0039] like Figure 2A In the interface 200A shown, the terminal device 110 can present the first image 210 in the interface. Exemplarily, the interface 200A can also include a "Modify" control 215, a "Download" control 220, a "Share" control 225, and an "Add as Work" control 230. In some examples, if the terminal device 110 detects that the user 140 clicks the "Download" control 220, the first image 210 can be saved locally or to a target address. In some examples, if the terminal device 110 detects that the user 140 clicks the "Share" control 225, the first image 210 can be shared to a target session or a target address. In some examples, if the terminal device 110 detects that the user 140 clicks the "Add as Work" control 230, the first image 210 can be saved to the collection. It can be understood that Figure 2A The controls shown are exemplary only and are not intended to be limiting.

[0040] If the terminal device 110 detects that the user 140 clicks the "Modify" control 215, the following may be presented: Figure 2B After receiving the user 140 clicking the "Modify" control 215, the terminal device 110 presents a panel 240 for selecting a processing operation. Figure 2B In the example of , panel 240 may include options for multiple processing operations, such processing operations may include, for example, high-definition quality, image expansion, partial redrawing, and partial erasing. In some examples, if terminal device 110 detects that user 140 clicks on any position in the area corresponding to the partial redraw in panel 240, interface 200C may be presented.

[0041] like Figure 2CIn the interface 200C shown, the terminal device 110 presents a control 250, an interface element 255, and a prompt input entry 260 when detecting that the user 140 clicks on any position in the area corresponding to the partial redrawing in the panel 240. Exemplarily, the user 140 can adjust the size of the area selection control by sliding the circular area in the control 250. The interface element 255 can be used to display the numerical value of the area selection control and can slide together with the circular area in the control 250. The terminal device 110 can receive a selection of the first area in the first image via the area selection control. In some examples, the area selection control can be presented by any reasonable graphics such as a circle, an arrow, etc. The terminal device 110 can receive user input including adjustment information via the prompt input entry 260. In some examples, the prompt input entry 260 can include corresponding prompt words (e.g., "describe the redrawn content or directly generate...") and a "send" control 265. If the terminal device 110 receives the user 140's selection of the first area in the first image via the area selection control, it can present an interface 200D.

[0042] The above-described method of determining the first area (i.e., determining the area to be adjusted) is only exemplary and is not intended to be limiting. In the embodiments of the present disclosure, the first area can be determined in various suitable ways. In some embodiments, the terminal device 110 can receive the selection of the initial area in the first image and the description information for adjusting the first image. For example, the initial area can be the area corresponding to the selection of the first image received by the terminal device 110 for the first time. For example, the description information can be "keep the buildings and sky in the first image". For another example, the description information can be "keep the characters in the first image". Then, the terminal device 110 can update the initial area to the first area based on the description information. For example, the terminal device 110 can expand, reduce, move, etc. the initial area based on the description information to determine the first area. In some examples, assuming that the description information indicates "keep the buildings and grass", the terminal device 110 can expand the initial area based on the description information to determine the area other than the buildings and grass in the first image as the first area. In this embodiment, the first area is determined by combining the selection operation with the description information.

[0043] In some embodiments, the first area may be determined based on at least one of the following: a natural language input indicating the area to be adjusted, a predetermined type of operation on the area in the first image, or an interactive control for selecting the area to be adjusted. The natural language input may include text input or voice input from the user, and may describe the characteristics of the area to be adjusted in the first image. The predetermined type of operation on the area in the first image may allow the user to specify the area to be adjusted. In some examples, the predetermined type of operation on the area in the first image may include a smearing operation, a circling operation, and the like. The interactive control for selecting the area to be adjusted may provide the user with what kind of appropriate area selection controls, such as a box selection control, and the like.

[0044] like Figure 2D In the illustrated interface 200D, the first region 270 of the first image 210 selected by the user 140 is highlighted in a specific visual style to be visually distinguished from unselected regions.

[0045] like Figure 2E In the interface 200E shown, if the terminal device 110 receives the user input (which may include adjustment information) from the user 140 in the prompt input entry 260, the prompt input entry 275 containing the user input is presented. In some examples, the input content of the user input may be "Replace the selected area with the sky". In some examples, the input content of the user input may be "Partial redraw: Replace the selected area with green grassland". If the terminal device 110 detects that the user 140 clicks the "Send" control 265, the user input may be provided to the digital assistant and the interface 200F is presented.

[0046] like Figure 2F In the interface 200F shown, if the terminal device 110 provides the user input to the digital assistant, a user input display box 280 and a picture processing display interface 285 may be presented. The user input display box 280 may include a reference picture of the first image and the input content entered by the user. The picture processing display interface 285 may include an "image generating" icon 286 and a generation degree prompt word 287 (for example, "image generating 78%"). Interface 200F also includes a "regenerate" area 288. If the terminal device 110 receives a click from the user 140 on any position in the "regenerate" area 288, the image generation operation may be performed again.

[0047] In some embodiments, the first image is determined based on an interactive message sent by the user to the digital assistant. For example, the interactive message may be "draw an image, in which there is a little girl walking on the street on a rainy day, and the little girl is holding a white puppy in her right hand". In this embodiment, the terminal device 110 can determine the adjustment target for the first area based on the adjustment information and the interactive message previously used to generate the first image. Then, the terminal device 110 can update the first image to the second image according to the adjustment target. In some examples, the adjustment information may include "replace the puppy in the image with a kitten". Accordingly, in the second image, the puppy is replaced with a kitten.

[0048] like Figure 2G In the interface 200G shown, after detecting that the image generation is completed, the terminal device 110 presents the second image 290 as the adjustment result of the first image 210. The second area 295 corresponding to the first area 270 in the second image 290 has the adjusted visual effect. In some examples, if the input content entered by the user is "replace the selected area with a big tree", the visual content presented by the second area 295 includes a big tree. In some examples, if the input content entered by the user is "replace the selected area with a starry sky", the visual content presented by the second area 295 includes the starry sky.

[0049] In some embodiments, the adjustment of the visual effect of the first area may include a local effect update. The local effect update may also be referred to as a local modification or a local erase. The local effect update may include an update of part of the content in the local area, an update of the attributes of the presented content, and the like.

[0050] In some embodiments, the second region may include visual content that is partially the same as that of the first region. That is, the second region obtained by updating the local effect of the first region may include visual content that is partially the same as that of the first region (the part that has not been updated with local effects) and visual content that is partially different from that of the first region (the part that has been updated with local effects). FIG. 3A to FIG. 3E Describe an example. FIG. 3A to FIG. 3E Schematic diagrams of example interfaces 300A to 300E for local update according to some embodiments of the present disclosure are shown. Interfaces 300A to 300E may be composed of, for example, Figure 1 The terminal device 110 shown is provided.

[0051] like Figure 2A and Figure 3A In the example interfaces 200A and 300A shown, after the user 140 clicks the "Modify" control 215 in the interface 200A, the interface 300A may be presented. Figure 3AAs shown, the operation editing area 310 is presented in the interface 300A. Exemplarily, the operation editing area 310 may include the first image 210, the processing operation selection area, and the prompt input entrance. In some examples, the processing operation selection area may include a "high definition" control, a "enlargement" control, a "partial redraw" control, and a "partial erase" control. In some examples, the prompt input entrance may be presented with a prompt word, such as "Describe how you want to edit the picture". If the terminal device 110 detects that the user 140 clicks on any position of the "partial erase" control in the processing operation selection area, the interface 300B is presented.

[0052] like Figure 3B In the interface 300B shown, the terminal device 110 can receive the first area 320 of the first image 210 selected by the user 140 through the area selection control. In the interface 300B, the first area 320 selected by the user 140 is highlighted in a specific visual style to visually distinguish it from the unselected area. The interface 300B may also include an operation prompt box 325. Exemplarily, the operation prompt box 325 may include a thumbnail of the first image and an editing area 326 (e.g., "partial erase"). In addition, the terminal device 110 may receive user input through the editing area 326 in the operation prompt box 325. In some examples, the terminal device 110 may display the name of the selected processing operation, such as "partial erase" in the editing area 326 without receiving user input.

[0053] like Figure 3C In the interface 300C shown, the terminal device 110 may present an operation prompt box 330 when receiving user input. Exemplarily, the edit area 335 of the operation prompt box 330 may present the input content entered by the user, such as "erase XXX but do not erase YYY". If the terminal device 110 detects that the user 140 clicks the "Send" control 335 in the edit area 326, the interface 300D is presented.

[0054] like Figure 3D In the interface 300D shown, in the interactive interface with the digital assistant, after the terminal device 110 provides the operation prompt box containing the user input to the digital assistant, it can present the picture processing display interface 340 in the interactive interface. Exemplarily, the picture processing display interface 340 may include an "image generating" icon 386 and a generation degree prompt word 387 (e.g., "image generating 75%").

[0055] like Figure 3EIn the interface 300E shown, after detecting that the image generation is completed, the terminal device 110 presents the second image 350 as the adjustment result of the first image 210. Exemplarily, the second area 355 corresponding to the first area 320 in the second image 350 has the adjusted visual effect. Continuing with the above example, if the user input is "erase XXX but do not erase YYY", then the second area 355 may include the visual content of YYY but no longer include the visual content of XXX.

[0056] In some embodiments, the second region may have at least one dimension of image attributes that are different from the first region. In some examples, such image attributes may be any suitable attributes, such as but not limited to one of color, transparency, brightness, and clarity. FIG. 4A to FIG. 4C Describe an example. FIG. 4A to FIG. 4C Schematic diagrams of other example interfaces 400A to 400C for local update according to some embodiments of the present disclosure are shown. Interfaces 400A to 400C may be composed of, for example, Figure 1 The terminal device 110 shown is provided.

[0057] like Figure 3B and Figure 4A In the example interfaces 300B and 400A shown, after the terminal device 110 detects the user input of the user 140 in the editing area 326 of the interface 300B, the interface 400A may be presented.

[0058] like Figure 4A In the interface 400A shown, the terminal device 110 may present an operation prompt box 430. Exemplarily, the operation prompt box 430 may include a thumbnail of the first image and an editing area containing a user input of "slightly erase but do not change the original appearance". More preferably, if the terminal device 110 detects that the user 140 clicks the "Send" control 335 in the editing area 435, the interface 400B is presented.

[0059] like Figure 4B As shown, in the interactive interface 400B with the digital assistant, after the terminal device 110 provides the operation prompt box containing the user input to the digital assistant, the image processing display interface 445 can be presented in the interactive interface.

[0060] like Figure 4CIn the interface 400C shown, after detecting that the image generation is completed, the terminal device 110 can present the second image 440 as the adjustment result of the first image 210. Exemplarily, the second area 455 corresponding to the first area 320 in the second image 440 has the adjusted visual effect. Continuing with the above example, when the user input is "slightly erase but do not change the original appearance", the visual content itself in the second area 455 remains unchanged relative to the first area, but the color becomes lighter. It should be understood that this is only exemplary, and in addition to color, the brightness, chroma, clarity, etc. of the area can also be changed, for example.

[0061] Some example embodiments of the present disclosure will be described below. In some embodiments described above, the local area to be adjusted and the adjustment information are selected or provided by the user 140 through an interface element. Alternatively or additionally, in some embodiments, the local area to be adjusted and / or the adjustment information for the local area can be determined based on an interactive message between the user and the digital assistant.

[0062] In some embodiments, the terminal device 110 may determine the first area based on one or more first interaction messages received from the user 140 to the digital assistant. Exemplarily, one or more first interaction messages may indicate image adjustment requirements. In some examples, the first interaction message may be "select all clouds in the image", then the corresponding first area is composed of all clouds in the first image. In some examples, the first interaction message may be "select the little girl in the image", then the corresponding first area is the area where the little girl in the first image is located. In more examples, the first interaction message may also be "I want to select the lower right corner of the image". In this way, the user 140 can express the selection requirement for the first area in the first image through natural language interaction with the digital assistant.

[0063] In some embodiments, the terminal device 110 may select a candidate area in the first image based on an element or position in the first image indicated by one or more interactive messages. In some examples, the elements in the first image may indicate a girl, a cup, a sun, etc. In some examples, the position in the first image may indicate the middle, the lower right corner, the upper left corner, etc. The terminal device 110 may present indicative information about the candidate area. Exemplarily, the terminal device 110 may determine a candidate area based on the interactive message between the user 140 and the digital assistant, and present the candidate area to the user 140. The terminal device 110 may determine the candidate area as the first area after receiving an affirmative indication for the candidate area. In some examples, the user 140 may also make further adjustments to the candidate area before confirming the candidate area.

[0064] In some embodiments, the terminal device 110 may receive one or more second interaction messages sent by the user 140 to the digital assistant, and the one or more second interaction messages describe the image visual requirements. The terminal device 110 may determine the adjustment information based on the one or more second interaction messages. In some examples, the user 140 may express the image processing requirements to the digital assistant through voice input after confirming the first area. For example, the user 140 inputs the following content through voice: "Change the little boy in the image to a little girl." In this way, the user 140 can express the image processing requirements through interaction with the digital assistant, and not necessarily through input box input.

[0065] In some embodiments, the second image may be generated in the following manner. Exemplarily, the terminal device 110 may generate a partial image using a machine learning model based on the first region and the adjustment information. The terminal device 110 determines the second image based on the partial image and the region other than the first region in the first image. Furthermore, the terminal device 110 may use the second image as a processing result of the first image.

[0066] In this way, the embodiment of the present disclosure can accurately process a part of the first image through user input, and can also improve image processing efficiency.

[0067] Example procedures, devices and equipment

[0068] Figure 5 FIG. 5 is a flowchart showing an example process 500 of image processing according to some embodiments of the present disclosure. The process 500 may be implemented at the terminal device 110. Figure 1 The process 500 is described below.

[0069] like Figure 5 As shown, in box 510, the terminal device 110 presents a first image during the interaction between the user and the digital assistant.

[0070] In block 520 , the terminal device 110 determines a first region in the first image and adjustment information for the first region based on an interactive operation from the user, where the adjustment information indicates adjustment of a visual effect of the first region.

[0071] In box 530, in response to receiving the adjustment instruction for the first area, the terminal device 110 presents a second image as the adjustment result of the first image, and a second area corresponding to the first area in the second image has the adjusted visual effect.

[0072] In some embodiments, determining the first area and adjustment information includes: in response to receiving a local adjustment request for the first image, presenting an area selection control and a prompt input entrance; receiving a selection of the first area in the first image via the area selection control; and receiving user input including adjustment information via the prompt input entrance.

[0073] In some embodiments, determining the first region includes: receiving a selection of an initial region in the first image and description information for adjusting the first image; and updating the initial region to the first region based on the description information.

[0074] In some embodiments, the first image is determined based on an interaction message sent by the user to the digital assistant, and the second image is generated in the following manner: based on the adjustment information and the interaction message, determining an adjustment target for the first area; and updating the first image to the second image according to the adjustment target.

[0075] In some embodiments, the first area is determined based on at least one of: a natural language input indicating the area to be adjusted, a predetermined type of operation on the area in the first image, or an interactive control for selecting the area to be adjusted.

[0076] In some embodiments, determining the first area in the first image includes: receiving one or more first interaction messages issued by a user to a digital assistant, the one or more first interaction messages indicating image adjustment requirements; and determining the first area based on the one or more first interaction messages.

[0077] In some embodiments, determining the first area based on one or more first interaction messages includes: selecting a candidate area in the first image based on elements or positions in the first image indicated by the one or more interaction messages; presenting indicative information about the candidate area; and determining the candidate area as the first area in response to receiving a positive indication for the candidate area.

[0078] In some embodiments, determining the adjustment information includes: receiving one or more second interaction messages sent by the user to the digital assistant, the one or more second interaction messages describing the image visual requirements; and determining the adjustment information based on the one or more second interaction messages.

[0079] In some embodiments, the adjustment of the visual effect of the first region includes partial repainting, and the second region includes visual content generated based on the adjustment information.

[0080] In some embodiments, the adjustment of the visual effect of the first region includes a local effect update, and the second region includes partially the same visual content as the first region, or has image attributes of at least one dimension different from the first region.

[0081] In some embodiments, the second image is generated by: generating a local image using a machine learning model based on the first area and the adjustment information; and determining the second image based on the local image and an area in the first image other than the first area.

[0082] The embodiments of the present disclosure also provide corresponding devices for implementing the above methods or processes. Figure 6 A schematic structural block diagram of an example apparatus 600 for image processing according to some embodiments of the present disclosure is shown. The apparatus 600 may be implemented as or included in the terminal device 110. Each module / component in the apparatus 600 may be implemented by hardware, software, firmware, or any combination thereof.

[0083] like Figure 6 As shown, the device 600 includes: a presentation module 610, configured to present a first image during the interaction between the user and the digital assistant; a determination module 620, configured to determine a first area in the first image and adjustment information for the first area based on an interactive operation from the user, the adjustment information indicating an adjustment to the visual effect of the first area; and a receiving module 630, configured to present a second image as an adjustment result of the first image in response to receiving an adjustment indication for the first area, wherein a second area in the second image corresponding to the first area has an adjusted visual effect.

[0084] In some embodiments, the determination module 620 is also configured to present an area selection control and a prompt input entrance in response to receiving a local adjustment request for the first image; receive a selection of a first area in the first image via the area selection control; and receive user input including adjustment information via the prompt input entrance.

[0085] In some embodiments, the determination module 620 is further configured to receive a selection of an initial region in the first image and description information for adjusting the first image; and based on the description information, update the initial region to the first region.

[0086] In some embodiments, the first image is determined based on an interaction message sent by the user to the digital assistant, and the second image is generated in the following manner: based on the adjustment information and the interaction message, determining an adjustment target for the first area; and updating the first image to the second image according to the adjustment target.

[0087] In some embodiments, the first area is determined based on at least one of: a natural language input indicating the area to be adjusted, a predetermined type of operation on the area in the first image, or an interactive control for selecting the area to be adjusted.

[0088] In some embodiments, the determination module 620 is also configured to receive one or more first interaction messages issued by the user to the digital assistant, the one or more first interaction messages indicating image adjustment requirements; and determine the first area based on the one or more first interaction messages.

[0089] In some embodiments, the determination module 620 is also configured to select a candidate area in the first image based on elements or positions in the first image indicated by one or more interaction messages; present indicative information about the candidate area; and in response to receiving a positive indication for the candidate area, determine the candidate area as the first area.

[0090] In some embodiments, the determination module 620 is also configured to receive one or more second interaction messages issued by the user to the digital assistant, the one or more second interaction messages describing the image visual requirements; and determine the adjustment information based on the one or more second interaction messages.

[0091] In some embodiments, the adjustment of the visual effect of the first region includes partial repainting, and the second region includes visual content generated based on the adjustment information.

[0092] In some embodiments, the adjustment of the visual effect of the first region includes a local effect update, and the second region includes partially the same visual content as the first region, or has image attributes of at least one dimension different from the first region.

[0093] In some embodiments, the second image is generated by: generating a local image using a machine learning model based on the first area and the adjustment information; and determining the second image based on the local image and an area in the first image other than the first area.

[0094] Figure 7 A block diagram of an electronic device 700 is shown in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 7 The electronic device 700 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Figure 7 The electronic device 700 shown can be used to implement Figure 1 terminal device 110.

[0095] like Figure 7As shown, the electronic device 700 is in the form of a general electronic device. The components of the electronic device 700 may include, but are not limited to, one or more processors or processing units 710, a memory 720, a storage device 730, one or more communication units 740, one or more input devices 770, and one or more output devices 760. The processing unit 710 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 720. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 700.

[0096] The electronic device 700 typically includes a plurality of computer storage media. Such media may be any accessible media that is accessible to the electronic device 700, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 720 may be a volatile memory (e.g., registers, caches, random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 730 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data and may be accessed within the electronic device 700.

[0097] The electronic device 700 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 7 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 720 may include a computer program product 727 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0098] The communication unit 740 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 700 can be implemented with a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 700 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0099] The input device 770 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 760 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 700 may also communicate with one or more external devices (not shown) through the communication unit 740 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 700, or communicate with any device that allows the electronic device 700 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0100] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0101] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0102] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0103] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0104] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0105] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. An image processing method, comprising: During the interaction between the user and the digital assistant, presenting a first image; Based on an interactive operation from the user, determining a first area in the first image and adjustment information for the first area, the adjustment information indicating an adjustment to a visual effect of the first area; as well as In response to receiving an adjustment instruction for the first area, a second image is presented as an adjustment result of the first image, and a second area in the second image corresponding to the first area has an adjusted visual effect.

2. The method according to claim 1, wherein determining the first region and the adjustment information comprises: In response to receiving a local adjustment request for the first image, presenting a region selection control and a prompt input entry; receiving, via the region selection control, a selection of the first region in the first image; as well as A user input including at least a portion of the adjustment information is received via the prompt input entry.

3. The method of claim 1 , wherein determining the first region comprises: receiving a selection of an initial region in the first image and descriptive information for adjusting the first image; as well as Based on the description information, the initial area is updated to the first area.

4. The method according to claim 1, wherein the first image is determined based on an interactive message sent by the user to the digital assistant, and the second image is generated by: determining an adjustment target for the first area based on the adjustment information and the interaction message; and According to the adjustment target, the first image is updated to the second image.

5. The method of claim 1, wherein the first region is determined based on at least one of: Natural language input indicating the area to be adjusted, performing a predetermined type of operation on a region in the first image, or Interactive control for selecting the area to be adjusted.

6. The method of claim 1 , wherein determining the first region in the first image comprises: receiving one or more first interaction messages sent by the user to the digital assistant, wherein the one or more first interaction messages indicate image adjustment requirements; as well as The first area is determined based on the one or more first interaction messages.

7. The method of claim 6, wherein determining the first area based on the one or more first interaction messages comprises: Selecting a candidate region in the first image based on elements or positions in the first image indicated by the one or more interaction messages; presenting indication information about the candidate area; as well as In response to receiving a positive indication for the candidate region, the candidate region is determined to be the first region.

8. The method of claim 1 , wherein determining the adjustment information comprises: receiving one or more second interaction messages sent by the user to the digital assistant, wherein the one or more second interaction messages describe image visual requirements; as well as The adjustment information is determined based on the one or more second interaction messages. 9 . The method of claim 1 , wherein the adjustment of the visual effect of the first region comprises local re-rendering, and the second region comprises visual content generated based on the adjustment information.

10. The method of claim 1, wherein the adjustment of the visual effect of the first region comprises a local effect update, and the second region comprises partially the same visual content as the first region, or has image attributes of at least one dimension different from the first region.

11. The method according to claim 1, wherein the second image is generated by: Based on the first region and the adjustment information, generating a partial image using a machine learning model; and The second image is determined based on the partial image and a region of the first image excluding the first region.

12. An apparatus for image processing, comprising: a presentation module configured to present a first image during interaction between a user and a digital assistant; a receiving module configured to determine, based on an interactive operation from the user, a first region in the first image and adjustment information for the first region, wherein the adjustment information indicates an adjustment to a visual effect of the first region; as well as The processing module is configured to present a second image as an adjustment result of the first image in response to receiving an adjustment instruction for the first area, wherein a second area corresponding to the first area in the second image has an adjusted visual effect.

13. An electronic device comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 11 when executed by the at least one processing unit.

14. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 11.

Images