Image processing method and device

Abstract

The invention discloses an image processing method and device, and belongs to the technical field of image.The method comprises the steps that first input of first text content is received, and the first text content is used for describing image elements corresponding to the processing requirement of a user for a first image; under the condition that the first image is displayed, in response to the first input, image elements, corresponding to the first text content, in the first image are processed based on the first text content and text description information, and the text description information is used for describing the image content of the first image.

Description

Technical Field

[0001] The present application belongs to the field of image technology, and specifically relates to an image processing method and device. Background Art

[0002] Usually, after a user takes an image with a mobile phone, the taken image can be processed by the image processing software of the mobile phone. For example, if the user needs to adjust the color of a person's clothing in the image, the user first needs to manually determine the image area of ​​the person's clothing that needs to be adjusted through the image processing software in the mobile phone, and then select the color that needs to be adjusted to trigger the image processing software to adjust the color of the image part selected by the user, so that the processed image can meet the user's needs.

[0003] However, in the above method, the user needs to manually draw the image area on the image through the image processing software so that the mobile phone can determine the image area that needs to be adjusted, and then the electronic device can perform subsequent image processing according to the user's needs. In the process of image editing, the steps that the user needs to perform are relatively cumbersome and time-consuming, so the flexibility and convenience of image editing using electronic devices are poor. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an image processing method and device, which can improve the flexibility and convenience of the way in which electronic devices perform image processing.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, which includes: receiving a first input of a first text content, wherein the first text content is used to describe an image element corresponding to a user's processing requirement for a first image; when displaying the first image, in response to the first input, processing the image element in the first image corresponding to the first text content based on the first text content and text description information, wherein the text description information is used to describe the image content of the first image.

[0006] In a second aspect, an embodiment of the present application provides an image processing device, which includes: a receiving module and a processing module; the receiving module is used to receive a first input of a first text content, and the first text content is used to describe the image elements corresponding to the user's processing requirements for the first image; the processing module is used to, when displaying the first image, respond to the first input received by the receiving module, and process the image elements in the first image corresponding to the first text content based on the first text content and text description information, and the text description information is used to describe the image content of the first image.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0008] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0009] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0010] In a sixth aspect, an embodiment of the present application provides a computer program / program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0011] In an embodiment of the present application, an electronic device may receive a first input of a first text content, where the first text content is used to describe an image element corresponding to a user's processing requirement for a first image, so that when the first image is displayed, in response to the first input, the image element corresponding to the first text content in the first image is processed based on the first text content and text description information, where the text description information is used to describe the image content of the first image. In this solution, since the user may input the first text content according to the processing requirement for the first image, so that the electronic device may determine the image element corresponding to the first text content in the first image based on the input first text content and the image description information of the first image, the electronic device may process the image element corresponding to the first text content based on the user's input, so that the processed first image may meet the user's requirement, thereby improving the flexibility and convenience of the electronic device in performing image editing. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is one of the flow charts of an image processing method provided in an embodiment of the present application;

[0013] Figure 2 This is the second flowchart of an image processing method provided in an embodiment of the present application;

[0014] Figure 3 This is one of the example schematic diagrams of an album interface provided in an embodiment of the present application;

[0015] Figure 4AThis is a second example schematic diagram of an album interface provided in an embodiment of the present application;

[0016] Figure 4B This is a third example diagram of an album interface provided in an embodiment of the present application;

[0017] Figure 5A This is one of the example schematic diagrams of displaying text description information provided by the embodiment of the present application;

[0018] Figure 5B This is a second example schematic diagram of displaying text description information provided by an embodiment of the present application;

[0019] Figure 5C This is a third example schematic diagram of displaying text description information provided by an embodiment of the present application;

[0020] Figure 5D This is a fourth example schematic diagram of displaying text description information provided by an embodiment of the present application;

[0021] Fig. 6A This is one of the example schematic diagrams of editing text description information provided by the embodiment of the present application;

[0022] Figure 6B It is one of the example schematic diagrams of displaying an edited image provided by an embodiment of the present application;

[0023] Fig. 7A This is the second example schematic diagram of editing text description information provided by the embodiment of the present application;

[0024] Figure 7B This is a second example schematic diagram of displaying an edited image provided by an embodiment of the present application;

[0025] Fig. 8A This is a third example schematic diagram of editing text description information provided by an embodiment of the present application;

[0026] Figure 8B This is a third example schematic diagram of displaying an edited image provided by an embodiment of the present application;

[0027] Fig. 9A This is a fourth example schematic diagram of editing text description information provided by an embodiment of the present application;

[0028] Fig. 9B This is a fourth example schematic diagram of displaying an edited image provided by an embodiment of the present application;

[0029] Fig. 10A This is one of the example schematic diagrams of a marking image element provided in an embodiment of the present application;

[0030] Fig. 10B This is a second example schematic diagram of a marking image element provided in an embodiment of the present application;

[0031] Fig.11 This is the third flowchart of an image processing method provided in an embodiment of the present application;

[0032] Fig. 12A This is a fourth example diagram of an album interface provided in an embodiment of the present application;

[0033] Fig. 12B This is a fifth example diagram of an album interface provided in an embodiment of the present application;

[0034] Fig. 12C This is a fifth example schematic diagram of displaying an edited image provided by an embodiment of the present application;

[0035] Fig.13 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;

[0036] Fig.14 This is one of the hardware structure diagrams of an electronic device provided in an embodiment of the present application;

[0037] Fig.15 This is the second schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0039] The terms "first", "second", etc. in the specification of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are usually a class, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0040] The terms "at least one (item)", "at least one of" and the like in the specification of the present application refer to any one, any two or a combination of more than two of the objects included therein. For example, at least one (item) of a, b, and c can be represented by: "a", "b", "c", "a and b", "a and c", "b and c" and "a, b and c", where a, b, and c can be single or multiple. Similarly, "at least two (items)" refers to two or more, and its meaning is similar to that of "at least one (item)".

[0041] The image processing method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0042] The image processing method provided in the embodiment of the present application can be applied to scenarios where users need to edit images.

[0043] The image processing method provided in the embodiments of the present application is exemplarily described below using some specific scenarios as examples.

[0044] Scenario 1: The user needs to delete image elements contained in the image

[0045] If the image contains an image element that the user needs to delete, such as image element 1, the user can adjust the text description information corresponding to the image, such as deleting the text content corresponding to the image element 1 in the text description information, to trigger the electronic device to delete the specified image element 1 in the first image according to the user's needs.

[0046] Scenario 2: The user needs to add image elements to the image

[0047] If the image does not contain the image element required by the user, such as image element 2, the user can adjust the text description information corresponding to the image, such as adding text content corresponding to the image element 2, to trigger the electronic device to add image element 2 at the position specified by the user in the first image according to the user's needs.

[0048] Scenario 3: The user needs to modify the image elements contained in the image

[0049] If the image contains image elements that do not meet user needs, such as image element 3, the user can adjust the text description information corresponding to the image, such as modifying the text content corresponding to image element 3, to trigger the electronic device to modify image element 3 in the first image according to user needs.

[0050] It should be noted that the above-mentioned scenarios 1 to 3 are merely illustrative examples of some scenarios in which the embodiments of the present application may be applied. In actual implementation, the embodiments of the present application can also be applied to any possible scenarios in which images are adjusted based on image elements, and the embodiments of the present application are not limited here.

[0051] Based on the above-mentioned scenario applied by the embodiment of the present application, the image processing method provided by the embodiment of the present application, the electronic device can receive a first input of a first text content, and the first text content is used to describe the image element corresponding to the user's processing requirement for the first image, so that when the first image is displayed, in response to the first input, the image element corresponding to the first text content in the first image is processed based on the first text content and text description information, and the text description information is used to describe the image content of the first image. In this scheme, since the user can input the first text content according to the processing requirement for the first image, so that the electronic device can determine the image element corresponding to the first text content in the first image according to the input first text content and the image description information of the first image, the electronic device can process the image element corresponding to the first text content based on the user's input, so that the processed first image can meet the user's needs, thus improving the flexibility and convenience of the electronic device's way of performing image editing.

[0052] The image processing method provided in the embodiment of the present application is executed by an image processing device, which may be an electronic device, or a functional module or entity in an electronic device, and the embodiment of the present application does not limit this. The image processing method provided in the embodiment of the present application will be exemplarily described below using an electronic device as an example.

[0053] The present application provides an image processing method. Figure 1 FIG. 1 is a flow chart of an image processing method provided by an embodiment of the present application. Figure 1 As shown, the image processing method provided in the embodiment of the present application may include the following steps 201 and 202.

[0054] Step 201: An electronic device receives a first input of a first text content.

[0055] In the embodiment of the present application, the first text content is used to describe the image elements corresponding to the user's processing requirements for the first image.

[0056] In some embodiments of the present application, the first text content may include but is not limited to at least one of the following: words, phrases, sentences, and paragraphs.

[0057] It should be noted that the total number of words contained in the above-mentioned first text content can be determined according to the actual needs of the user, and this application does not limit it here.

[0058] In some embodiments of the present application, the first input to the first text content may include, but is not limited to, any of the following: deletion input, addition input, modification input, and copy input.

[0059] Step 202: When displaying the first image, the electronic device processes the image element corresponding to the first text content in the first image based on the first text content and the text description information in response to the first input.

[0060] In the embodiment of the present application, the above text description information is used to describe the image content of the first image.

[0061] In some embodiments of the present application, the electronic device may obtain image feature information of the first image through a convolutional neural network (CNN). Then, the electronic device may input the image feature information of the first image into a generator included in a generative adversarial network (GANs) to obtain text description information corresponding to the first image based on the image feature information of the first image.

[0062] It should be noted that the above-mentioned CNN may include but is not limited to: a convolutional layer, an activation layer, a pooling layer and a fully connected layer.

[0063] Among them, the convolution layer can extract the data features contained in the image through the convolution kernel based on the received image;

[0064] The activation layer can increase the nonlinearity of the network by using a nonlinear activation function, such as ReLU;

[0065] The pooling layer can reduce the spatial size of the data corresponding to the image by performing downsampling operations, and reduce the calculation in the network by extracting more abstract features;

[0066] The fully connected layer can convert the image features extracted by the network into the final output of CNN. The final output information can be classification labels, regression values, or results of other tasks.

[0067] It should be noted that the above GANs is a deep learning model, and the model may at least include: a generator model and a discriminator model. The generator model is used to generate simulated data; the discriminator model is used to determine whether the input data is real or generated.

[0068] In some embodiments of the present application, while the above text description information includes the name of the image element in the first image, the text description information may also include description information of the image element, and grammatical components such as conjunctions and prepositions.

[0069] In some embodiments of the present application, the above-mentioned image elements may include but are not limited to any of the following: people, animals, objects.

[0070] In some embodiments of the present application, the above-mentioned description information of the image element may include but is not limited to any of the following: information describing the image scene, information describing the action of the image element, and information describing the attribute of the image element.

[0071] Exemplarily, the above-mentioned image scene may include but is not limited to any of the following: "ocean", "beach", "grassland", "desert", "sky";

[0072] The actions of the above-mentioned image elements may include but are not limited to any of the following: "standing", "running", "jumping", "lying sideways", "lying flat";

[0073] The attributes of the above image elements may include but are not limited to any of the following: shape, color, and material.

[0074] In some embodiments of the present application, the above-mentioned grammatical components such as conjunctions and prepositions can improve the language fluency of the text description information.

[0075] In some embodiments of the present application, the first image may be at least one image that needs to be edited by the user among images stored in the electronic device.

[0076] In some embodiments of the present application, the first image may include image elements corresponding to the first text content, or may not include image elements corresponding to the first text content.

[0077] In some embodiments of the present application, the above processing of the image elements may include but is not limited to at least one of the following: deleting the image element, adding the image element, modifying the image element, and replacing the image element.

[0078] In some embodiments of the present application, the electronic device may process the image elements corresponding to the first text content in the first image based on the first text content and the text description information through a deep learning model, wherein the deep learning model may include but is not limited to any of the following: a generative adversarial network, a diffusion generation model.

[0079] In the image processing method provided in the embodiment of the present application, since the user can input the first text content according to the processing requirements of the first image, so that the electronic device can determine the image elements corresponding to the first text content in the first image according to the input first text content and the image description information of the first image, the electronic device can process the image elements corresponding to the first text content based on the user's input, so that the processed first image can meet the user's needs. In this way, the flexibility and convenience of the way in which the electronic device performs image editing can be improved.

[0080] In some embodiments of the present application, the first input may include two implementations: one is that when the electronic device displays text description information corresponding to the first image, the user can edit and input the displayed text description information in real time based on the first text content; the other is that when the electronic device displays other interfaces other than the image display interface, the user can copy and input the text content in the currently displayed interface. The following describes these two implementations in detail.

[0081] In some embodiments of the present application, Figure 1 ,like Figure 2 As shown, before the above step 201, the image processing method provided in the embodiment of the present application may further include steps 301 and 302.

[0082] Step 301: The electronic device receives a second input of a first image.

[0083] In some embodiments of the present application, the second input may be an input that triggers the electronic device to display corresponding description information in text form according to the first image.

[0084] In some embodiments of the present application, the second input may include any one of the following: click input, slide input, gesture input, or other feasible input, which is not limited in the embodiments of the present application.

[0085] In some embodiments of the present application, the above-mentioned click input can be any number of click inputs.

[0086] In some embodiments of the present application, the above-mentioned sliding operation can be a sliding operation in any direction, such as sliding up, sliding down, sliding to the left or sliding to the right, etc., which is not limited in the embodiments of the present application.

[0087] In some embodiments of the present application, the above-mentioned gesture input may include but is not limited to at least one of the following: click gesture, sliding gesture, drag gesture, pressure recognition gesture, long press gesture, area change gesture, double press gesture, double click gesture, specific gesture input or other possible gesture inputs. The specific gesture input form can be determined according to actual needs and is not limited to this in the embodiments of the present application.

[0088] Step 302: The electronic device displays text description information in response to the second input.

[0089] In some embodiments of the present application, the above-mentioned text description information may be description information corresponding to the complete first image, or may be description information corresponding to a partial image in the first image.

[0090] In some embodiments of the present application, the electronic device may determine the range of the first image corresponding to the displayed text description information according to the input position of the second input.

[0091] Exemplarily, when the second input is a long press input of a blank area in the first image, the electronic device can display the descriptive information corresponding to the complete first image; when the second input is a long press input of an image element contained in the first image, the electronic device can display the descriptive information corresponding to the image element.

[0092] In some embodiments of the present application, the electronic device may display a floating window in response to the second input to display at least two options, and then the electronic device may receive a user's selection input for one of the options to display corresponding text description information.

[0093] For example, the electronic device is a mobile phone. Figure 3 As shown, the mobile phone can display image 11 through the album interface 10, that is, the first image mentioned above, the upper left corner of the first image contains a "flower" placed in a "flower pot", the upper right corner contains a "dog", the lower right corner contains a "football", and the lower left corner contains a "boy" with short hair and glasses. If the user wants to trigger the electronic device to display the text description information corresponding to image 11, as shown in FIG. Figure 4A As shown, the user can long press the image area corresponding to the "dog" in the image 11, that is, the second input, as shown in FIG. Figure 4BAs shown, the mobile phone is triggered to display a floating window 12 in the bottom area of ​​the album interface 10. The floating window 12 contains two options, such as the "all images" option and the "partial image" option, to prompt the user to select the image range for which text description information needs to be displayed. In particular, when the user selects the "all images" option, the mobile phone can display the text description information corresponding to the complete image 11; when the user selects the "partial image" option, the mobile phone can display the text description information of the image element corresponding to the input position of the second input, that is, the text description information corresponding to the "dog" in the image 11.

[0094] In some embodiments of the present application, when the first image includes multiple image elements, the description order of the multiple image elements in the above text description information may include but is not limited to any of the following: an order determined from top to bottom; an order determined from left to right; an order determined in a clockwise direction starting from the upper left corner; an order determined in a counterclockwise direction starting from the upper left corner.

[0095] In some embodiments of the present application, the electronic device may overlay and display a floating window in the interface displaying the first image to display text description information.

[0096] For example, in combination Figure 4B , if the user selects the "All Images" option, then Figure 5A As shown, the mobile phone can superimpose a floating window 13 in the album interface 10 to display the text description information corresponding to the complete image 11. For example, "Starting from the upper left corner, you can see a large blooming flower in a flowerpot in a clockwise direction. Then, a lively and cute dog appears in the picture, sitting on the ground with its tongue sticking out. Further down, a football comes into view, giving people a sense of movement. Finally, a little boy wearing glasses appears in the picture. He wears short sleeves, carries a backpack, has short and neat hair, and looks very energetic."

[0097] In some embodiments of the present application, since the display size of the interface of the electronic device is limited, the electronic device may display part of the text description information corresponding to the first image in a floating window to reduce the occlusion of the first image by the floating window.

[0098] For example, Figure 5B As shown, the mobile phone can overlay a floating window 14 on the bottom area of ​​the album interface 10 to display part of the text description information corresponding to the complete image 11. For example, "Starting from the upper left corner, you can see a blooming flower in a flowerpot in the clockwise direction. Then, ...", so the user can slide down in the floating window 14 to view the content of the text description information that is not displayed.

[0099] It should be noted that when an electronic device displays text description information through a floating window, the display parameters of the above floating window, such as display position, display size, transparency, etc., can be determined according to the actual needs of the user, and the specific display parameters are not limited in this application.

[0100] In some embodiments of the present application, the electronic device may jump from the interface displaying the first image to display a new interface to display the text description information in full screen.

[0101] For example, in combination Figure 4B , if the user selects the "All Images" option, then Figure 5C As shown, the mobile phone can jump from the album interface 10 to display a new interface, such as interface 20, to display the text description information corresponding to the complete image 11 in full screen. For example, "Starting from the upper left corner, you can see a large blooming flower in a flowerpot in a clockwise direction. Then, a lively and cute dog appears in the picture, sitting on the ground with its tongue sticking out. Further down, a football comes into view, giving people a sense of movement. Finally, a little boy wearing glasses appears in the picture. He is wearing short sleeves, carrying a backpack, and his hair is short and neat, looking very energetic." At the same time, the interface 20 can also include an "OK" control, so that after editing the first text content later, the user can click the "OK" control to trigger the mobile phone to edit the first image.

[0102] In some embodiments of the present application, when the electronic device has a foldable screen or multiple display screens, the electronic device can display the first image through one screen and display text description information of the first image through other screens.

[0103] For example, Figure 5D As shown, the electronic device is a mobile phone with two screens. The mobile phone can display the following on the left half of the screen: Figure 3 The album interface 10 shown in FIG. 10 and the right half of the screen displays the following Figure 5C Interface 20 is shown.

[0104] In some embodiments of the present application, the electronic device may display an adjustment control in a floating window or interface displaying text description information to help the user switch the display form of the text description information, for example, from displaying the text description information through a floating window to displaying the text description information through an interface.

[0105] In some embodiments of the present application, if the above-mentioned first image is multiple images, then when the electronic device displays an image list interface in an album application, the user can select the multiple images through a second input to trigger the electronic device to display text description information corresponding to the multiple images respectively.

[0106] In some embodiments of the present application, when the electronic device displays text description information, the first input may include but is not limited to: deletion input, addition input, and modification input.

[0107] In some embodiments of the present application, when the text description information includes the first text content, the first input may be a deletion input or a modification input for the first text content in the text description information.

[0108] In some embodiments of the present application, when the first text content corresponds to an image element in the first image, the deletion input may be an input for deleting at least part of the description information of the image element.

[0109] In some embodiments of the present application, when the text description information does not include the first text content, the first input may be an input for adding the first text content to the text description information.

[0110] In some embodiments of the present application, after the electronic device receives the above-mentioned editing information, the electronic device may also display a "withdraw" control and a "reset" control in the interface displaying the text description information. Therefore, when the user is editing the first text content, the user can click and input the "withdraw" control displayed in the interface to cancel the latest edited content; the user can click and input the "reset" control displayed in the interface to cancel all edited content.

[0111] In the embodiment of the present application, since the user can edit the text description information corresponding to the first image according to the processing requirements of the first image, when the electronic device displays the text description information corresponding to the first image, the user can more conveniently determine the first text content that needs to be edited. In this way, the user can adjust the image according to actual needs, thereby further improving the flexibility and convenience of the way in which the electronic device performs image editing.

[0112] In some embodiments of the present application, the above-mentioned first input can be an input for deleting the first text content in the text description information; the "based on the first text content and the text description information, processing the image elements corresponding to the first text content in the first image" in the above-mentioned step 202 can be specifically implemented through the following step 202a.

[0113] Step 202a: The electronic device deletes the image element corresponding to the first text content in the first image.

[0114] In some embodiments of the present application, after the electronic device receives the above-mentioned deletion input, the electronic device may directly cancel the display of the first text content; or may mark the deleted first text content in the interface.

[0115] In some embodiments of the present application, the above-mentioned marking method may include but is not limited to at least one of the following: displaying a strikethrough on the first text content, adjusting the color of the first text content, and reducing the font size of the first text content.

[0116] In some embodiments of the present application, the electronic device may, in response to a deletion input for the first text content, first determine an image element in the first image that corresponds to the first text content, and then, through an image editing algorithm, segment and delete the image element in the first image.

[0117] For example, in combination Figure 3 and Figure 5C , if the user wants to delete the “football” contained in the image 11, then after the mobile phone displays the text description information of the image 11 in full screen through the interface 20, the user can trigger the mobile phone to delete the description of “football” in the text description information, that is, “further down, a football comes into view, giving people a sense of movement.”, that is, the first text content mentioned above, and then, Fig. 6A As shown, the mobile phone can display a strikethrough on the deleted text content to prompt the user of the deleted text content. Then, when the user is sure that the deleted text content is correct, the user can click the "OK" control in the interface 21, thereby triggering the mobile phone to delete the deleted text content. Figure 3 In the image 11 shown, the image element of "soccer" is deleted, such as Figure 6B As shown, the image 31 is displayed in the album interface 10 .

[0118] In some embodiments of the present application, after the above step 202a, the electronic device may also perform image filling processing on the image area corresponding to the deleted image element in the first image after deletion processing based on the image data of the image area adjacent to the deleted image element in the original first image.

[0119] For example, in combination Figure 3 and Figure 5C, if the user wants to delete the backpack of the "boy" in image 11, then after the mobile phone displays the text description information of image 11 in full screen through interface 20, the user can trigger the mobile phone to delete the part of the description about the "boy's" clothing in the text description information, that is, delete the "carrying a backpack" in "Finally, a little boy wearing glasses appeared in the picture. He was wearing short sleeves, carrying a backpack, with short and neat hair, and looked very energetic." Therefore, after the user confirms to trigger the mobile phone to perform the corresponding operation, the mobile phone can Figure 3 In the image 11 shown, the image of the backpack displayed on the shoulder part of the "boy" is deleted, and the image area of ​​the backpack displayed on the shoulder part of the "boy" is filled with the image of the "short-sleeved shirt" worn by the "boy".

[0120] In an embodiment of the present application, the user can delete the first text content in the text description information according to actual needs to trigger the electronic device to delete the image element corresponding to the first text content in the first image. In other words, the electronic device can adjust the image elements of the first image according to the user's editing of the image description information. In this way, the flexibility and convenience of the way in which the electronic device performs image editing can be improved.

[0121] In some embodiments of the present application, the above-mentioned first input is an input for adding the first text content in the text description information; the "based on the first text content and the text description information, processing the image elements in the first image corresponding to the first text content" in the above-mentioned step 202 can be specifically implemented through the following step 202b.

[0122] Step 202b: The electronic device adds an image element corresponding to the first text content in the first image.

[0123] In some embodiments of the present application, after the electronic device receives the above-mentioned addition input, the electronic device may mark the newly added first text content in the interface.

[0124] In some embodiments of the present application, the above-mentioned marking method may include but is not limited to at least one of the following: adjusting the color of the first text content, increasing the font size of the first text content, displaying an underline below the first text content, and displaying the first text content in bold.

[0125] In some embodiments of the present application, the electronic device can, in response to an additional input of a first text content, first obtain an image element corresponding to the first text content, and then, based on the position of the first text content in the text description information, determine the display position of the image element corresponding to the first text content on the first image, and add it to the first image.

[0126] In some embodiments of the present application, the method for the electronic device to obtain the image element corresponding to the first text content may include but is not limited to any of the following:

[0127] Generate corresponding image elements based on the first text content by using a model that can be used for text images in a deep learning algorithm;

[0128] Based on the first text content, searching in an image element library stored in the electronic device or in the cloud, and determining the matched image element as the image element corresponding to the first text content;

[0129] Based on the first text content, an image element corresponding to the first text content is searched through a network.

[0130] In some embodiments of the present application, when the first text content includes a description related to the display position, the electronic device can directly determine the display position of the image element corresponding to the first text content in the first image.

[0131] For example, in combination Figure 3 and Figure 5C , if the user wants to add a "sun" to the upper right of the "flower" in the image 11, after the mobile phone displays the text description information of the image 11 in full screen through the interface 20, the user can trigger the description of the "sun" in the text description information of the mobile phone, that is, the first text content mentioned above, such as "there is a sun in the upper right of the flower". Then, Fig. 7A As shown, the mobile phone can boldly display the newly added text content in the text description information displayed on the interface 22. Then, when the user determines that the newly added text content is correct, the user can click the "OK" control in the interface 22, thereby triggering the mobile phone to confirm the new text content based on the text description information displayed on the interface 22. Figure 3 The image 11 shown shows the image element of "sun" at the upper right of "flower", such as Figure 7B As shown, the image 32 is displayed in the album interface 10 .

[0132] In some embodiments of the present application, when the first text content does not contain a description related to the display position, the electronic device may determine the display position of the image element corresponding to the first text content based on the association relationship between the first text content and the image elements contained in the first image.

[0133] For example, assuming that the first image includes the sky and the ocean. If the first text content is a description of "fish" but does not clearly indicate the location of the "fish", the electronic device can determine that there is an association relationship between "fish" and "ocean" based on the image elements included in the first image, such as "sky" and "ocean", and thus add the image element of "fish" in the display area of ​​"ocean".

[0134] In some embodiments of the present application, after the above step 202b, the electronic device may further adjust the image display parameters of the newly added image elements based on the image display parameters of the original first image to indicate the display effect of the first image.

[0135] In some embodiments of the present application, the above-mentioned image display parameters may include but are not limited to at least one of the following: brightness, contrast, and saturation.

[0136] In an embodiment of the present application, the user can add a first text content to the text description information according to actual needs to trigger the electronic device to add an image element corresponding to the first text content to the first image. The electronic device can adjust the image elements of the first image accordingly based on the user's editing of the image description information. In this way, the flexibility and convenience of the way in which the electronic device performs image editing can be improved.

[0137] In some embodiments of the present application, the above-mentioned first input is an input for modifying the second text content in the text description information into the first text content; the above-mentioned step 202 of "based on the first text content and the text description information, processing the image elements in the first image corresponding to the first text content" can be specifically implemented through the following step 202c.

[0138] Step 202c: The electronic device adjusts the image elements in the first image corresponding to the second text content to image elements corresponding to the first text content.

[0139] In some embodiments of the present application, the first input may include but is not limited to at least one of the following: input of a noun, input of an adjective, and input of a verb.

[0140] In some embodiments of the present application, after the electronic device receives the above-mentioned modification input, the electronic device may mark the modified first text content in the interface.

[0141] In some embodiments of the present application, the above-mentioned marking method may include but is not limited to at least one of the following: adjusting the color of the first text content, increasing the font size of the first text content, displaying an underline below the first text content, and displaying the first text content in bold.

[0142] In some embodiments of the present application, the electronic device may, in response to the first input of the second text content, first determine the image element corresponding to the second text content in the first image, and segment and delete the image element in the first image through an image editing algorithm. Then, the electronic device may obtain the image element corresponding to the first text content according to the first text content, and display the image element corresponding to the first text content in the display area of ​​the image element corresponding to the second text content.

[0143] It should be noted that the method for the electronic device to obtain the image element corresponding to the first text content can refer to the relevant description in the above step 202b, and the embodiment of the present application will not be described in detail here.

[0144] In some embodiments of the present application, when the above-mentioned first input is the first input of a noun, the electronic device may replace the image element corresponding to the second text content with the image element corresponding to the first text content.

[0145] For example, in combination Figure 3 and Figure 5C If the user wants to change the word "football" in the image 11 to "basketball", after the mobile phone displays the text description information of the image 11 in full screen through the interface 20, the user can trigger the mobile phone to change the word "football" in the text description information to "basketball". Fig. 8A As shown, the mobile phone can increase the font size of the modified text content in the text description information displayed on the interface 23. Then, when the user determines that the modified text content is correct, the user can click the "OK" control in the interface 23, thereby triggering the mobile phone to Figure 3 In the image 11 shown, replace "football" with "basketball", such as Figure 8B As shown, the image 33 is displayed in the album interface 10 .

[0146] In some embodiments of the present application, when the above-mentioned first input is a first input of any one of an adjective or a verb, the electronic device can adjust the display parameters corresponding to the second text content to the display parameters corresponding to the first text content.

[0147] For example, suppose that the first image contains a girl wearing a red skirt, and the text description information corresponding to the first image contains "a girl wearing a red skirt". If the user wants to change the color of the skirt in the first image, the user can change "a girl wearing a red skirt" to "a girl wearing a blue skirt". Then, the electronic device can adjust the color of the skirt in the first image from red to blue.

[0148] As another example, assume that the first image includes a standing person, and the text description information corresponding to the first image contains "a standing person". If the user wants to adjust the action of the person in the first image from standing to sitting, the "a standing person" can be modified to "a person sitting on a chair". Then, the electronic device can adjust the action of the person in the first image, and can add a chair to the first image to realize "a person sitting on a chair". Alternatively, when the first image includes a chair, the electronic device can adjust the display position of the person to the display position of the chair in the first image; or alternatively, the electronic device can adjust the display position of the chair in the first image to the display position of the person.

[0149] In some embodiments of the present application, after the above step 202c, the electronic device may further adjust the image display parameters of the modified image elements based on the image display parameters of the original first image to indicate the display effect of the first image.

[0150] In some embodiments of the present application, the above-mentioned image display parameters may include but are not limited to at least one of the following: brightness, contrast, and saturation.

[0151] In an embodiment of the present application, the user can modify the first text content in the text description information according to actual needs to trigger the electronic device to modify the image elements in the first image corresponding to the first text content. Therefore, the electronic device can adjust the image elements of the first image accordingly based on the user's editing of the image description information. In this way, the flexibility and convenience of the way in which the electronic device performs image editing can be improved.

[0152] In some embodiments of the present application, the above-mentioned first input is used to adjust the display order of the first text content in the text description information; the above-mentioned step 202 of "based on the first text content and the text description information, processing the image elements in the first image corresponding to the first text content" can be specifically implemented through the following step 202d.

[0153] Step 202d: The electronic device adjusts the display position of the image element corresponding to the first text content in the first image.

[0154] In some embodiments of the present application, after the electronic device receives the above adjustment input, the electronic device may mark the adjusted first text content in the interface.

[0155] In some embodiments of the present application, the above-mentioned marking method may include but is not limited to at least one of the following: adjusting the color of the first text content, increasing the font size of the first text content, displaying an underline below the first text content, and displaying the first text content in bold.

[0156] In some embodiments of the present application, when the text description information includes content related to the arrangement order, the electronic device can adjust the display order of the image elements in the first image based on the adjusted text description information.

[0157] For example, in combination Figure 3 and Figure 5C , if the user wants to change the display position of "football" and "boy" in the image 11, after the mobile phone displays the text description information of the image 11 in full screen through the interface 20, the user can trigger the mobile phone to adjust the description order of "football" and "boy" in the text description information. Then, Fig. 9A As shown, the mobile phone can add an underline to the adjusted text content in the text description information displayed on the interface 24. Then, when the user determines that the adjusted text content is correct, the user can click the "OK" control in the interface 24, thereby triggering the mobile phone to Figure 3 As shown in the image 11, the display positions of "soccer" and "boy" are adjusted, such as Fig. 9B As shown, the image 34 is displayed in the album interface 10 .

[0158] In an embodiment of the present application, the user can adjust the position of the first text content in the text description information according to actual needs to trigger the electronic device to adjust the display position of the image element corresponding to the first text content in the first image. Therefore, the electronic device can adjust the image elements of the first image accordingly based on the user's editing of the image description information. In this way, the flexibility and convenience of the way in which the electronic device performs image editing can be improved.

[0159] It should be noted that the methods for marking the first text content in the above steps 202a to 202d are different. Thus, the user can determine the image editing method corresponding to the text content through different marking methods.

[0160] In some embodiments of the present application, when the text description information of the first image displayed by the electronic device does not affect the display of the first image, the electronic device can edit the first image in real time according to the user's editing of the first text content.

[0161] For example, in combination Figure 5D Since the mobile phone displays the first image and the text description information corresponding to the first image through the left half screen and the right half screen respectively, the mobile phone can synchronously edit the first image displayed on the left half screen according to the user's editing of the text description information displayed on the right half screen of the mobile phone. In this way, the user can more conveniently determine whether the adjusted first image meets his needs.

[0162] In some embodiments of the present application, when the first image is a plurality of images, the electronic device may perform corresponding image editing processing according to the adjusted text description information corresponding to each image.

[0163] In some embodiments of the present application, when the above-mentioned first image is multiple images, the electronic device can display a text editing box for uniformly processing the text description information of the multiple images in the interface of the text description information corresponding to the multiple images. Then, the electronic device can perform the same editing processing on the multiple images according to the first text information input by the user in the text editing box.

[0164] It should be noted that, if there are images among the multiple images that cannot be edited or that have failed to be edited, the electronic device may display a prompt message to prompt the user which image has failed to be edited, and may prompt the user the reason why the image editing has failed.

[0165] In some embodiments of the present application, after the above step 302, the image processing method provided by the embodiment of the present application may further include the following steps 401 and 402, or the following steps 401 and 403.

[0166] Step 401: The electronic device receives a third input of a third text content in displayed text description information.

[0167] In some embodiments of the present application, the third text content may be at least part of the nouns included in the text description information.

[0168] In some embodiments of the present application, the third input may be an input for adjusting the display form of the third text content.

[0169] In some embodiments of the present application, the third input may include any one of the following: click input, slide input, gesture input, or other feasible input, which is not limited in the embodiments of the present application.

[0170] Step 402: The electronic device marks an image element corresponding to a third text content in the first image in response to a third input.

[0171] In some embodiments of the present application, the above-mentioned marking method may include but is not limited to any of the following: displaying a marking box, adjusting the color, thickening the edge, and displaying an arrow.

[0172] For example, in combination Figure 5B ,like Fig. 10AAs shown, if the user wants to view the image element corresponding to the "flower" in the text description information, the user can long press the text content "flower", which is the third input of the third text content. Then, the mobile phone can display a marking box 15 in the area of ​​the image element "flower" in the image 11 to prompt the user of the image element corresponding to the text content "flower".

[0173] In some embodiments of the present application, if the floating window of the text description information displayed in the interface blocks the image element corresponding to the third text content, the electronic device can first adjust the display parameters of the floating window, such as the display position and display size. Then, if the image element corresponding to the third text content is not blocked, the image element is marked.

[0174] In this way, the user can compare text and image more conveniently by marking the image elements through the electronic device, thereby facilitating the user to determine the image elements that need to be edited later.

[0175] Step 403: In response to the third input, the electronic device replaces the third text content with an image element corresponding to the third text content in the displayed text description information.

[0176] In some embodiments of the present application, when the electronic device displays the text description information in full screen, the electronic device may replace the third text content with an image element corresponding to the third text content in response to the third input.

[0177] For example, in combination Figure 5C , when the mobile phone displays the text description content in full screen through the interface 20, if the user wants to view the image elements in the image 11, the user can long press the blank area in the interface 20, such as Fig. 10B As shown, the mobile phone can convert “flowers”, “flower pots”, “dog”, “football” and “little boy” in the text description information into image elements in the first image through interface 25 .

[0178] In this way, the user can more conveniently determine the description of the image element in the text description information by replacing the displayed image element, thereby facilitating the user to determine the first text content that needs to be edited later.

[0179] In some embodiments of the present application, Figure 1 ,like Fig.11 As shown, the above step 201 can be specifically implemented through the following step 201a, and the above step 202 can be specifically implemented through the following step 202e.

[0180] Step 201a: The electronic device receives a copy input of the first text content.

[0181] In some embodiments of the present application, the first text content may be text content displayed in any interface of the electronic device.

[0182] Step 202e: In response to the copy input, the electronic device processes the image element corresponding to the first text content in the first image when the first text content matches the text description information.

[0183] In some embodiments of the present application, the matching of the first text content and the text description information may be understood as: the first text content and the text description information contain descriptions of the same image elements.

[0184] For example, the first text content and the text description information contain the same keywords, or the scene corresponding to the first text content is the same as the scene of the text description information, or other possible matching relationships.

[0185] In some embodiments of the present application, when an electronic device displays an image, the electronic device can automatically obtain text description information corresponding to the image, and then, when it is detected that the clipboard includes copied text content, detect whether the text description information corresponding to the image matches the copied text content.

[0186] In some embodiments of the present application, the above-mentioned processing methods for image elements may include but are not limited to at least one of the following: deleting image elements, adding image elements, adjusting the color of image elements, and adjusting the display position of image elements.

[0187] For example, suppose the first text content copied by the mobile phone according to the user's input is: "Today's weather is really good, with blue sky, white clouds, bright sunshine, and coconut trees on the beach. It's just that I didn't bring a sun hat today, otherwise this photo would be perfect." Therefore, when the mobile phone displays Fig. 12A In the case of the image 16 shown, the mobile phone can obtain the text description information corresponding to the image 16, such as "a girl stands leisurely on the beach, she has a smile on her face, she is wearing a simple short-sleeved dress, and her long hair flutters gently in the sea breeze. Behind the girl, there are two coconut trees and rolling waves. In the sky, the sun is high, emitting warm and soft light. A few clouds float leisurely in the sky, contrasting with the sun, forming a harmonious and beautiful picture."

[0188] Further, based on the copied first text content and the text description information corresponding to the image 16, the mobile phone determines that both contain relevant descriptions of "white clouds", "sun", "sea" and "coconut trees", so the mobile phone determines that the first text content matches the text description information of the image 16. Fig. 12BAs shown, the mobile phone may display a floating window 17, which includes prompt information "Do you want to adjust the image elements according to the copied content?", a "Cancel" control and an "OK" control. Then, after the user clicks the "OK" control in the floating window 17, the mobile phone may, based on the first text content "It is because I did not wear a sun hat today, otherwise this photo would be perfect", Fig. 12C As shown, "sun hat" is added to "girl" contained in image 16. Thus, the step of processing the image element corresponding to the first text content in the first image is realized.

[0189] In some embodiments of the present application, after the electronic device receives a copy input of a first text content, when the electronic device displays an image list interface of an album application, the electronic device can determine that the user has a need to modify the image based on the copied first text content, and the electronic device can provide the user with at least one image with the same attributes based on the attributes of the interface for copying the first text content.

[0190] The above attributes may include but are not limited to at least one of the following: creation time, creation location, and latest modification date.

[0191] For example, assuming that the first text content is the text content copied by the user from the memo, when the user triggers the mobile phone to display the image list interface in the album application, the electronic device can first determine at least one image that the user may need to modify based on the first text content according to the creation time and creation location corresponding to the first text content, and display it in a designated area of ​​the image list interface, such as the side area. Then, the user can more conveniently select the first image that needs to be adjusted.

[0192] In an embodiment of the present application, the electronic device can edit the first image based on the copied first text content, thereby improving the flexibility and convenience of the electronic device in performing image editing.

[0193] For various scenarios in which the embodiments of the present application can be applied, combined with the various implementation schemes of the embodiments of the present application described above, specific examples are given below to illustrate the implementation process of the embodiments of the present application in various scenarios. Fig. 12A The image 16 shown is provided as an example.

[0194] Scenario 1: The user needs to delete image elements contained in the image

[0195] Example 1: When the mobile phone displays image 16, if the user wants to delete the "coconut tree" contained in image 16, the user can first long press the display area of ​​the "coconut tree" in image 16 to trigger the mobile phone to display a floating window, such as floating window 1, on image 16, and two options can be displayed in the floating window 1, such as the "full image" option and the "partial image" option, to instruct the user to select the range of text descriptions to be displayed. If the user selects the "partial image" option, the mobile phone can display the text description information corresponding to the "coconut tree" in the image area that does not block the "coconut tree", such as "behind the girl, there are two coconut trees and rolling waves". Then, if the user wants to determine the correspondence between the "coconut tree" described in the text description information and the image, the user can long press the "coconut tree" in the text description information to trigger the mobile phone to display a marking box in image 16 to mark the image elements that the user may need to modify.

[0196] Furthermore, the user can delete the relevant description of "coconut trees", that is, "two coconut trees". Then, the mobile phone can delete the image of "coconut trees" based on image 16, and fill and adjust the displayed image of "coconut trees" based on the area adjacent to the display area of ​​​​the "coconut trees", thereby providing the user with an image that meets the needs.

[0197] Scenario 2: The user needs to add image elements to the image

[0198] Example 2, when the mobile phone displays image 16, if the user wants to add a "sun hat" to image 16, the user can first long press image 16 to trigger the mobile phone to display a floating window superimposed on image 16, such as floating window 1, and two options can be displayed in the floating window 1, such as the "all images" option and the "partial image" option, to instruct the user to select the range of text descriptions to be displayed. If the user selects the "all images" option, the mobile phone can jump to display a new interface to display the text description information corresponding to image 16 in full screen, such as "A girl stands leisurely on the beach. She has a smile on her face and wears a simple short-sleeved dress. Her long hair flutters gently in the sea breeze. Behind the girl, there are two coconut trees and rolling waves. In the sky, the sun is high, emitting warm and soft light. A few clouds float leisurely in the sky, contrasting with the sun, forming a harmonious and beautiful picture." Then, if the user wants to determine the correspondence between the image elements described in the text description information and the image, the user can click on the control displayed in the interface for adjusting the display form of the text content to trigger the mobile phone to replace the text "girl" displayed in the interface with the "girl" image in image 16, the text "coconut tree" with the "coconut tree" image in image 16, the text "sun" with the "sun" image in image 16, and the text "clouds" with the "clouds" image in image 16.

[0199] Furthermore, the user can add a description about the sun hat in the description information about the "girl" in the text description information, such as "she is wearing a sun hat on her head". Then, the mobile phone can add an image of the "sun hat" on the head of the "girl" based on image 16, and adjust the display parameters of the "sun hat" accordingly based on the contrast, brightness and other display parameters of the display area of ​​the "girl", thereby providing the user with an image that meets the needs.

[0200] Example 3: If the user wants to add a "sun hat" to image 16, the user can first copy the description containing "sun hat" in other applications except the photo album application. For example, the user can copy "Today's weather is really good, with blue sky, white clouds, bright sunshine, and coconut trees on the beach in the travel record in the memo. It's just that I didn't bring a sun hat today, otherwise this photo would be perfect", which is the first text content. Then, the user can trigger the mobile phone to run the photo album application. When the mobile phone displays the image list interface, the mobile phone can filter the images contained in the photo album application according to the creation time and creation place of the travel record of the first text content copied by the user, so as to display matching images to the user. Then, the user can select the image that needs to be adjusted, that is, image 16.

[0201] When the mobile phone displays image 16, the mobile phone can obtain text description information corresponding to image 16, such as "a girl stands leisurely on the beach, smiling, wearing a simple short-sleeved dress, and her long hair fluttering gently in the sea breeze. Behind the girl, there are two coconut trees and rolling waves. In the sky, the sun is high, emitting warm and soft light. A few clouds float leisurely in the sky, contrasting with the sun, forming a harmonious and beautiful picture." Based on the copied first text content and the text description information corresponding to image 16, the mobile phone determines that both contain relevant descriptions of "white clouds", "sun", "sea" and "coconut trees", so the mobile phone determines that the first text content matches the text description information of image 16, so the mobile phone can display a floating window, such as floating window 2, which includes prompt information "Do you want to adjust the picture elements according to the copied content?", "Cancel" control and "OK" control. Then, after the user clicks the "OK" control in the floating window 2, the mobile phone can add a "sun hat" to the "girl" contained in the image 16 based on the first text content "It's just that I didn't wear a sun hat today, otherwise this photo would be perfect", thereby providing the user with an image that meets the needs.

[0202] Scenario 3: The user needs to modify the image elements contained in the image

[0203] Example 4, when the mobile phone displays image 16, if the user adjusts the style of the dress worn by the "girl" in image 16, the user can first long press image 16 to trigger the mobile phone to display a floating window superimposed on image 16, such as floating window 1, and two options can be displayed in the floating window 1, such as the "all images" option and the "partial image" option, to instruct the user to select the range of text descriptions to be displayed. If the user selects the "all images" option, the mobile phone can jump to display a new interface to display the text description information corresponding to image 16 in full screen, such as "A girl stands leisurely on the beach, with a smile on her face, wearing a simple short-sleeved dress, and her long hair fluttering gently in the sea breeze. Behind the girl, there are two coconut trees and rolling waves. In the sky, the sun is high, emitting warm and soft light. A few clouds float leisurely in the sky, contrasting with the sun, forming a harmonious and beautiful picture." Then, if the user wants to determine the correspondence between the image elements described in the text description information and the image, the user can click on the control displayed in the interface for adjusting the display form of the text content to trigger the mobile phone to replace the text "girl" displayed in the interface with the "girl" image in image 16, the text "coconut tree" with the "coconut tree" image in image 16, the text "sun" with the "sun" image in image 16, and the text "clouds" with the "clouds" image in image 16.

[0204] Furthermore, the user can adjust the description of the dress in the description information about the "girl" in the text description information, such as changing "wearing a simple short-sleeved dress" to "wearing a simple long-sleeved dress". Then, the mobile phone can adjust the style of the dress worn by the "girl" based on image 16, and adjust the display parameters of the "dress" accordingly based on the contrast, brightness and other display parameters of the display area of ​​the "girl", thereby providing the user with an image that meets the needs.

[0205] Example 5: If the user adjusts the style of the dress worn by the "girl" in image 16, the user can first copy the description containing "dress" in other applications except the photo album application. For example, the user can copy "Today's weather is really good, with blue sky, white clouds, bright sunshine, and coconut trees on the beach. It would be nice if I wore a long-sleeved dress today" in the travel record in the memo, which is the first text content. Then, the user can trigger the mobile phone to run the photo album application. When the mobile phone displays the image list interface, the mobile phone can filter the images contained in the photo album application according to the creation time and creation place of the travel record of the first text content copied by the user, so as to display matching images to the user. Then, the user can select the image that needs to be adjusted, that is, image 16.

[0206] When the mobile phone displays image 16, the mobile phone can obtain text description information corresponding to image 16, such as "a girl stands leisurely on the beach, smiling, wearing a simple short-sleeved dress, and her long hair fluttering gently in the sea breeze. Behind the girl, there are two coconut trees and rolling waves. In the sky, the sun is high, emitting warm and soft light. A few clouds float leisurely in the sky, contrasting with the sun, forming a harmonious and beautiful picture." Based on the copied first text content and the text description information corresponding to image 16, the mobile phone determines that both contain relevant descriptions of "white clouds", "sun", "sea", "coconut trees" and "dress", so the mobile phone determines that the first text content matches the text description information of image 16, so the mobile phone can display a floating window, such as floating window 2, which includes prompt information "Do you want to adjust the picture elements according to the copied content?", "Cancel" control and "OK" control. Then, after the user clicks the "OK" control in floating window 2, the mobile phone can adjust the "short-sleeved dress" worn by the "girl" in image 16 to a "long-sleeved dress" based on the "I wish I was wearing a long-sleeved dress today" in the first text content, thereby providing the user with an image that meets the needs.

[0207] It should be noted that the above-mentioned method embodiments, or various possible implementation methods in each method embodiment, can be executed separately, or, under the premise that there is no contradiction, can also be executed in combination with each other. The specific implementation can be determined according to actual usage requirements, and the embodiments of the present application do not limit this.

[0208] It should be noted that the image processing method provided in the embodiment of the present application can be executed by an image processing device. In the embodiment of the present application, an image processing device executing the image processing method is taken as an example to illustrate the image processing device provided in the embodiment of the present application.

[0209] Fig.13 A possible structural diagram of an image processing device involved in an embodiment of the present application is shown. Fig.13 As shown, the image processing device 70 may include: a receiving module 71 and a processing module 72;

[0210] The receiving module 71 is used to receive a first input of a first text content, where the first text content is used to describe an image element corresponding to a user's processing requirement for a first image;

[0211] The processing module 72 is used to process the image elements corresponding to the first text content in the first image based on the first text content and text description information in response to the first input received by the receiving module 71 when displaying the first image, and the text description information is used to describe the image content of the first image.

[0212] In a possible implementation, the image processing device 70 provided in the embodiment of the present application may also include: a display module; the above-mentioned receiving module 71 is also used to receive a second input for the first image before receiving a first input for the first text content; the above-mentioned display module is used to display text description information in response to the second input received by the receiving module 71.

[0213] In a possible implementation, the first input is an input for deleting the first text content in the text description information; and the processing module 72 is specifically configured to delete the image element corresponding to the first text content in the first image.

[0214] In a possible implementation, the first input is an input for adding a first text content to the text description information; the processing module 72 is specifically configured to add an image element corresponding to the first text content to the first image.

[0215] In a possible implementation, the first input is an input for modifying the second text content in the text description information into the first text content; the processing module 72 is specifically used to adjust the image elements in the first image corresponding to the second text content into the image elements corresponding to the first text content.

[0216] In a possible implementation, the first input is used to adjust the display order of the first text content in the text description information; the processing module 72 is specifically used to adjust the display position of the image element corresponding to the first text content in the first image.

[0217] In a possible implementation, the above-mentioned receiving module 71 is specifically used to receive a copy input of the first text content; the above-mentioned processing module 72 is specifically used to process the image elements corresponding to the first text content in the first image in response to the copy input when the first text content matches the text description information.

[0218] In a possible implementation, the above-mentioned receiving module 71 is also used to receive a third input for a third text content in the displayed text description information after the display module displays the text description information; the above-mentioned display module is used to mark the image element corresponding to the third text content in the first image in response to the third input received by the receiving module 71, or, in the displayed text description information, replace the third text content with the image element corresponding to the third text content.

[0219] In the image processing device provided in the embodiment of the present application, since the user can input the first text content according to the processing requirements of the first image, so that the image processing device can determine the image elements corresponding to the first text content in the first image according to the input first text content and the image description information of the first image, the image processing device can process the image elements corresponding to the first text content based on the user's input, so that the processed first image can meet the user's needs. In this way, the flexibility and convenience of the way in which the image processing device performs image editing can be improved.

[0220] The image processing device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0221] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0222] The image processing device provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and will not be described again here to avoid repetition.

[0223] Alternatively, if Fig.14 As shown, an embodiment of the present application further provides an electronic device 90, including a processor 91 and a memory 92, wherein the memory 92 stores programs or instructions that can be executed on the processor 91, and when the program or instructions are executed by the processor 91, the various steps of the above-mentioned image processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0224] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0225] Fig.15 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0226] The electronic device 100 includes but is not limited to components such as a radio frequency unit 101, a network module 102, an audio output unit 103, an input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, and a processor 110.

[0227] Those skilled in the art will appreciate that the electronic device 100 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.15 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0228] The user input unit 107 is used to receive a first input of a first text content, where the first text content is used to describe an image element corresponding to a user's processing requirement for the first image;

[0229] The processor 110 is used to process the image elements corresponding to the first text content in the first image in response to the first input based on the first text content and the text description information when displaying the first image, where the text description information is used to describe the image content of the first image.

[0230] Optionally, the user input unit 107 is further configured to receive a second input of the first image before receiving a first input of the first text content; and the display unit 106 is configured to display text description information in response to the second input.

[0231] Optionally, the first input is an input for deleting a first text content in the text description information; the processor 110 is specifically configured to delete an image element corresponding to the first text content in the first image.

[0232] Optionally, the first input is an input for adding a first text content to the text description information; the processor 110 is specifically configured to add an image element corresponding to the first text content to the first image.

[0233] Optionally, the first input is an input for modifying the second text content in the text description information into the first text content; the processor 110 is specifically used to adjust the image elements corresponding to the second text content in the first image into image elements corresponding to the first text content.

[0234] Optionally, the first input is used to adjust the display order of the first text content in the text description information; the processor 110 is specifically used to adjust the display position of the image element corresponding to the first text content in the first image.

[0235] Optionally, the user input unit 107 is specifically used to receive a copy input of the first text content; the processor 110 is specifically used to process the image element corresponding to the first text content in the first image in response to the copy input when the first text content matches the text description information.

[0236] Optionally, the user input unit 107 is further used to receive a third input of a third text content in the displayed text description information after displaying the text description information; the display unit 106 is further used to mark an image element corresponding to the third text content in the first image in response to the third input, or to replace the third text content with an image element corresponding to the third text content in the displayed text description information.

[0237] In the electronic device provided in the embodiment of the present application, since the user can input the first text content according to the processing requirements of the first image, so that the electronic device can determine the image elements corresponding to the first text content in the first image according to the input first text content and the image description information of the first image, the electronic device can process the image elements corresponding to the first text content based on the user's input, so that the processed first image can meet the user's needs. In this way, the flexibility and convenience of the way in which the electronic device performs image editing can be improved.

[0238] The electronic device provided in the embodiment of the present application can implement each process implemented in the above method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0239] The beneficial effects of various implementations in this embodiment can be specifically referred to the beneficial effects of the corresponding implementations in the above method embodiment. To avoid repetition, they will not be described again here.

[0240] It should be understood that in the embodiment of the present application, the input unit 104 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes a touch panel 1071 and at least one of other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0241] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0242] The processor 110 may include one or more processing units; optionally, the processor 110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 110.

[0243] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0244] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0245] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0246] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0247] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0248] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0249] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0250] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Receiving a first input of a first text content, where the first text content is used to describe an image element corresponding to a user's processing requirement for a first image; When the first image is displayed, in response to the first input, image elements in the first image corresponding to the first text content are processed based on the first text content and text description information, and the text description information is used to describe the image content of the first image.

2. The method according to claim 1, characterized in that Before receiving the first input of the first text content, the method further includes: receiving a second input of the first image; In response to the second input, the text description information is displayed.

3. The method according to claim 2, characterized in that The first input is an input for deleting the first text content in the text description information; The processing of the image element corresponding to the first text content in the first image based on the first text content and the text description information includes: The image element corresponding to the first text content is deleted from the first image.

4. The method according to claim 2, characterized in that: The first input is an input for adding the first text content to the text description information; The processing of the image element corresponding to the first text content in the first image based on the first text content and the text description information includes: An image element corresponding to the first text content is added to the first image.

5. The method according to claim 2, characterized in that: The first input is an input for modifying the second text content in the text description information into the first text content; The processing of the image element corresponding to the first text content in the first image based on the first text content and the text description information includes: The image element in the first image corresponding to the second text content is adjusted to the image element corresponding to the first text content.

6. The method according to claim 2, characterized in that The first input is an input for adjusting the display order of the first text content in the text description information; The processing of the image element corresponding to the first text content in the first image based on the first text content and the text description information includes: The display position of the image element in the first image corresponding to the first text content is adjusted.

7. The method according to claim 1, characterized in that The receiving a first input of the first text content includes: receiving a copy input of the first text content; The step of processing, in response to the first input, the image element corresponding to the first text content in the first image based on the first text content and the text description information includes: In response to the copy input, when the first text content matches the text description information, an image element in the first image corresponding to the first text content is processed.

8. The method according to claim 2, characterized in that: After displaying the text description information, the method further includes: receiving a third input of a third text content in the displayed text description information; In response to the third input, an image element corresponding to the third text content is marked in the first image, or, in the displayed text description information, the third text content is replaced with an image element corresponding to the third text content.

9. An image processing device, characterized in that: The image processing device comprises: a receiving module and a processing module; The receiving module is used to receive a first input of a first text content, where the first text content is used to describe an image element corresponding to a user's processing requirement for a first image; The processing module is used to process the image elements corresponding to the first text content in the first image based on the first text content and text description information in response to the first input received by the receiving module when displaying the first image, and the text description information is used to describe the image content of the first image.

10. The device according to claim 9, characterized in that The image processing device further includes: a display module; The receiving module is further configured to receive a second input of the first image before receiving a first input of the first text content; The display module is used to display the text description information in response to the second input received by the receiving module.

11. The device according to claim 10, characterized in that The first input is an input for deleting the first text content in the text description information; The processing module is specifically configured to delete image elements corresponding to the first text content in the first image.

12. The device according to claim 10, characterized in that The first input is an input for adding the first text content to the text description information; The processing module is specifically configured to add an image element corresponding to the first text content to the first image.

13. The device according to claim 10, characterized in that The first input is an input for modifying the second text content in the text description information into the first text content; The processing module is specifically configured to adjust the image element in the first image corresponding to the second text content to the image element corresponding to the first text content.

14. The device according to claim 10, characterized in that The first input is used to adjust the display order of the first text content in the text description information; The processing module is specifically used to adjust the display position of the image element in the first image corresponding to the first text content in the first image.

15. The device according to claim 9, characterized in that The receiving module is specifically configured to receive a copy input of the first text content; The processing module is specifically configured to, in response to the copy input, process an image element in the first image corresponding to the first text content when the first text content matches the text description information.

16. The device according to claim 10, characterized in that The receiving module is further configured to receive a third input of a third text content in the displayed text description information after the display module displays the text description information; The display module is further used to mark the image element corresponding to the third text content in the first image in response to the third input received by the receiving module, or to replace the third text content with the image element corresponding to the third text content in the displayed text description information.

Images