Picture processing method, electronic device and storage medium

Abstract

Provided in the embodiments of the present application are a picture processing method, an electronic device and a storage medium, which are applied to the technical field of image processing, and can perform elimination processing on pictures more efficiently and flexibly. The picture processing method comprises: displaying an editing interface of a first picture, the first picture comprising a first image and a stackable element; in response to detecting an elimination operation on a first element in the first image, controlling to stop displaying the stackable element on the editing interface; in response to the elimination operation on the first element in the first image, eliminating the first element on the editing interface; and displaying on the editing interface the first image from which the first element has been eliminated, and restoring displaying the stackable element.

Description

Image processing method, electronic device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on November 3, 2023, with application number 202311461263.8 and invention name “A method for eliminating operations in mobile phone picture editing”, and the Chinese patent application filed with the State Intellectual Property Office on November 17, 2023, with application number 202311556165.2 and invention name “Picture processing method, electronic device and storage medium”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of image processing technology, and in particular to an image processing method, an electronic device, and a storage medium. Background Art

[0003] With the development of electronic devices, the image processing functions they support are becoming increasingly diverse. Users can now process images through the editing functions of image processing applications on their electronic devices. For example, electronic devices can beautify portraits in images, adjust the size of images, or remove objects from images, among other functions.

[0004] For example, when an electronic device performs a process of removing elements from an image, the process may be inefficient, inflexible, and inconvenient, and the image processing result may not meet user expectations, affecting the user's experience.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide an image processing method, electronic device, and storage medium. After the electronic device eliminates elements in a first image on an editing interface of the first image, the eliminated image is directly displayed on the editing interface. Not only is the entire elimination process simple to operate, but the elimination effect can also be intuitively seen, making the elimination process efficient and flexible. During the elimination operation, the superimposed elements are not displayed on the editing interface. After the elimination operation is completed, the superimposed elements are redisplayed on the editing interface, avoiding the problem of superimposed elements obscuring other elements in the image, resulting in an inability to accurately process other elements in the image.

[0007] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0008] In a first aspect, the present application provides an image processing method, comprising:

[0009] Display an editing interface for a first picture, the first picture including a first image and an overlay element; in response to detecting an elimination operation on a first element in the first image, control the overlay element to stop being displayed on the editing interface; in response to the elimination operation on the first element in the first image, eliminate the first element on the editing interface; display the first image after elimination of the first element on the editing interface and resume displaying the overlay element.

[0010] Among them, superimposed elements can be text, graffiti, watermarks, stickers, etc.

[0011] It can be understood that in the editing interface of the first picture, when the electronic device detects that the elimination operation of the first element has started, it can control the superimposed elements to stop displaying in the editing interface, so as to avoid the superimposed elements from blocking the elements in the first picture, resulting in the user being unable to accurately eliminate the elements in the first picture.

[0012] Optionally, displaying the first image after removing the first element and restoring the display of the overlay element on the editing interface includes:

[0013] In response to the elimination operation being completed, the first image after the first element is eliminated is displayed on the editing interface and the overlay elements are restored to display.

[0014] In one case, when the electronic device detects that the elimination operation is completed, the electronic device controls the overlay element to resume display on the editing interface.

[0015] Optionally, in response to detecting a deletion operation on the first element in the first image, controlling the overlay element to stop being displayed on the editing interface includes:

[0016] In response to detecting a removal operation on a first element in a first image, if it is determined that a distance between the first overlay type element and a first position is less than a first distance, controlling the first overlay type element to stop display, the first position being a position on the display screen where the removal operation is performed;

[0017] The above-mentioned image processing method may further include:

[0018] If it is determined that the distance between the second overlay element and the first position is greater than the first distance, the second overlay element is controlled to continue to be displayed.

[0019] Optionally, the image processing method further includes:

[0020] When the distance between the stopped first overlay element and the first position is greater than the first distance, the first overlay element is restored to display.

[0021] It can be understood that when the superimposed element is close to the position where the removal operation is performed on the display screen, in order to prevent the superimposed element from affecting other elements in the image, the superimposed element can be controlled to stop display. When the superimposed element is far from the position where the removal operation is performed on the display screen, the superimposed element does not affect the element to be removed, and the superimposed element does not need to be controlled to stop display. Optionally, the editing interface includes a first control, and the above-mentioned image processing method may further include:

[0022] In response to a triggering operation on the first control, determining to use a first elimination method to eliminate elements in the first image, the first elimination method being used to eliminate the elements selected by the elimination operation;

[0023] In response to a deletion operation on a first element in a first image, deleting the first element on an editing interface includes:

[0024] In response to a deletion operation on the first element in the first image, the first element selected by the deletion operation is deleted on the editing interface.

[0025] The first control may be a smart elimination control, and the electronic device may determine to eliminate the first element in a smart elimination manner in response to a user triggering the smart elimination control. Thus, when the electronic device detects the elimination operation on the first element, the electronic device may eliminate the entire first element.

[0026] Optionally, the editing interface may further include a second control, and the above-mentioned image processing method may further include:

[0027] In response to a triggering operation on the second control, determining to use a second elimination method to eliminate the element in the first image, the second elimination method being to eliminate a selected partial area of ​​the element targeted by the elimination operation;

[0028] In response to a deletion operation on the second element in the first image, a region selected by the deletion operation in the second element is deleted on the editing interface.

[0029] The second element may be a residual portion of the first element, or may be another element in the first image that is different from the first element. The second control may be a manual elimination control.

[0030] That is to say, after the electronic device eliminates the first element using the intelligent elimination method in response to the user's operation, there may be a problem that the first element is not completely eliminated. In this case, the electronic device can continue to eliminate the remaining part of the first element using the manual elimination method in response to the user's operation.

[0031] Alternatively, after the electronic device eliminates the first element using the intelligent elimination method, the electronic device may further eliminate a second element in the first image that is different from the first element using the manual elimination method.

[0032] Optionally, the editing interface includes a second control, and the above-mentioned image processing method further includes:

[0033] In response to a triggering operation on the second control, determining to eliminate the first element using a second elimination method;

[0034] In response to a deletion operation on a first element in a first image, deleting the first element on an editing interface includes:

[0035] In response to a deletion operation on a first element on a first picture, a region selected by the deletion operation in the first element is deleted on an editing interface.

[0036] Optionally, the elimination operation includes a click or slide operation, and in response to detecting the elimination operation on the first element in the first image, controlling the overlay element to stop being displayed on the editing interface includes:

[0037] In response to detecting a click or slide operation on the first element, the overlay element is controlled to stop being displayed on the editing interface.

[0038] It can be understood that the electronic device detects a click or slide operation on the first element, determines that the elimination operation starts, and controls the overlay element to stop displaying on the editing interface in response to the click or slide operation.

[0039] Optionally, in response to the deletion operation on the first element in the first image, after the first element is deleted on the editing interface, the image processing method may further include:

[0040] A target color is determined according to a color on the first image that is within a preset range where the first element is located; and the position where the first element is located is filled with the target color.

[0041] Wherein, after the first element is eliminated on the editing interface, the position of the first element on the first image is filled with the target color.

[0042] In this way, the electronic device uses the target color to fill the position where the first element is located after the first element is eliminated, so that the filled image is more natural and realistic.

[0043] Optionally, the editing interface includes a save button. After the first image and the superimposed elements are displayed on the editing interface after the first element is eliminated, the method further includes:

[0044] In response to detecting a triggering operation on the save button, a second picture is saved, where the second picture includes the first image after the first element is eliminated and the overlay element.

[0045] Therefore, after the electronic device eliminates the first element in the first image, the electronic device can save the second image in response to the user's triggering operation of the save button in the editing interface. The eliminated image can be saved without jumping to the page, which reduces the user's operation and improves the user experience.

[0046] In a second aspect, the present application provides an electronic device comprising: a display screen, the display screen being used to display an editing interface for an image; one or more processors; and a memory; wherein the memory stores one or more computer programs, and the one or more computer programs include instructions, which, when executed by the processor, enable the electronic device to perform an image processing method as described in any one of the above-mentioned first aspects.

[0047] In a third aspect, the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on an electronic device, the electronic device executes the image processing method as described in any one of the first aspects.

[0048] In a fourth aspect, the present application provides a computer program product, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the image processing method as described in any one of the first aspects.

[0049] It can be understood that the electronic device described in the second aspect, the computer storage medium described in the third aspect, and the computer program product described in the fourth aspect are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] FIG1 is an example of image processing in related art;

[0051] FIG2 is a second example of image processing in related art;

[0052] FIG3 is a third example of image processing in related art;

[0053] FIG4 is a fourth example of image processing in related art;

[0054] FIG5 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0055] FIG6 is a software structure diagram of an electronic device provided in an embodiment of the present application;

[0056] FIG7 is a schematic diagram of a logic layer provided in an embodiment of the present application;

[0057] FIG8 is a schematic diagram of a physical layer provided in an embodiment of the present application;

[0058] FIG9 is an example diagram of an image processing method provided in an embodiment of the present application;

[0059] FIG10 is a second example of the image processing method provided in an embodiment of the present application;

[0060] FIG11 is a third example of the image processing method provided in an embodiment of the present application;

[0061] FIG12 is a fourth example of the image processing method provided in an embodiment of the present application;

[0062] FIG13 is a fifth example of the image processing method provided in an embodiment of the present application;

[0063] FIG14 is a flowchart of an image processing method provided in an embodiment of the present application;

[0064] FIG15 is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] The following describes the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " represents "or." For example, A / B can represent A or B. "And / or" in this document is merely a description of the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, or B exists alone.

[0066] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0067] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0068] In related technologies, electronic devices are equipped with applications (APPs) for editing images, such as gallery apps and third-party image editing apps. Electronic devices can edit images in response to user operations, including but not limited to cropping, adjusting filters, beautifying, annotating, doodling, mosaicing, and removing.

[0069] For example, Figure 1 is an example of image processing in the related art. Taking a mobile phone as an electronic device and a gallery APP as an example, the image processing process is described as follows.

[0070] As shown in the interface in (a) of Figure 1 , the gallery APP of the mobile phone may include one or more pictures. After the mobile phone receives the operation of the user selecting a picture in the gallery APP, as shown in Figure 101, the display interface of the mobile phone switches from (a) in Figure 1 to (b) in Figure 1 .

[0071] As shown in Figure 1(b), the phone displays an interface including a picture 101, a share icon, a favorite icon, an edit icon 102, a delete icon, and more icons. When the phone detects a user triggering an operation on an icon, the phone can perform a corresponding operation on the picture 101. For example, when the phone detects a user triggering an operation on the edit icon 102, the phone's display interface switches from Figure 1(b) to Figure 1(c).

[0072] As shown in Figure 1(c), the interface displayed on the phone includes an image 101 and a primary toolbar 103. The primary toolbar 103 may include a cropping icon, a filter icon, an adjustment icon, and a more icon 104. When the phone detects a user triggering an icon, it can perform corresponding processing on the image 101. For example, when the phone detects a user triggering the more icon 104, the phone's display interface switches from Figure 1(c) to Figure 1(d).

[0073] As shown in FIG1(d), the interface displayed on the mobile phone includes a picture 101, a primary toolbar 103, and a secondary toolbar 105. As shown in FIG1(d), the secondary toolbar 105 may include a beauty icon, a markup icon 106, a graffiti icon, an erase icon, and a mosaic icon.

[0074] In some embodiments, the secondary toolbar 105 may also include a watermark icon, a photo frame icon, a blur icon, a retain color icon, a sticker icon, and the like.

[0075] It is understandable that, given the limited screen size of a mobile phone, the contents of the secondary toolbar 105 may not be fully displayed. When the mobile phone receives a user sliding operation on the secondary toolbar 105, the secondary toolbar 105 may display hidden contents. For example, in the mobile phone interface, the contents displayed on the secondary toolbar 105 include a beauty icon, an annotation icon, a graffiti icon, an erase icon, and a mosaic icon, while hidden contents include a watermark icon, a photo frame icon, a blur icon, a retain color icon, and a sticker icon. In response to the user sliding the secondary toolbar, the secondary toolbar of the mobile phone interface may display the hidden contents.

[0076] When the phone detects that the user has triggered the annotation icon 106, the phone's display interface displays a text input interface (not shown in FIG. 1 ), where the user can enter text, such as "The weather is great." After the phone detects that the user has confirmed the text input, the phone displays the interface shown in FIG. 1 (e). As shown in FIG. 1 (e), the interface displayed on the phone is still a secondary page, which includes the image 108, "The weather is great" 107, the primary toolbar 103, and the secondary toolbar 105.

[0077] After adding text to the picture, the mobile phone can also perform other processing on the picture. For example, the mobile phone can doodle on the picture 108 in response to the user's doodle operation. For example, as shown in (e) in Figure 1, after the mobile phone detects that the user has triggered the doodle icon 109, the user can doodle on the display screen of the mobile phone. After the mobile phone detects that the user has confirmed the doodle operation, the doodle added by the user can be retained in the picture displayed on the mobile phone. As shown in (f) in Figure 1, the secondary page displayed by the mobile phone includes the picture 111, the doodle 112, "The weather is great 107", the primary toolbar 103 and the secondary toolbar.

[0078] In some embodiments, when a user doodles on an image, a doodle function bar (not shown in FIG. 1 ) may be displayed on the display interface of the mobile phone. The doodle toolbar may include functions such as pen shape, color, thickness, and eraser. In response to a user's selection, the mobile phone may select a doodle shape using the pen shape function in the doodle toolbar; select a doodle color using the color function in the doodle function bar in response to a user's selection; determine the thickness of the line used for the doodle using the thickness function in the doodle function bar in response to a user's selection; and erase the doodle using the eraser function in the doodle function bar in response to a user's operation.

[0079] FIG1 shows a mobile phone adding text and graffiti to a picture, obtaining a picture 111 including graffiti and text. The mobile phone can remove objects in the picture 111.

[0080] Figure 2 illustrates a second example of image processing in the related art. As shown in Figure 2(a), the mobile phone displays an interface including an image 111, a doodle 112, "Nice weather 107," a primary toolbar 103, and a secondary toolbar 105. For example, when the mobile phone detects a user triggering the dismiss icon 201, the mobile phone's display switches from Figure 2(a) to Figure 2(b).

[0081] As shown in the interface in (b) of Figure 2, the interface displayed on the mobile phone includes picture 111 and elimination toolbar 202 (i.e., a three-level toolbar). Among them, the elimination toolbar includes smart elimination controls, manual elimination controls, elimination cancellation icons, and elimination confirmation icons. The user can choose to perform smart elimination operations or manual elimination operations on picture 111 in the elimination toolbar. For example, after the mobile phone detects that the user has selected the manual elimination control, after the mobile phone detects that the user has eliminated the boat in picture 111, when the mobile phone detects that the user has triggered the elimination confirmation icon 203, the display interface of the mobile phone is switched from (b) in Figure 2 to (c) in Figure 2.

[0082] As shown in FIG2( b ), when the user deletes the picture 111 , the mobile phone display interface does not display the graffiti 112 or the phrase “Nice weather” 107 . That is, when the mobile phone performs the deletion operation, the picture 111 does not display any superimposed elements.

[0083] As shown in Figure 2(c), the phone displays an interface that includes image 204 (i.e., the image after the deletion operation), the primary toolbar 103, the secondary toolbar 105, and a save button 205. When the phone detects that the user triggers the save button 205, it saves the image 204, and the phone's display switches from Figure 2(c) to Figure 2(d). As shown in Figure 2(d), the phone's display includes image 204, a share icon, a favorite icon, an edit icon, a delete icon, and more icons.

[0084] As shown in Figure 2, while the phone is removing an image, the phone's display interface does not display the overlay elements. When the phone saves the removed image 204, the overlay elements reappear on the phone's display interface. Furthermore, in the third-level toolbar interface for the removal function, after removing an image, the user must first click the "Confirm" icon to return to the second-level toolbar interface and click the "Save" button in the second-level toolbar interface to save and export the processed image. This shows that after the phone removes certain objects from an image, multiple operations are required to save the removed image.

[0085] Similarly, after a user uses a third-party application on a mobile phone to remove certain elements from a picture, multiple operations are required before the mobile phone can successfully save the removed picture.

[0086] Figure 3 is a third example of image processing in the related art. The interface shown in Figure 3 (a) is the homepage of a third-party application that processes images. When the mobile phone detects that the user triggers the image beautification icon 301, the mobile phone's display interface switches from Figure 3 (a) to Figure 3 (b).

[0087] As shown in the interface in (b) of FIG3 , the gallery of the mobile phone may include one or more pictures. After the mobile phone receives the operation of the user selecting a picture in the gallery, as shown in FIG302 , the display interface of the mobile phone switches from (b) in FIG3 to (c) in FIG3 .

[0088] As shown in the interface in (c) of Figure 3, the display interface of the mobile phone includes an image 302 and a primary toolbar. Among them, the primary toolbar may include text icons, a graffiti pen icon, a tool icon 304, and a decoration icon. When the mobile phone detects that the user has triggered the tool icon 304, the display interface of the mobile phone may display a secondary toolbar 305. Among them, the secondary toolbar 305 may include: an eraser pen icon 306, a magic cutout icon, a blur icon, a background icon, etc. When the mobile phone detects that the user has triggered the eraser pen icon 306, the display interface of the mobile phone switches from (c) in Figure 3 to (d) in Figure 3.

[0089] It is understandable that, given the limited screen size of a mobile phone, the contents of the secondary toolbar 305 may not be fully displayed. When the mobile phone receives a sliding operation on the secondary toolbar 305 by the user, the secondary toolbar 305 may display hidden contents. For example, in the mobile phone interface, the contents displayed in the secondary toolbar 305 include beauty icons, annotation icons, mosaic icons, and the like.

[0090] As shown in Figure 3(d), the phone's display interface includes an image 302 and an eraser tool bar 307. In the eraser tool bar 307, the user can select the size of the pen tip used to erase the image. After the phone detects the user's erase operation on the boat in image 302 and triggers the erase confirmation icon 308, the phone's display interface switches from Figure 3(d) to Figure 3(e).

[0091] As shown in FIG3(d), when the user deletes the image 302, the display interface of the mobile phone does not display the word "tourist" 303. That is, when the mobile phone performs the deletion operation, no superimposed elements are displayed in the image 302.

[0092] As shown in Figure 3(e), the phone displays an interface including image 309, a primary toolbar, a secondary toolbar 305, and a save button 310. Image 309 is the image after image 302 is deleted. When the phone detects that the user has triggered save button 310, it saves image 309, and the phone's display switches from Figure 3(e) to Figure 3(f).

[0093] In one embodiment, when a gallery APP or a third-party application in a mobile phone performs a removal operation, there may be a problem in which the superimposed elements in the image are not hidden, resulting in the superimposed elements being removed.

[0094] For example, Figure 4 illustrates an example of image processing in the related art. As shown in Figure 4(a), the mobile phone's display interface includes an image 401, a primary toolbar, and a secondary toolbar 403. Image 401 includes an overlay element 402. After the mobile phone detects a user triggering the erase pen icon 404 in secondary toolbar 403, the display interface switches from Figure 4(a) to Figure 4(b).

[0095] As shown in Figure 4(b), the phone's display interface includes an image 401 and an eraser toolbar 405. Image 401 includes text within an overlay element. When the phone detects a user's erase operation on image 401, the overlay element within image 401 remains hidden; that is, the overlay element within the image remains unchanged during the erase operation. When the phone detects the user triggering the erase confirmation icon 406, the phone's display interface switches from Figure 4(b) to Figure 4(c).

[0096] As shown in FIG4(c), the mobile phone's display interface includes a picture 407 (i.e., the picture after the elimination process) and a save button 408. When the mobile phone detects that the user triggers the save button 408, the mobile phone executes the operation of saving the picture 407.

[0097] As shown in Figure 4, while the phone is removing objects from an image, the phone's display shows overlay elements. To save the removed image, the phone returns to the secondary toolbar and clicks the "Save" button to save and export the processed image. This indicates that when the phone removes certain objects from an image, overlay elements obscure the image. Saving the removed image requires multiple operations.

[0098] In some embodiments, when the gallery app or third-party application in the mobile phone executes the elimination function, the superimposed elements in the picture may not be hidden, and the superimposed elements may cover the eliminated objects, resulting in the user being unable to accurately eliminate certain elements in the picture.

[0099] As shown in Figure 4(a), suppose a user wants to erase the area covered by "tourist" 402 in image 401. When the phone detects the user triggering the eraser pen 404, "tourist" 402 is still displayed on the phone interface. As a result, because the superimposed element "tourist" 402 covers the area of ​​the object to be erased, the user cannot accurately erase part of the area in image 401.

[0100] To this end, an embodiment of the present application provides an image processing method, which is applied to an electronic device. During the process of the electronic device displaying the editing interface of the first image, in response to detecting the elimination operation of the first element in the first image, the method controls the superimposed elements to stop being displayed on the editing interface, and in response to the elimination operation of the first element in the first image, eliminates the first element. In response to the end of the elimination operation, the first image and the superimposed elements after the elimination of the first element are displayed on the editing interface. Thus, after the electronic device eliminates the elements in the first image in the editing interface of the first image, the image after the elimination process is directly displayed on the editing interface. Not only is the entire elimination process simple to operate, but the elimination effect can also be intuitively seen. In addition, when it is detected that the elimination operation of the first element in the first image starts, the superimposed elements are controlled to stop being displayed, thereby avoiding the problem that the superimposed elements block the elements in the first image, resulting in the inability to accurately eliminate the elements in the first image. For example, the image processing method provided in the embodiments of the present application can be applied to mobile phones, tablet computers, personal computers (PCs), personal digital assistants (PDAs), smart watches, netbooks, wearable electronic devices, augmented reality (AR) devices, virtual reality (VR) devices, vehicle-mounted devices, smart cars, smart speakers, and other electronic devices with display screens, and the embodiments of the present application do not impose any restrictions on this.

[0101] As shown in FIG5 , FIG5 is a structural diagram of an electronic device provided in an embodiment of the present application.

[0102] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0103] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0104] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0105] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0106] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly retrieve it from the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0107] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0108] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0109] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0110] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0111] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0112] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0113] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with a network and other devices via wireless communication technology. Wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. GNSS may include the global positioning system (GPS), the global navigation satellite system (GLONASS), the Beidou navigation satellite system (BDS), the quasi-zenith satellite system (QZSS) and / or the satellite based augmentation system (SBAS).

[0114] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0115] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0116] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0117] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0118] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0119] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0120] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0121] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or removed from the electronic device 100 by inserting it into or removing it from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0122] The software system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present invention, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device.

[0123] FIG6 is a software structure diagram of an electronic device provided in an embodiment of the present application.

[0124] It is understood that a layered architecture divides software into several layers, each with distinct roles and divisions of labor. Layers communicate with each other via software interfaces. In some embodiments, the Android system may include an application layer (abbreviated as the application layer), an application framework layer (abbreviated as the framework layer), a system library, and a kernel layer.

[0125] The above application layer may include a series of application packages.

[0126] As shown in Figure 6, the application package may include system applications. System applications refer to applications that are installed in the electronic device before leaving the factory. For example, system applications may include programs such as camera, gallery, calendar, music, short message, memo, and weather. The gallery includes at least one image.

[0127] Application packages can also include third-party applications, which are applications that users install by downloading the installation package from an app store (or app market). Examples include map applications, food delivery applications, reading applications (such as e-books), social applications, and travel applications.

[0128] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0129] As shown in FIG6 , the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, and the like.

[0130] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0131] Content providers are used to store and retrieve data and make it accessible to applications. This data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0132] The view system includes visual controls, such as controls for displaying text and images. The phone manager is used to provide communication functions for electronic devices, such as managing call status (including answering and hanging up).

[0133] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0134] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically, without requiring user interaction. For example, the Notification Manager can be used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include displaying text messages in the status bar, emitting alert sounds, vibrating the phone, or flashing indicator lights.

[0135] Android Runtime includes core libraries and a virtual machine. Android Runtime is responsible for scheduling and management of the Android system.

[0136] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0137] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0138] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0139] The surface manager is used to manage the display subsystem and provide the fusion of two-dimensional and three-dimensional layers for multiple applications.

[0140] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0141] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0142] A 2D graphics engine is a drawing engine for 2D drawings.

[0143] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver and sensor driver.

[0144] Taking the picture processing method of the embodiment of the present application as an example, the pictures in the picture gallery application can be divided into logical layers as described in Figure 7 and physical layers as described in Figure 8 according to different processing functions.

[0145] For example, FIG7 is a schematic diagram of a logic layer provided in an embodiment of the present application.

[0146] As shown in FIG7 , the logical layers from top to bottom may include a photo frame, color functions, overlay materials, effect materials, and a content layer.

[0147] The photo frame is displayed independently on the periphery of the screen, making it unaffected by other content or functions. For example, if the phone receives the user's operation of adding a photo frame to a picture and then the user's operation of adding a filter to the picture, the color of the photo frame will not be affected by the filter and will not change.

[0148] Color functions may include preserving colors, and the color functions act on all colored content in the image.

[0149] The overlay material may include at least one of text, watermark, sticker or graffiti, and the overlay material is applied to a part of the image.

[0150] Effect materials can include at least one of mosaic, blur, filter, beautify, adjustment, or removal. Adjustment functions apply to the image or video itself. For example, users can adjust the brightness, saturation, and other aspects of the image. In addition to adjustment functions, the upper-layer material in other effect materials can affect the lower-layer material. For example, if a user adds a mosaic to a filtered image, the mosaic will be applied to the filtered image.

[0151] The content layer can include images and canvases, where images refer to the images themselves used for image processing.

[0152] In the logical layer shown in Figure 7, the materials or functions of the upper layer will affect the materials or functions of the lower layer, and the materials or functions of the lower layer will not affect the materials or functions of the upper layer. This ensures that no matter how the user's operation sequence changes, consistent image processing results can be obtained. Taking filters and adjustments as an example, the mobile phone receives the following operations in sequence: the user's operation of adding a filter to the image, and the user's operation of adjusting the image using the adjustment function. Because the adjustment function targets the image layer in the logical layer, rather than the image after the filter is added, the above image processing results are consistent with the effect of the user adjusting the image using the adjustment function and then adding a filter.

[0153] As shown in Figure 7, based on actual user needs, the cropping function is placed below the text, watermark, and sticker layers, and above the graffiti layer. When the phone receives a user's cropping command and crops the image, the cropping function applies to the graffiti layer, effects, and content layers below the cropped image. It does not apply to the text, watermark, and sticker layers above the cropped image, color functions, or photo frames.

[0154] That is to say, on the one hand, considering that the graffiti and mosaics added by the user to the picture may be strongly related to the content of the picture, for example, when the user adds graffiti to the picture or circles a word in the picture, the user does not want the relative position of the graffiti and the picture to change during the cropping process, or when the user adds a mosaic to the picture to cover the face of the person in the picture, the user does not want the relative position of the mosaic and the picture to change during the cropping process, so the graffiti and mosaic are placed in the lower layer of the cropping, so that the graffiti and mosaic can always be displayed on the picture during the cropping process, and the relative position with the picture remains unchanged. The user can see the graffiti and mosaic intuitively and crop the mosaic and graffiti according to the user's actual needs.

[0155] On the other hand, considering that users do not want to partially or completely crop the text, watermarks and / or stickers added to the picture during the cropping process, the text, watermarks and stickers are placed on the upper layer of the cropping, so that the text, watermarks and stickers can be temporarily hidden during the cropping process. After the cropping is completed, the text, watermarks and stickers can be redisplayed on the cropped picture, and the cropping function does not affect the text, watermarks and stickers.

[0156] It is understood that there is no coupling relationship between the layers, and the content of each layer can be adjusted according to actual needs. For example, if a user adds text to an image and uses text to label the names of various items in the image, the user's requirement is that the relative position of the text and the image remain unchanged during the cropping process. To meet this user requirement, the text can be moved to the lower layer after cropping. This embodiment of the present application does not specifically limit this.

[0157] For example, Figure 8 is a schematic diagram of the physical layers provided in an embodiment of the present application, wherein the physical layers are the actual layer order corresponding to each image processing function during the image processing process.

[0158] Corresponding to the image processing process of Figures 1 to 4, the order of physical layers from top to bottom can include: photo frame physical layer, color retention physical layer, overlay material physical layer, mosaic physical layer, blur physical layer, filter physical layer, beauty physical layer, adjustment physical layer, picture physical layer and canvas physical layer.

[0159] The overlay material can include at least one of stickers, watermarks, text, and graffiti. Corresponding to the image editing process in Figures 1-4, the overlay material physical layers, from top to bottom, can include a graffiti rectangle layer, a sticker layer, a watermark layer, a text layer, and a graffiti 123 layer.

[0160] The order of the above physical layers can be relatively fixed. One or more real layers can be added to each physical layer.

[0161] For example, taking the mosaic physical layer as an example, the mobile phone adds the first mosaic to the picture in response to the user's operation, and the first real mosaic layer will be added to the mosaic physical layer. Afterwards, the mobile phone adds text to the picture in response to the user's operation, and a real text layer can be added to the text physical layer. After adding the text, the user returns to the mosaic function. Further, the mobile phone adds the second mosaic to the picture in response to the user's operation, and the second real mosaic layer will be added to the mosaic physical layer. That is to say, the first mosaic and the second mosaic are in the mosaic physical layer.

[0162] Similarly, for text, graffiti, watermarks, and stickers added to a picture by an electronic device, a new layer can be created for each element in the physical layer of the overlay class elements in sequence according to the order in which the user added them. For example, if the mobile phone responds to the user's operation and adds the first graffiti to the picture, then the first real graffiti layer can be added to the physical layer of the overlay class material. After that, if the mobile phone responds to the user's operation and adds text to the picture, then a real text layer can be added to the physical layer of the overlay class material. After adding the text, the user returns to the graffiti function. Further, if the mobile phone responds to the user's operation and adds the second graffiti to the picture, then the second real graffiti layer can be added to the physical layer of the overlay class material. In other words, the first graffiti layer, the text layer, and the second graffiti layer can be in the physical layer of the overlay class material in order from bottom to top.

[0163] It's understandable that for the first graffiti, regardless of how much graffiti content it contains, a real graffiti layer can be created for it. For example, if the first graffiti content is "123," although "1," "2," and "3" correspond to three different strokes, in the graffiti physical layer, "123" corresponds to one real layer, not three. The second graffiti is processed similarly and will not be explained in detail.

[0164] For a picture including multiple layers, for example, the picture 111 shown in (f) in FIG1 includes a canvas, a picture, graffiti 112 and text 107 , the mobile phone can select the graffiti 112 in the graffiti layer in response to the user clicking the graffiti 112 .

[0165] Based on the logical and physical layers, the embodiments of the present application can distinguish and process elements in each logical or physical layer, so that when certain elements in an image are removed, some elements can be displayed while others are hidden. After the removal action is completed, the elements that were not displayed can be re-displayed in the removed image, thereby solving the problem of superimposed elements covering the elements to be removed in the image, resulting in the user being unable to accurately remove certain elements in the image. The specific solution will be described in detail in subsequent embodiments and will not be elaborated here.

[0166] In the following, the electronic device 100 is still a mobile phone, and the application for editing pictures is a gallery APP as an example, and the picture processing method of the present application is exemplarily described with reference to the accompanying drawings.

[0167] For example, Figure 9 is an example diagram of the image processing method provided in an embodiment of the present application. The gallery APP of a mobile phone may include one or more pictures. After the mobile phone receives the operation of the user selecting a picture in the gallery APP and performs an editing operation on the picture, the interface of the mobile phone is displayed as shown in Figure 9 (a). The interface displayed by the mobile phone includes picture 901, a share icon, a favorite icon, an edit icon 902, a delete icon and more icons. Among them, picture 901 includes a text layer and a sticker layer, the text layer includes "weather" 903, and the sticker layer includes a sticker smiley face 900. When the mobile phone detects that the user has triggered an icon, the mobile phone can perform a corresponding operation on picture 901. For example, when the mobile phone detects that the user has triggered the edit icon 102, the display interface of the mobile phone is switched from (a) in Figure 9 to (b) in Figure 9.

[0168] As shown in FIG9( b ), the interface displayed by the mobile phone includes a picture 901 and a first-level toolbar 904. The first-level toolbar 904 may include a graffiti icon, a text icon, an erase icon 905, a mosaic icon, and a watermark icon.

[0169] In some embodiments, the primary toolbar 904 may also include a beauty icon, a filter icon, a photo frame icon, a blur icon, a retain color icon, a sticker icon, and the like.

[0170] It is understandable that when the screen size of the mobile phone is limited, the content of the first-level toolbar 904 may not be fully displayed. When the mobile phone receives a sliding or dragging operation of the user on the first-level toolbar 904, the first-level toolbar 904 can display hidden content. For example, in the mobile phone interface, the content displayed by the first-level toolbar 904 includes graffiti icons, text icons, elimination icons, mosaic icons and watermark icons, and the hidden content includes beauty icons, filter icons, photo frame icons, blur icons, retain color icons, and sticker icons. In response to the user's sliding or dragging operation on the first-level toolbar 904, the first-level toolbar of the mobile phone interface can display hidden content. In this way, the user can select the function they want to use in the first-level toolbar without having to expand the second-level toolbar, which makes it easier for the user to find various functions, reduces the number of interface jumps, and makes the user's operation more convenient.

[0171] When the mobile phone detects that the user triggers the removal icon 905 in the first-level toolbar 904, the display interface of the mobile phone switches from (b) in FIG. 9 to (c) in FIG. 9.

[0172] As shown in the interface in Figure 9 (c), the interface displayed on the mobile phone includes an image 901, a primary toolbar, a secondary toolbar, and prompt information. Among them, the prompt information is used to prompt how to eliminate elements in the image. As shown in Figure 9 (c), the prompt information is "Click or slide on the target object to eliminate it." The secondary toolbar includes a smart elimination control 906 and a manual elimination control 907. When the mobile phone detects that the user triggers the smart elimination control 906, the display interface of the mobile phone switches from Figure 9 (c) to Figure 9 (d).

[0173] Similarly, when the mobile phone detects that the user triggers the manual elimination control 907, the display interface of the mobile phone switches from (a) in Figure 11 to (b) in Figure 11. The detailed process of manually eliminating elements in the picture will be described later and will not be repeated here.

[0174] As shown in the interface of Figure 9(d), when the phone detects a user sliding on the boat in image 901, it automatically identifies the boat. After the phone recognizes the outline of the boat, the phone's display switches from Figure 9(d) to Figure 9(e).

[0175] As shown in Figure 9(e), the phone's display interface includes an image 901, a primary toolbar, and a secondary toolbar. The boat in image 901 is being recognized. Image 901 does not include the text "Weather" 903 or the smiley face 900. When the phone detects that the user's sliding operation has ended, the phone's display interface switches from Figure 9(e) to Figure 9(f).

[0176] In this embodiment of the present application, after removing an element from an image, the phone automatically identifies the colors surrounding the removed element and uses the identified colors to fill in the area where the removed element was located. For example, as shown in Figure 9(f), after removing a boat from an image, the phone uses the colors surrounding the boat in image 909 to fill in the area where the boat was located. This makes the removed image appear more realistic and natural.

[0177] As shown in Figure 9(f), the mobile phone's display interface includes an image 909, a primary toolbar, a secondary toolbar, and a save button 908. Image 909 is a post-deletion image and includes the text "Weather" 903 and a smiley face 900. When the mobile phone detects that the user has triggered the save button 908, it saves the post-deletion image 909.

[0178] It should be noted that after the mobile phone saves the picture 909 , the mobile phone can also edit the saved picture 909 in response to the user's operation.

[0179] As can be seen from the image processing process of Figure 9 above, the mobile phone can intelligently eliminate elements in the image in response to the user's trigger operation. When the mobile phone detects the user's sliding operation on an element in the image, the mobile phone can identify the element. When the mobile phone detects that the user's sliding operation is completed, the display interface of the mobile phone displays the image after the intelligent elimination process. In this way, the user can intuitively see the image after the elimination process. If the user is satisfied with the image after the elimination process, the user does not need to trigger the mobile phone to jump to other interfaces, and can trigger the save button to save the processed image. Compared with Figures 2 to 4, the user needs to perform a return operation and jump to the save interface to save the processed image. In the embodiment of the present application, the mobile phone can directly save the image after the elimination process in the editing interface, reducing the user's trigger operation and making it more convenient for the user to save the image after the elimination process.

[0180] The above Figure 9 is an exemplary introduction to the process of eliminating elements in the picture after the mobile phone detects that the user has performed a sliding operation on the elements in the picture. The following is an exemplary introduction to the process of eliminating elements in the picture after the mobile phone detects that the user has performed a click operation on the elements in the picture in combination with Figure 10.

[0181] Figure 10 is an example of the second diagram of the image processing method provided by an embodiment of the present application. As shown in the interface of Figure 10 (a), the display interface of the mobile phone includes an image 901, a primary toolbar, and a secondary toolbar. Among them, the image 901 includes the text "Weather" 903, a smiley face 900, a seaside background, and a person 1001. When the mobile phone detects that the user triggers the smart elimination control 906, the display interface of the mobile phone switches from Figure 10 (a) to Figure 10 (b).

[0182] As shown in the interface of Figure 10(b), when the phone detects that the user has clicked on the character 1001, the phone's display interface switches from Figure 10(b) to Figure 10(c). As shown in the interface of Figure 10(c), the phone automatically recognizes the outline of the character 1001. When the phone detects that the user has stopped clicking on the character 1001, the phone's display interface switches from Figure 10(c) to Figure 10(d). The click operation can be a single click, a double click, or a multi-click operation.

[0183] As shown in FIG10(d), the mobile phone's display interface includes a picture 1002 and a save button 908. Picture 1002 is a post-deletion picture and includes the text "Weather" 903 and a smiling face 900. When the mobile phone detects that the user has triggered the save button 908, the mobile phone saves picture 1002.

[0184] It should be noted that after the mobile phone eliminates the person 1001, the mobile phone can automatically identify the color of the area surrounding the person 1001 in the picture 1002, and use the color of the area surrounding the person 1001 to fill the position where the person 1001 is located, making the picture after the elimination process more realistic and natural.

[0185] As shown in the image processing process in Figure 10 above, when the phone detects a user clicking on an element in the image, it can automatically identify that element. When the phone detects that the user's click operation has ended, the phone's display interface displays the image after the intelligent deletion process. This allows the user to intuitively see the deleted image. If the user is satisfied with the deleted image, the user can trigger the Save button to save the processed image without triggering the phone to jump to other interfaces. This reduces the user's triggering operations and makes it more convenient for users to save deleted images.

[0186] The following describes the process of manually removing elements from a picture in response to a user's manual removal operation by a mobile phone in conjunction with FIG. 11 .

[0187] Figure 11 is an example of the third diagram of the image processing method provided in an embodiment of the present application. In the editing interface shown in Figure 11 (a), when the mobile phone detects that the user triggers the manual deletion control 907, the display interface of the mobile phone switches from Figure 11 (a) to Figure 11 (b).

[0188] As shown in Figure 11(b), the phone displays an image 901, a primary toolbar, and a secondary toolbar. Image 901 includes a smiley face sticker 900 and the text "Weather" 903. The secondary toolbar includes a smart erase control 906, a manual erase control 907, and a manual adjustment icon 1101. Manual adjustment icon 1101 is used to adjust the thickness of the eraser pen used when manually erasing an image. When the phone detects the user manually erasing the boat in image 901, the phone's display shows the erasure process, as shown in Figure 11(c).

[0189] As shown in Figure 11(c), the phone's display interface includes an image 1102, a primary toolbar, and a secondary toolbar. Image 1102 does not include the text "Weather" 903 or the smiley face sticker 900. When the phone detects that the user has stopped sliding the image, the phone's display interface switches from Figure 11(c) to Figure 11(d).

[0190] As shown in Figure 11(d), the phone's display interface includes an image 1103, a primary toolbar, a secondary toolbar, and a save button 908. Image 1103 is a manually deleted image. It includes the text "Weather" 903 and a smiley face sticker 900. When the phone detects the user triggering the save button 908, it saves image 1103.

[0191] It should be noted that, after the mobile phone erases part of the area of ​​the boat in the picture 1103, the mobile phone can automatically identify the color around the erased boat and use the identified color to fill the area where the boat is erased.

[0192] As shown in the image processing process in Figure 11, when the phone detects a user sliding an element in the image, it removes the element in the area where the user slid, and then displays the image after the removal process on the phone's display interface, allowing the user to see the removal effect in real time. When saving the removed image, the user simply triggers the Save button to save the processed image, reducing the user's triggering operations and making it more convenient for the user to save the removed image.

[0193] In some embodiments, after the mobile phone performs intelligent removal processing on certain elements in the image, the mobile phone can also continue to manually remove the elements in the image in response to the user's sliding operation. In this way, by combining intelligent removal and manual removal, some elements in the image can be removed more accurately.

[0194] For example, Figure 12 is an example Figure 4 of the image processing method provided in an embodiment of the present application. As shown in the interface in Figure 12 (a), the interface displayed on the mobile phone includes a picture 901, a primary toolbar, a secondary toolbar, and prompt information. Among them, the picture 901 includes a smiley face sticker 900, the text "weather" 903, and a boat 1201. The secondary toolbar includes a smart elimination control 906 and a manual elimination control 907. When the mobile phone detects that the user triggers the smart elimination control 906, the display interface of the mobile phone switches from Figure 12 (a) to Figure 12 (b).

[0195] As shown in the interface in (b) of Figure 12, when the mobile phone detects that the user triggers the boat 1201 in picture 901, the mobile phone automatically identifies the boat 1201. When the mobile phone detects that the user's triggering operation on the boat 1201 is completed, the display interface of the mobile phone switches from (b) in Figure 12 to (c) in Figure 12.

[0196] As shown in Figure 12(c), the phone's display still includes a portion of boat 1201 within image 901. In this case, the user can manually delete the remaining portion of boat 1201. When the phone detects that the user has triggered manual delete control 907, the phone's display switches from Figure 12(c) to Figure 12(d).

[0197] As shown in the interface of Figure 12 (d), when the phone detects a user's sliding operation, it removes the remaining portion of boat 1201, and the phone's display interface displays the removal process in real time. When the phone detects that the user's sliding operation ends, the phone's display interface switches from Figure 12 (d) to Figure 12 (e). The image 1202 shown in Figure 12 (e) is the image that has been removed through both intelligent removal and manual removal. When the phone detects that the user triggers the save button 908, the phone saves image 1202.

[0198] In some embodiments, when a mobile phone detects a user sliding or clicking on an image, the overlay elements in the image displayed on the phone's display interface are hidden. That is, while the phone is removing elements from the image, the overlay elements are not displayed. This prevents overlay elements from obscuring the original elements in the image, preventing the user from accurately removing them, and improves the accuracy of image processing.

[0199] It should be explained that according to the logical layer shown in Figure 7, in the logical layer, the elimination function is in the lower layer of the overlay class materials and effect class materials, so the elimination function can only eliminate the original elements in the picture, and cannot eliminate the text, watermarks, stickers, graffiti, etc. added to the picture. During the elimination process performed by the mobile phone, the overlay class elements in the picture are temporarily not displayed. After the mobile phone determines that the elimination operation is completed, the overlay class elements are redisplayed on the picture. Therefore, during the elimination process, the mobile phone controls the overlay class elements to be temporarily not displayed in the picture, avoiding the obstruction of the picture by the overlay class elements, resulting in the problem that the user cannot accurately eliminate the elements in the picture.

[0200] For example, in the interfaces shown in (a), (b), and (c) of Figure 9 , image 901 includes the text "Weather" 903 and a smiley face sticker 900. When the phone detects that the user has swiped the boat in image 901, triggering the removal process, the phone's display interface no longer displays the text "Weather" 903 and the smiley face sticker 900, as shown in (d) and (e) of Figure 9 . When the phone detects that the user has finished swiping, the phone's display interface displays the text "Weather" 903 and the smiley face sticker 900 again, as shown in (f) of Figure 9 .

[0201] As another example, in the interface shown in (a) of Figure 10, images 901 all include the text "Weather" 903 and a smiley face sticker 900. When the phone detects that the user has clicked on the person 1001 in image 901, triggering the removal process, the phone's display interface no longer displays the text "Weather" 903 and the smiley face sticker 900, as shown in (b) and (c) of Figure 10. When the phone detects that the user's click operation has ended, the phone's display interface displays the text "Weather" 903 and the smiley face sticker 900 again, as shown in (d) of Figure 10.

[0202] As another example, in the interfaces shown in (a) and (b) of Figure 11, image 901 includes the text "Weather" 903 and a smiley face sticker 900. When the phone detects that the user has swiped the boat in image 901, triggering the removal process, the phone's display interface no longer displays the text "Weather" 903 and the smiley face sticker 900, as shown in the interface shown in (c) of Figure 11. When the phone detects that the user has finished swiping, the phone's display interface displays the text "Weather" 903 and the smiley face sticker 900 again, as shown in the interface shown in (d) of Figure 11.

[0203] In Figures 9 to 11, the superimposed elements include text and stickers as examples for introduction. The superimposed elements can also include graffiti, watermarks, etc., which will not be introduced one by one here.

[0204] In other embodiments, when a mobile phone detects a user sliding an image during a removal operation, if the mobile phone detects that the distance between an overlay element in the image and the user's sliding touch point is less than a first distance, the mobile phone controls the overlay element to be hidden. The first distance is a preset value, for example, 10dp, 12dp, etc.

[0205] For example, Figure 13 is an example Figure 5 of the image processing method provided in an embodiment of the present application. As shown in the interface in (a) of Figure 13, when the mobile phone detects that the user triggers the manual elimination control 907, the display interface of the mobile phone switches from (a) in Figure 13 to (b) in Figure 13.

[0206] As shown in the interface of Figure 13(b), when the phone detects a user sliding on the boat in image 901, the phone detects that the distance between the superimposed text element "Weather" 903 and the sliding contact is less than a first distance, and the phone's display interface does not display the superimposed text element "Weather" 903. During the user's sliding operation to remove elements from the image, the position of the sliding contact is as shown in the interface of Figure 13(c). The phone detects that the distance between the superimposed text element "Weather" 903 and the sliding contact is less than the first distance, and the phone's display interface does not display the superimposed text element "Weather" 903. At this point, the phone detects that the distance between the sliding contact and the superimposed text element "Weather" 903 is greater than the first distance, and the phone's display interface displays the superimposed text element "Weather" 903. When the phone detects that the user's sliding operation has ended, that is, the user lifts their hand off the display screen, the phone's display interface displays the removed image 1310, which displays the superimposed text element "Weather" 903 and the smiling face sticker 900, as shown in Figure 13(d). When the mobile phone detects that the user triggers the save button 908 , the mobile phone saves the picture 1310 .

[0207] In an embodiment of the present application, when a video is stored in the gallery APP of the mobile phone, the mobile phone can also eliminate the elements in each frame of the video. The specific elimination process can be referred to the above-mentioned process of eliminating the elements in the picture, which will not be repeated here.

[0208] The specific implementation of the image processing method provided in the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0209] 14 is a flowchart of an image processing method provided in an embodiment of the present application, which can be applied to the electronic devices shown in FIG5 and FIG6. As shown in FIG14, the image processing method can include S1400 to S1430.

[0210] S1400: Displaying an editing interface for a first picture.

[0211] The first image includes a first image and an overlay element. For example, the first image may be image 901 shown in FIG. 9(a). The overlay element may be a watermark, sticker, text, or graffiti. For example, the overlay element may be the text "Weather 903" and the sticker smiley face 900 shown in FIG. 9(a).

[0212] The editing interface of the first image refers to an interface for editing elements in the first image. For example, the editing interface may be the interface shown in (b)-(d) of Figure 9, or the interface shown in (a)-(d) of Figure 10, or the interface shown in (a)-(d) of Figure 11.

[0213] S1410: In response to detecting a deletion operation on a first element in a first image, controlling the overlay element to stop being displayed on the editing interface.

[0214] In an embodiment of the present application, when the electronic device detects the start of an erasure operation on the first element in the first image, in response to the start of the erasure operation, the electronic device controls the overlay element to stop displaying.

[0215] As shown in Figure 9, when the phone detects the user's removal of the boat in image 901, it controls the display of the text "Weather 903" and the smiley face sticker 900. Overlay elements are not displayed in the editing interface shown in Figures 9 (d) and (e). After the removal operation is completed, the overlay elements reappear in the editing interface of image 901, as shown in Figure 9 (f).

[0216] In an embodiment of the present application, in response to a deletion operation on a first element in a first image, the electronic device may determine whether to stop displaying any overlay element on the first image based on the distance between the overlay element and a first position. The first position is the position on the display screen where the deletion operation occurs, for example, the position on the display screen where a user swipes.

[0217] In one case, if the electronic device determines that the distance between the first overlay element and the first position is less than a first distance (the first distance is a preset distance), the electronic device controls the first overlay element to stop displaying. For example, the first distance can be 10dp, 12dp, etc. When the electronic device determines that the distance between the first overlay element that has stopped displaying and the first position is greater than the first distance, the electronic device controls the first overlay element to resume displaying.

[0218] In another case, if the electronic device determines that the distance between the second overlay element and the first position is greater than the first distance, the electronic device controls the second overlay element to continue to be displayed.

[0219] For example, at the position in the interface shown in FIG13(b), the phone detects that the distance between the superimposed element sticker smiley face 900 and the first position is greater than the first distance, and the phone's display interface continues to display the superimposed element sticker smiley face 900. At this time, the phone detects that the distance between the first position and the superimposed element text "Weather" 903 is less than the first distance, and the phone's display interface stops displaying the superimposed element text "Weather" 903. When the phone detects that the distance between the first position and the superimposed element text "Weather" 903 is greater than the first distance, as shown in FIG13(c), the phone's display interface redisplays the text "Weather" 903.

[0220] It should be noted that the aforementioned elimination operation may be a click operation or a slide operation. The click operation may be a single-click operation, a double-click operation, or a multi-click operation. For example, in response to detecting a click or slide operation on the first element, the electronic device controls the overlay element to stop displaying on the editing interface.

[0221] S1420: In response to the elimination operation on the first element in the first image, eliminate the first element on the editing interface.

[0222] In an embodiment of the present application, the editing interface of the first picture includes a first control and a second control. The first control may be a smart delete control, and the second control may be a manual delete control. Of course, the first control may also be a manual delete control, and the second control may be a smart delete control, which is not limited here.

[0223] Exemplarily, the first control may be the smart elimination control in FIG. 9 to FIG. 13 , and the second control may be the manual elimination control in FIG. 9 to FIG. 13 .

[0224] In one embodiment, the electronic device may, in response to a user triggering an operation on a first control, determine to use a first elimination method to eliminate a first element. The first elimination method is used to eliminate the element selected by the elimination operation. Thereafter, the electronic device may also, in response to a user triggering an operation on a second control, determine to use a second elimination method to eliminate a second element. The second elimination method is used to eliminate a selected portion of the element targeted by the elimination operation.

[0225] The second element can be a remnant of the first element, or it can be another element unrelated to the first element. For example, the first element is the boat shown in Figure 12(a), and the second element is the remnant of the boat shown in Figure 12(c). Another example is the first element is the boat shown in Figure 10(a), and the second element is the person shown in Figure 10(a).

[0226] In another embodiment, the electronic device may directly respond to the triggering operation of the second control and use the second elimination method to eliminate the first element. For example, as shown in FIG13(a), the mobile phone may respond to the user triggering the manual elimination control 907 and determine to use the manual elimination method to eliminate the first element.

[0227] In an embodiment of the present application, after the electronic device eliminates the first element, it determines a target color based on the colors within a preset range on the first image where the first element is located. The electronic device can then fill the location where the first element was located with the target color. In other words, the electronic device can determine the target color based on the colors surrounding the first element. After the first element is eliminated on the editing interface, the location where the first element was located on the first image is filled with the target color.

[0228] The above target colors can be the same color or different colors, which is not limited here.

[0229] S1430: Display the first image after removing the first element on the editing interface and restore the display of the overlay elements.

[0230] Exemplarily, as shown in (d) in FIG13 , the mobile phone detects that the sliding operation is ended. In response to the sliding operation being ended, the first image after eliminating the boat portion is displayed on the editing interface and the overlay elements are restored to display.

[0231] In an embodiment of the present application, the editing interface further includes a save button. After the electronic device displays the first image after the first element is removed and the overlay elements on the editing interface, the electronic device can save a second image in response to detecting a triggering operation on the save button. The second image includes the first image after the first element is removed and the overlay elements.

[0232] For example, as shown in FIG9( f ), the mobile phone detects a user triggering operation on a save button 908 , and in response to the triggering operation, the mobile phone saves the picture 909 .

[0233] It can be seen that the user can directly trigger the save button in the editing interface to save the image after the elimination process. The user does not need to switch pages and can save the processed image in the editing interface, which has the advantages of simple and convenient operation.

[0234] As shown in Figure 15, an embodiment of the present application discloses an electronic device, which may be the mobile phone described above. The electronic device may specifically include: a touch screen 1501, wherein the touch screen 1501 includes a touch sensor 1506 and a display screen 1507; one or more processors 1502; a memory 1503; one or more application programs (not shown); and one or more computer programs 1504. The above components may be connected via one or more communication buses 1505. The one or more computer programs 1504 are stored in the memory 1503 and configured to be executed by the one or more processors 1502. The one or more computer programs 1504 include instructions that can be used to perform the relevant steps in the above embodiments.

[0235] It is understandable that, in order to realize the above functions, the above-mentioned electronic devices and the like include hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present invention.

[0236] The embodiment of the present application can divide the functional modules of the above-mentioned electronic device etc. according to the above-mentioned method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0237] A schematic diagram of a possible configuration of the electronic device involved in the above embodiment, in which each functional module is divided according to its function, may include a display unit, a transmission unit, and a processing unit. It should be noted that all relevant content of each step involved in the above method embodiment can be referred to in the functional description of the corresponding functional module and will not be repeated here.

[0238] An embodiment of the present application further provides an electronic device comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors and are configured to store computer program code, which includes computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the above-described related method steps to implement the image processing method of the above-described embodiment.

[0239] An embodiment of the present application further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the image processing method in the above-mentioned embodiment.

[0240] An embodiment of the present application further provides a computer program product, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the image processing method in the above-mentioned embodiment.

[0241] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory to enable the device to execute the image processing method performed by the electronic device in the above-mentioned method embodiments.

[0242] Among them, the electronic device, computer-readable storage medium, computer program product or device provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0243] Through the description of the above embodiments, those skilled in the art will clearly understand that for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0244] The functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0245] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as flash memory, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk.

[0246] The above is only a specific embodiment of the present application, but the scope of protection of this application is not limited to this. Any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for processing an image, characterized in that: The method comprises: Displaying an editing interface of a first picture, wherein the first picture includes a first image and an overlay element; In response to detecting a deletion operation on the first element in the first image, controlling the superimposed element to stop being displayed on the editing interface; In response to the elimination operation on the first element in the first image, eliminating the first element on the editing interface; The first image after the first element is eliminated is displayed on the editing interface, and the superimposed element is restored to be displayed.

2. The method according to claim 1, characterized in that The step of displaying the first image after eliminating the first element on the editing interface and restoring the display of the superimposed element comprises: In response to the elimination operation being completed, the first image after the first element is eliminated is displayed on the editing interface, and the overlay element is restored to display.

3. The method according to claim 1, characterized in that In response to detecting a deletion operation on the first element in the first image, controlling the superimposed element to stop being displayed on the editing interface includes: In response to detecting a removal operation on a first element in the first picture, if it is determined that the distance between the first overlay type element and a first position is less than a first distance, controlling the first overlay type element to stop displaying, the first position being a position where the removal operation acts on the display screen; The method further comprises: If it is determined that the distance between the second overlay element and the first position is greater than the first distance, the second overlay element is controlled to continue to be displayed.

4. The method according to claim 3, characterized in that The method further comprises: When the distance between the first overlay element that stops being displayed and the first position is greater than the first distance, the first overlay element is restored to be displayed.

5. The method according to any one of claims 1 to 4, characterized in that: The editing interface includes a first control, and the method further includes: In response to a triggering operation on the first control, determining to use a first elimination method to eliminate elements in the first image, the first elimination method being used to eliminate the elements selected by the elimination operation; In response to the elimination operation on the first element in the first image, eliminating the first element on the editing interface includes: In response to the elimination operation on the first element in the first image, the first element selected by the elimination operation is eliminated on the editing interface.

6. The method according to claim 5, characterized in that The editing interface includes a second control, and the method further includes: In response to a triggering operation on the second control, determining to use a second elimination method to eliminate the element in the first image, the second elimination method being to eliminate a selected partial area of ​​the element targeted by the elimination operation; In response to a removal operation on a second element in the first image, a region selected by the removal operation in the second element is removed on the editing interface.

7. The method according to any one of claims 1 to 6, characterized in that: The editing interface includes a second control, and the method further includes: In response to a triggering operation on the second control, determining to eliminate the first element by adopting a second elimination method; In response to the elimination operation on the first element in the first image, eliminating the first element on the editing interface includes: In response to a removal operation on a first element on the first picture, a region selected by the removal operation in the first element is removed on the editing interface.

8. The method according to any one of claims 1 to 7, characterized in that: The elimination operation includes a click or a slide operation, and in response to detecting the elimination operation on the first element in the first image, controlling the superimposed element to stop being displayed on the editing interface includes: In response to detecting a click or slide operation on the first element, the overlay element is controlled to stop being displayed on the editing interface.

9. The method according to any one of claims 1 to 8, characterized in that: In response to the operation of removing the first element in the first image, after removing the first element on the editing interface, the method further includes: Determine a target color according to a color on the first image that is within a preset range where the first element is located; The target color is used to fill the position where the first element is located; the target color is filled at the position where the first element is located on the first image after the first element is eliminated on the editing interface.

10. The method according to any one of claims 1 to 9, characterized in that: The editing interface includes a save button. After the first image after eliminating the first element and the superimposed element are displayed on the editing interface, the method further includes: In response to detecting a triggering operation on the save button, a second picture is saved, where the second picture includes the first image after eliminating the first element and the overlay-type element.

11. An electronic device, characterized in that: include: A display screen, wherein the display screen is used to display an editing interface of a picture; one or more processors; Memory; One or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the processor, the electronic device executes the method as described in any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are read and executed by one or more processors, the method according to any one of claims 1 to 10 is executed.

Images