Image processing method, electronic device, and computer-readable storage medium

Abstract

This application relates to the field of terminal technology, and particularly to image processing methods, electronic devices, and computer-readable storage media. In this method, when displaying a first image, the electronic device can acquire first user input, delete a first target object from the first image based on the first user input, and reconstruct a second target object from the first image based on a reference image to obtain a second image. The second target object can be an object in the first image occluded by the first target object, and the reference image can contain the second target object. That is, when eliminating the first target object, this application can reconstruct the second target object occluded by the first target object based on a reference image containing the second target object. This improves the accuracy of detail generation of the second target object and the accuracy of the reconstructed second target object, thereby improving the visual effect of the second image after eliminating the first target object and enhancing the user experience.

Description

Technical Field

[0001] This application belongs to the field of terminal technology, and in particular relates to image processing methods, electronic devices and computer-readable storage media. Background Technology

[0002] Artificial intelligence (AI) technology can be widely applied in image processing, such as object removal. When removing an object from an image using AI, it also removes any objects that the object occludes. To ensure a good visual effect after object removal, it's necessary to reconstruct the occluded objects. Current image processing methods typically reconstruct the occluded objects based on image data from the object's surrounding area. However, reconstructing the occluded objects based on surrounding image data is prone to errors in detail generation, resulting in poor accuracy and negatively impacting the user experience. Summary of the Invention

[0003] This application provides an image processing method, an electronic device, and a computer-readable storage medium, which can improve the accuracy of detail generation for occluded objects, improve the accuracy of reconstructed objects, thereby improving the visual effect of the image after object removal and enhancing the user experience.

[0004] In a first aspect, embodiments of this application provide an image processing method applied to an electronic device, the method comprising:

[0005] The electronic device displays a first image, which includes a first target object and a second target object, wherein the second target object is occluded by the first target object in the first image.

[0006] The electronic device acquires a first user input, which triggers the electronic device to delete a first target object in the first image.

[0007] The electronic device deletes the first target object based on the first user input, and reconstructs the second target object in the first image based on the reference image to obtain a second image, wherein the reference image contains the second target object.

[0008] In the image processing method described above, when the electronic device displays a first image, it can detect a first user input that triggers the deletion of a first target object from the first image. Upon detecting the first user input, the electronic device can delete the first target object from the first image based on the first user input, and can reconstruct a second target object in the first image based on a reference image to obtain a second image. The second target object can be an object occluded by the first target object in the first image, and the reference image can contain the second target object. The second image can include the second target object with accurate details. That is, when eliminating the first target object, the electronic device can reconstruct the second target object occluded by the first target object based on a reference image containing the second target object, which can improve the accuracy of the detail generation of the second target object, thereby improving the accuracy of the reconstructed second target object, enhancing the visual effect of the second image after eliminating the first target object, and improving the user experience.

[0009] In some embodiments, the reference image includes an image selected by the user.

[0010] In the image processing method provided in this embodiment, when deleting an object, the user can select a reference image, so that the electronic device can reconstruct the second target object occluded by the first target object in the first image based on the reference image selected by the user, which can improve the accuracy of the reconstruction of the second target object.

[0011] In other embodiments, the reference image includes an image determined by the electronic device based on the first image.

[0012] In the image processing method provided in this embodiment, the electronic device can determine the reference image itself based on the first image, eliminating the need for the user to select the reference image, thus simplifying user operation and improving user experience.

[0013] In one possible implementation, the reference image includes an image determined by the electronic device based on the second target object in the first image.

[0014] In the image processing method provided in this embodiment, the electronic device can directly determine a reference image based on a second target object in a first image. That is, after the user selects a first target object in the first image, the electronic device can determine a second target object in the first image that is occluded by the first target object. Based on the second target object, it can determine one or more reference images containing the second target object. Directly determining the reference image based on the second target object improves the speed and efficiency of reference image determination and enhances the accuracy of the reference image. This allows the electronic device to quickly and accurately reconstruct the second target object based on the reference image, improving the speed and efficiency of image removal and enhancing the user experience.

[0015] In some embodiments, the reference image includes at least one of an image from a gallery, an image obtained from the network, or an image from a media library.

[0016] In the image processing method provided in this embodiment, the electronic device can obtain reference images from a gallery application, the network, or other applications of the electronic device (such as a media library), which can improve the accuracy of the reference images, thereby enabling the electronic device to quickly and accurately reconstruct the second target object based on the reference images.

[0017] In some embodiments, before the electronic device deletes the first target object based on the first user input, the method further includes:

[0018] The electronic device performs recognition processing on the first image to determine the first region corresponding to the first target object;

[0019] The electronic device determines the first target object based on the first user input and the first region corresponding to the first target object.

[0020] In the image processing method provided in this embodiment, after acquiring the first image, the electronic device can perform recognition processing on the first image to determine the region corresponding to each object contained in the first image. This allows the electronic device to quickly and accurately determine the first target object based on the first user input and the regions corresponding to each object contained in the first image when the first user input is obtained. This can improve the deletion speed of the first target object and enhance the user experience.

[0021] In some embodiments, the first user input includes a first preset operation;

[0022] The electronic device determines the first target object based on the first user input and the first region corresponding to the first target object, including:

[0023] The electronic device determines the first position of the first preset operation in the first image;

[0024] The electronic device determines the first target object based on the first location and the first region corresponding to the first target object.

[0025] For example, the first preset operation includes a click operation, a long press operation, or a double-click operation.

[0026] In the image processing method provided in this embodiment, the user can select the first target object by clicking, long-pressing or double-clicking it, which simplifies the selection operation and makes it easier for the user to select the first target object, thereby improving the user experience.

[0027] In other embodiments, the first user input includes a second preset operation;

[0028] The electronic device determines the first target object based on the first user input and the first region corresponding to the first target object, including:

[0029] The electronic device determines the second region selected by the second preset operation in the first image;

[0030] The electronic device determines the first target object based on the second region and the first region corresponding to the first target object.

[0031] For example, the second preset operation includes a box selection operation.

[0032] In the image processing method provided in this embodiment, the user can select the first target object by drawing a box around the area corresponding to the first target object. That is, when the area corresponding to the user's box selection operation includes the first area corresponding to the first target object, the electronic device can determine the first target object based on the first area corresponding to the first target object, which can improve the accuracy of the determination of the first target object, thereby improving the accuracy of the deletion of the first target object and preventing the deletion of other redundant objects.

[0033] In other embodiments, the first user input includes a preset voice, which includes a first keyword, or the preset voice includes the first keyword and a second keyword; the first keyword is used to indicate a first target object to be deleted, and the second keyword is used to indicate a second position of the first target object to be deleted in the first image;

[0034] The electronic device determines the first target object based on the first user input and the first region corresponding to the first target object, including:

[0035] The electronic device determines the first target object based on the first keyword and the first region corresponding to the first target object;

[0036] Alternatively, the electronic device determines the first target object based on the first keyword, the second keyword, and the first region corresponding to the first target object.

[0037] In the image processing method provided in this embodiment, the user can select the first target object by voice, which simplifies the selection operation of the first target object, thereby making it more convenient for the user to select the first target object and improving the user experience.

[0038] In some embodiments, after the electronic device determines the first target object based on the first user input and the first region corresponding to the first target object, the method further includes:

[0039] The electronic device displays a first area corresponding to the first target object;

[0040] In response to an adjustment operation on a first region corresponding to the first target object, the electronic device adjusts the first region corresponding to the first target object.

[0041] In the image processing method provided in this embodiment, after determining the first target object, the electronic device can highlight the area corresponding to the determined first target object, so that the user can adjust the area corresponding to the first target object to accurately determine the first target object that the user wants to delete. This can reduce the possibility of accidental deletion of objects in the first image, improve the accuracy of object elimination, and enhance the user experience.

[0042] In some embodiments, the electronic device reconstructs a second target object in the first image based on a reference image to obtain a second image, including:

[0043] The electronic device acquires image data corresponding to the second target object in the reference image;

[0044] The electronic device reconstructs the second target object in the first image based on the image data to obtain the second image.

[0045] In the image processing method provided in this embodiment, the electronic device can determine the second target object in the reference image and directly fuse the image data corresponding to the second target object in the reference image with the first image to reconstruct the second target object, thereby obtaining the second image after reconstructing the second target object.

[0046] In other embodiments, the electronic device is provided with an image restoration model, which is trained based on the reference image;

[0047] The electronic device reconstructs the second target object in the first image based on the reference image to obtain a second image, including:

[0048] The electronic device reconstructs the second target object in the first image using the image restoration model to obtain a complete second target object;

[0049] The electronic device fuses the complete second target object with the first image based on the reference image to obtain the second image.

[0050] In the image processing method provided in this embodiment, the image inpainting model can perform inpainting of the second target object based on the supervision of a reference image, thereby obtaining a complete second target object. Alternatively, the image inpainting model can first learn the features of the reference image and then inpaint the second target object based on those features, obtaining a complete second target object. In other words, the image inpainting model can accurately inpaint the second target object based on the reference image, improving the accuracy of the inpainting and resulting in an accurately reconstructed second image of the second target object.

[0051] In some embodiments, after the electronic device reconstructs a second target object in the first image based on a reference image to obtain a second image, the method further includes:

[0052] The electronic device performs a smooth fusion process on the second image to obtain a smoothed fused second image.

[0053] In the image processing method provided in this embodiment, after reconstructing the second target object, the electronic device can smoothly blend the edges of the second target object in the second image, which can ensure the natural transition of the second image after reconstructing the second target object, improve the visual effect of the second image, and enhance the user experience.

[0054] Secondly, embodiments of this application provide an image processing apparatus applied to an electronic device, the apparatus comprising:

[0055] A display module is used to display a first image, the first image including a first target object and a second target object, wherein the second target object is occluded by the first target object in the first image;

[0056] An input acquisition module is used to acquire first user input, which triggers the electronic device to delete a first target object in the first image.

[0057] The image editing module is used to delete the first target object according to the first user input, and reconstruct the second target object in the first image according to the reference image to obtain a second image, wherein the reference image contains the second target object.

[0058] In some embodiments, the reference image includes an image selected by the user.

[0059] In other embodiments, the reference image includes an image determined by the electronic device based on the first image.

[0060] In one possible implementation, the reference image includes an image determined by the electronic device based on the second target object in the first image.

[0061] In some embodiments, the reference image includes at least one of an image from a gallery, an image obtained from the network, or an image from a media library.

[0062] In some embodiments, the apparatus further includes:

[0063] The image recognition module is used to perform recognition processing on the first image to determine the first region corresponding to the first target object; and to determine the first target object based on the first user input and the first region corresponding to the first target object.

[0064] In some embodiments, the first user input includes a first preset operation;

[0065] The image recognition module is specifically used to determine the first position of the first preset operation in the first image; and to determine the first target object based on the first position and the first region corresponding to the first target object.

[0066] In one possible implementation, the first preset operation includes a click operation, a long press operation, or a double-click operation.

[0067] In other embodiments, the first user input includes a second preset operation;

[0068] The image recognition module is further configured to determine the second region selected by the second preset operation in the first image; and to determine the first target object based on the second region and the first region corresponding to the first target object.

[0069] In one possible implementation, the second preset operation includes a box selection operation.

[0070] In other embodiments, the first user input includes a preset voice, which includes a first keyword, or the preset voice includes the first keyword and a second keyword; the first keyword is used to indicate a first target object to be deleted, and the second keyword is used to indicate a second position of the first target object to be deleted in the first image;

[0071] The image recognition module is further configured to determine the first target object based on the first keyword and the first region corresponding to the first target object;

[0072] Alternatively, the image recognition module is further configured to determine the first target object based on the first keyword, the second keyword, and the first region corresponding to the first target object.

[0073] In some embodiments, the device further includes an adjustment module:

[0074] The display module is also used to display the first area corresponding to the first target object;

[0075] The adjustment module is used to adjust the first region corresponding to the first target object in response to the adjustment operation on the first region corresponding to the first target object.

[0076] In some embodiments, the image editing module is specifically used to acquire image data corresponding to the second target object in the reference image; and to reconstruct the second target object in the first image based on the image data to obtain the second image.

[0077] In other embodiments, the electronic device is provided with an image restoration model, which is trained based on the reference image;

[0078] The image editing module is further configured to reconstruct the second target object in the first image using the image restoration model to obtain a complete second target object; and to fuse the complete second target object with the first image according to the reference image to obtain the second image.

[0079] In some embodiments, the apparatus further includes:

[0080] The smoothing module is used to perform smoothing and fusion processing on the second image to obtain the smoothed and fused second image.

[0081] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device implements the image processing method described in any one of the first aspects above.

[0082] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a computer, causes the computer to implement the image processing method described in any one of the first aspects above.

[0083] Fifthly, embodiments of this application provide a computer program product, the computer program product including a computer program, which, when run by an electronic device, causes the electronic device to implement the image processing method described in any one of the first aspects above.

[0084] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0085] Figure 1This is an example image showing the removal of objects from an image;

[0086] Figure 2 This is a schematic diagram of the structure of the electronic device to which the image processing method provided in the embodiments of this application is applicable;

[0087] Figure 3 This is a schematic diagram of the software architecture of the electronic device to which the image processing method provided in the embodiments of this application is applicable;

[0088] Figure 4 This is an example of the user interface of the image processing method provided in the embodiments of this application. Figure 1 ;

[0089] Figure 5 and Figure 6 This is an example of the user interface of the image processing method provided in the embodiments of this application. Figure 2 ;

[0090] Figure 7 and Figure 8 This is an example of the user interface of the image processing method provided in the embodiments of this application. Figure 3 ;

[0091] Figure 9 This is a flowchart illustrating the image processing method provided in the embodiments of this application. Figure 1 ;

[0092] Figure 10 This is a schematic diagram of image recognition and segmentation provided in an embodiment of this application;

[0093] Figure 11 This is a flowchart illustrating the image processing method provided in the embodiments of this application. Figure 2 . Detailed Implementation

[0094] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0095] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0096] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0097] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance or order.

[0098] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in yet another embodiment," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0099] Furthermore, the term "multiple" mentioned in the embodiments of this application should be interpreted as two or more.

[0100] The steps involved in the image processing method provided in this application are merely examples, and not all steps are mandatory, nor are all information or message contents required. They can be added or removed as needed during use. The same step or step or message with the same function in this application can be referenced and learned from each other in different embodiments.

[0101] The business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0102] AI technology can be widely applied in the field of image processing, such as in image editing, i.e., AI image editing. AI image editing refers to the intelligent optimization and / or repair of images through technologies such as deep learning, neural networks, or computer vision, such as color correction, contrast adjustment, skin smoothing, or object removal.

[0103] AI-powered image editing can involve image processing technologies such as image recognition, image segmentation, image enhancement, background removal, image reconstruction, or image synthesis.

[0104] Image recognition: This refers to the use of AI technology to identify objects, scenes, or colors in an image, providing intelligent assistance for image editing.

[0105] Image segmentation: This refers to using AI technology to segment an image, distinguish different regions and objects, and facilitate local editing.

[0106] Image enhancement: This can refer to enhancing image quality through AI technology. Examples include increasing resolution, improving color balance, and enhancing contrast.

[0107] Background removal: This refers to using AI technology to identify the foreground and background in an image and then removing the background. For example, it can be used for background replacement in images.

[0108] Image reconstruction: This can refer to image reconstruction using AI technology. For example, repairing damaged images or filling in missing parts.

[0109] Image compositing: This refers to the use of AI technology to complete complex image synthesis. For example, it involves naturally merging multiple image elements to create new visual effects.

[0110] Furthermore, AI image retouching can also involve AI models. For example, deep learning frameworks can be used to build AI models, which can collect large amounts of image data and train the AI ​​model using this data. This allows the AI ​​model to recognize image features and learn retouching rules, enabling it to optimize and / or repair images. The technologies involved in these AI models can include generative adversarial networks (GANs), convolutional neural networks (CNNs), and technologies such as intelligent neutral grayscale, intelligent color correction, and intelligent retouching. This allows the AI ​​model to achieve functions such as multi-scene adaptive recognition and parameter tuning, portrait blemish repair, and personalized image retouching.

[0111] When deleting an object from an image using AI-powered image editing technology, the objects it occludes are usually also deleted. To ensure the visual quality after object deletion, the occluded objects need to be reconstructed. For example, this can be done using image data from the surrounding area of ​​the original object. However, this method of reconstructing occluded objects based on surrounding image data is prone to errors in detail generation, resulting in poor accuracy of the reconstructed objects and negatively impacting the user experience.

[0112] For example, please see Figure 1 , Figure 1 An example diagram showing the removal of objects from an image is shown.

[0113] Users can Figure 1 An image is taken at the location shown in (a) and the captured image (e.g., image A) can be viewed through a gallery application. For example, as shown... Figure 1 As shown in (b), the electronic device can display captured image A in the gallery application. While displaying captured image A, the electronic device can also display an edit button 110. Additionally, the electronic device can display buttons for other functions, such as a share button, a favorite button, a delete button, and a button for viewing more information.

[0114] When a user wants to delete an object in image A (e.g., a person 130 in front of sign 120), the user can click the corresponding edit button 110. For example... Figure 1 As shown in (c), after detecting a click operation on the edit button 110, the electronic device can display buttons related to image processing functions. For example, it can display buttons 140 for the function to delete objects from the image (hereinafter referred to as the removal function), a button for brightening, a button for cropping, and a button for filters. The user can click the button 140 corresponding to the removal function. After clicking the button 140, the user can select the object to be removed (e.g., a person 130), for example, by selecting the area corresponding to the person 130. It should be understood that, as Figure 1 As shown in (c), when a click operation is detected on the edit button 110, the electronic device can also display the save button 150. After completing the editing of image A, the user can click the save button 150 to save the edited content.

[0115] After detecting a click on the button 140 corresponding to the deletion function, the electronic device can activate the deletion function and delete person 130 from image A after detecting a selection operation on the area corresponding to person 130. Since person 130 obscures sign 120, to ensure the visual effect of the image, after deleting person 130 from image A, the electronic device can reconstruct sign 120 based on the image data of the unobstructed portion of sign 120, obtaining the following result: Figure 1 The image shown in (d) (e.g., image B).

[0116] According to Figure 1 From (a) we can see that, Figure 1 The content of the sign 120 reconstructed in image B (d) is not the same as the original content of the sign 120. That is, when the electronic device reconstructs the sign 120 based on the image data in image A, the reconstructed details are not accurate, resulting in poor accuracy of the reconstructed sign 120 and affecting the user experience.

[0117] To address the aforementioned problems, embodiments of this application provide an image processing method, an electronic device, and a computer-readable storage medium. In this method, when the electronic device displays a first image, it can detect first user input, which can trigger the electronic device to delete a first target object from the first image. Upon detecting the first user input, the electronic device can delete the first target object based on the first user input and reconstruct a second target object in the first image based on a reference image to obtain a second image. The second target object can be an object in the first image occluded by the first target object, and the reference image can contain the second target object. That is, in this embodiment, when deleting the first target object, the reconstruction of the second target object occluded by the first target object can be based on a reference image containing the second target object. This improves the accuracy of generating details of the second target object, thereby improving the accuracy of the reconstructed second target object, enhancing the visual effect after deleting the first target object, improving the user experience, and exhibiting strong usability and practicality.

[0118] In this application embodiment, the electronic device can be a mobile phone, tablet computer, wearable device, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), desktop computer, etc. This application embodiment does not impose any restrictions on the specific type of electronic device.

[0119] The following first describes the electronic device involved in the embodiments of this application. Please refer to... Figure 2 , Figure 2 A schematic diagram of an electronic device 200 is shown.

[0120] Electronic device 200 may include a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, antenna 1, antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a headphone jack 270D, a sensor module 280, buttons 290, a camera 291, and a display screen 292, etc. The sensor module 280 may include a pressure sensor 280A, a gyroscope sensor 280B, a barometric pressure sensor 280C, a magnetic sensor 280D, an accelerometer 280E, a proximity sensor 280F, a proximity light sensor 280G, a fingerprint sensor 280H, a temperature sensor 280J, a touch sensor 280K, an ambient light sensor 280L, a bone conduction sensor 280M, etc.

[0121] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 200. In other embodiments of this application, the electronic device 200 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0122] Processor 210 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0123] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0124] The processor 210 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. This memory can store instructions or data that the processor 210 has just used or that are used repeatedly. If the processor 210 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 210, and thus improves the efficiency of the system.

[0125] In some embodiments, the processor 210 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0126] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 200. In other embodiments of this application, the electronic device 200 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0127] The charging management module 240 is used to receive charging input from the charger.

[0128] The power management module 241 is used to connect the battery 242, the charging management module 240, and the processor 210. The power management module 241 receives input from the battery 242 and / or the charging management module 240 to power the processor 210, internal memory 221, display 292, camera 291, and wireless communication module 260, etc.

[0129] The wireless communication function of electronic device 200 can be implemented through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modem processor, and baseband processor.

[0130] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 200 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0131] The mobile communication module 250 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 200. The mobile communication module 250 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 250 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 250 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 250 may be housed in the processor 210. In some embodiments, at least some functional modules of the mobile communication module 250 and at least some modules of the processor 210 may be housed in the same device.

[0132] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through audio devices (not limited to speaker 270A, receiver 270B, etc.) or displays images or videos through display screen 292. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 210 and may be housed in the same device as the mobile communication module 250 or other functional modules.

[0133] The wireless communication module 260 can provide solutions for wireless communication applications on the electronic device 200, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 260 can be one or more devices integrating at least one communication processing module. The wireless communication module 260 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 210. The wireless communication module 260 can also receive signals to be transmitted from processor 210, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0134] In some embodiments, antenna 1 of electronic device 200 is coupled to mobile communication module 250, and antenna 2 is coupled to wireless communication module 260, enabling electronic device 200 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0135] Electronic device 200 implements display functions through a GPU, display screen 292, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 292 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 210 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0136] Display screen 292 is used to display images, videos, etc. Display screen 292 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 200 may include one or N displays 292, where N is a positive integer greater than 1.

[0137] Electronic device 200 can perform shooting functions through ISP, camera 291, video codec, GPU, display screen 292 and application processor.

[0138] The ISP is used to process the data fed back by camera 291.

[0139] Camera 291 is used to capture still images or videos. In some embodiments, electronic device 200 may include one or N cameras 291, where N is a positive integer greater than 1.

[0140] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 200 selects a frequency, the DSP is used to perform Fourier transforms on the frequency energy.

[0141] Video codecs are used to compress or decompress digital video. Electronic device 200 may support one or more video codecs. Thus, electronic device 200 can play or record video in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0142] An NPU (Neural Processing Unit) is a neural network (NN) computing processor that, by borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, rapidly processes input information and can continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0143] The external storage interface 220 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 200. The external memory card communicates with the processor 210 through the external storage interface 220 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0144] Internal memory 221 can be used to store computer executable program code, which includes instructions. Internal memory 221 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 200 (such as audio data, phonebook, etc.). Furthermore, internal memory 221 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 210 executes various functional applications and data processing of electronic device 200 by running instructions stored in internal memory 221 and / or instructions stored in memory disposed in the processor.

[0145] Electronic device 200 can implement audio functions such as music playback and recording through audio module 270, speaker 270A, receiver 270B, microphone 270C, headphone jack 270D, and application processor.

[0146] Audio module 270 is used to convert digital audio information into analog audio signal output, and also to convert analog audio input into digital audio signal. Audio module 270 can also be used for encoding and decoding audio signals.

[0147] Buttons 290 include a power button, volume buttons, etc. Buttons 290 can be mechanical buttons or touch-sensitive buttons. Electronic device 200 can receive button input and generate key signal inputs related to user settings and function control of electronic device 200.

[0148] The software system of electronic device 200 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. For example, the software system of electronic device 200 can adopt a layered architecture such as Android operating system (OS), Harmony OS, or iOS. This application embodiment uses a layered architecture as an example to illustrate the software structure of electronic device 200.

[0149] Figure 3This is a software structure block diagram of an electronic device 200 according to an embodiment of this application.

[0150] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the operating system is divided into four layers, from top to bottom: the application layer, the application framework layer, the runtime and system libraries, and the kernel layer.

[0151] The application layer can include a series of application packages.

[0152] like Figure 3 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and image processing applications.

[0153] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0154] like Figure 3 As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, image processing module, etc.

[0155] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0156] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0157] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0158] The phone manager is used to provide communication functions for electronic devices 200. For example, it manages call status (including connection and disconnection).

[0159] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0160] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0161] The image processing module can be used to implement the image processing method provided in the embodiments of this application. That is, one or more applications in an electronic device can call the image processing module to remove objects from an image. For example, Figures 4-8 In the illustrated embodiment, when the image library calls the image processing module to remove an object (e.g., object A) from an image, after removing object A, the electronic device can reconstruct the object (e.g., object B) occluded by object A in the image based on a reference image. This can result in an accurately reconstructed image of object B, improving the accuracy of object B reconstruction and enhancing the visual effect of the image after removing object A. Similarly, when a social application needs to process an image, it can call the feature recognition module, removal module, reconstruction and fusion module of the image processing module in the system framework layer to perform removal and reconstruction processing on an object in the image.

[0162] The image processing module may include a feature recognition module, an elimination module, and a reconstruction and fusion module. The feature recognition module can be used to perform feature recognition on an image (e.g., a first image or a reference image to which object elimination is needed) to determine the objects contained in the image (e.g., determining a first target object to be eliminated in the first image and a second target object occluded by the first target object, or determining a second target object in the reference image, etc.). The elimination module can be used to eliminate the first target object in the first image identified by the feature recognition module; for example, it can set the pixel value corresponding to the first target object in the first image to a preset value. The reconstruction and fusion module can be used to reconstruct the second target object in the first image based on a reference image containing the second target object after eliminating the first target object in the first image, and then perform smooth fusion processing on the edges of the reconstructed second target object to obtain the image after eliminating the first target object.

[0163] Runtime consists of core libraries and a virtual machine. Runtime is responsible for the scheduling and management of the operating system.

[0164] The core library consists of two parts: one part is the functionalities that the Java language needs to call, and the other part is the core library of the operating system.

[0165] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0166] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0167] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0168] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0169] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0170] A 2D graphics engine is a graphics engine for 2D drawing.

[0171] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0172] The image processing method provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings and specific application scenarios.

[0173] It should be noted that the image processing method provided in this application embodiment can be integrated into image processing applications, for example, it can be used as a function (e.g., an elimination function) in an image processing application. When processing images through an image processing application, the elimination function can be used to remove unwanted objects from the image to improve image quality.

[0174] The image processing application can be a post-processing application suitable for photography, meaning it can process images obtained from photography, such as performing multi-scene adaptive recognition and parameter adjustment, portrait blemish repair, or personalized retouching, significantly saving time and costs associated with manual retouching. Alternatively, the image processing application can be an application suitable for enhancing everyday images. It should be understood that the image processing application can be an application specifically for image processing, or an application that includes image processing functions; this application embodiment does not impose any limitations on this and can be determined according to the actual scenario.

[0175] Alternatively, the image processing method provided in this application embodiment can be built into the gallery application of an electronic device, for example, as a function (e.g., a removal function) within the gallery application. When a user views an image through the gallery application, the user can edit and optimize the image using the relevant functions of the gallery application. For example, the user can use the removal function in the gallery application to delete unwanted objects from the image to optimize the visual effect of the image.

[0176] Alternatively, the image processing method provided in this application embodiment can be built into an electronic device and invoked by one or more applications in the electronic device to remove objects from the image. That is, when a user views an image through an application (e.g., application A), application A can invoke the image processing method provided in this application embodiment based on the user's relevant operations to process the image, thereby deleting unwanted objects from the image and optimizing the visual effect of the image.

[0177] In the image processing method provided in this application embodiment, when displaying a first image, the electronic device can detect a first user input, which triggers the electronic device to delete a first target object from the first image. Upon detecting the first user input, the electronic device can delete the first target object from the first image based on the first user input, and can reconstruct a second target object in the first image based on a reference image to obtain a second image. The second target object can be an object occluded by the first target object in the first image, and the reference image can contain the second target object. The second image can include the second target object with accurate details. That is, when eliminating the first target object in the first image, the electronic device can reconstruct the second target object occluded by the first target object based on a reference image containing the second target object. This improves the accuracy of detail generation for the second target object, thereby improving the accuracy of the reconstructed second target object, enhancing the visual effect of the second image after eliminating the first target object, and improving the user experience.

[0178] The following example illustrates the user interface for image processing provided in this application, using the example of an image processing method built into a gallery application.

[0179] Please see Figure 4 , Figure 4 An example of the user interface of the image processing method provided in this application embodiment is shown. Figure 1 .

[0180] like Figure 4As shown in (a), a user can view images through the gallery application, such as viewing the first image. That is, the electronic device can display the first image in the gallery application interface. When displaying the first image, the electronic device can also display an edit button 410. In addition, the electronic device can also display buttons for other functions, such as buttons for sharing, saving, deleting, and viewing more information.

[0181] When a user wants to delete an object in the first image (e.g., a person 430 in front of sign 420), the user can click the corresponding edit button 410. For example... Figure 4 As shown in (b), after detecting a click on the edit button 410, the electronic device can display buttons related to image processing functions. For example, it can display buttons 440 for the erase function, buttons for brighten, buttons for trim, and buttons for filters. The user can click the erase button 440 to activate the erase function. After clicking the erase button 440, the user can select the object to be erased (e.g., a person 430). For example, as... Figure 4 As shown in (b), the user can select the area corresponding to person 430. Or, as... Figure 4 As shown in (c), the user can click on the area corresponding to person 430. It should be understood that when a click on the edit button 410 is detected, the electronic device can also display the save button 450. After completing the editing of the first image (e.g., deleting person 430), the user can click the save button 450 to save the image.

[0182] After detecting a click on the button 440 corresponding to the deletion function, the electronic device can activate the deletion function and delete the person 430 from the first image after detecting a click or selection operation on the area corresponding to the person 430. After deleting the person 430 from the first image, the electronic device can reconstruct the sign 420 obscured by the person 430 based on a reference image determined by the electronic device. For example, the reference image determined by the electronic device can be as follows: Figure 1 As shown in (a) above, electronic devices can be based on Figure 1 The reference image shown in (a) is used to reconstruct the sign 420, resulting in the following: Figure 4 The image shown in (d) (for example, can be referred to as the second image).

[0183] It should be understood that the process by which the electronic device determines the reference image, and the process by which it reconstructs the occluded object based on the reference image, will be explained in detail later and will not be described here.

[0184] Please see Figure 5 and Figure 6 , Figure 5 and Figure 6 An example of the user interface of the image processing method provided in this application embodiment is shown. Figure 2 .

[0185] like Figure 5 As shown in (a), the user can view the first image through the gallery application. That is, the electronic device can display the first image in the gallery application interface. When displaying the first image, the electronic device can also display the corresponding edit button 510. In addition, the electronic device can also display buttons for other functions, such as the share button, the favorite button, the delete button, and the button for viewing more functions.

[0186] When a user wants to delete an object in the first image (e.g., a person 530 in front of sign 520), the user can click the corresponding edit button 510. For example... Figure 5 As shown in (b), after detecting a click on the edit button 510, the electronic device can display buttons related to image processing functions. For example, it can display buttons 540 for the erase function, buttons for brightening, and buttons for trimming. Additionally, the electronic device can display a button 560 for the reference image. It should be understood that upon detecting a click on the edit button 510, the electronic device can also display a save button 550. After completing the editing of the first image (e.g., deleting a person 530), the user can click the save button 550 to save the image.

[0187] like Figure 5 As shown in (b), the user can click the button 560 corresponding to the reference image to select a reference image. When the electronic device detects a click on the button 560 corresponding to the reference image, it can display... Figure 5 The image library interface 570 shown in (c) can display multiple images and selection buttons for each image. Users can select one or more images from the multiple images displayed in the image library interface 570 as reference images.

[0188] Alternatively, when the electronic device detects a click on button 560 corresponding to the reference image, it can display... Figure 5The image gallery interface 580 shown in (d) can display multiple images categorized by type. For example, the electronic device can display one or more images related to people, one or more images related to locations, and one or more images related to objects, etc. When the electronic device displays images categorized by type, it can also display a button to view more images for each category, allowing users to select the desired image. Users can select one or more images from the multiple images displayed in the image gallery interface 580 as reference images.

[0189] like Figure 5 (c) and Figure 5 As shown in (d), when the electronic device displays multiple images in the gallery interface 570 or 580, it can also display a save button 590. After the user completes the selection of reference images, the user can click the save button 590. For example, as shown in (d) Figure 5 As shown in (c), after the user selects image 57-1, the user can click the corresponding save button 590. When the electronic device detects the click operation on the save button 590, it can save the image selected by the user (e.g., image 57-1) and determine the image selected by the user (e.g., image 57-1) as the reference image.

[0190] In determining the reference image (e.g.) Figure 5 After the image 57-1 shown in (c) is displayed, the electronic device can display Figure 6 The interface shown in (a) indicates that the electronic device can return to displaying the first image. Figure 6 As shown in (a), the user can click the button 540 corresponding to the elimination function to activate it. After clicking the button 540, the user can select the object to be eliminated (e.g., a person 530). For example, as... Figure 6 As shown in (b), users can select the area corresponding to person 530. Or, as... Figure 6 As shown in (c), users can click on the area corresponding to person 530.

[0191] After detecting a click on the button 540 corresponding to the deletion function, the electronic device can activate the deletion function and delete the person 530 from the first image after detecting a click or selection operation on the area corresponding to the person 530. After deleting the person 530 from the first image, the electronic device can reconstruct the sign 520 obscured by the person 530 based on a user-selected reference image (e.g., image 57-1), obtaining... Figure 6 The second image shown in (d) is shown in the diagram.

[0192] Please see Figure 7 and Figure 8 , Figure 7 and Figure 8 An example of the user interface of the image processing method provided in this application embodiment is shown. Figure 3 .

[0193] like Figure 7 As shown in (a), the user can view the first image through the gallery application. That is, the electronic device can display the first image in the gallery application interface. When displaying the first image, the electronic device can also display the corresponding edit button 710. In addition, the electronic device can also display buttons for other functions, such as the share button, the favorite button, the delete button, and the button for viewing more functions.

[0194] When a user wants to delete an object in the first image (e.g., a person 730 in front of a sign 720), the user can click the corresponding edit button 710. For example... Figure 7 As shown in (b), after detecting a click on the edit button 710, the electronic device can display buttons related to image processing functions. For example, it can display buttons 740 for the erase function, buttons for brightening, and buttons for trimming. Additionally, the electronic device can display a button 760 for the reference image. The user can click the erase button 740 to activate the erase function. After clicking the erase button 740, the user can select the object to be erased (e.g., a person 730). For example, as... Figure 7 As shown in (b), after clicking the button 740 corresponding to the elimination function, the user can select the area corresponding to 730. Or, as... Figure 7 As shown in (c), after clicking the button 740 corresponding to the deletion function, the user can click the area corresponding to the person 730. It should be understood that when a click operation on the button 740 corresponding to the edit is detected, the electronic device can also display the button 750 corresponding to save. After completing the editing of the first image (e.g., deleting the person 730), the user can click the button 750 corresponding to save the image.

[0195] like Figure 7 As shown in (d), after selecting the object to be deleted (e.g., person 730), the user can click the button 760 corresponding to the reference image to select a reference image. When the electronic device detects a click on the button 760 corresponding to the reference image, it can display... Figure 8 The image library interface 870 is shown in (a). The image library interface 870 can display multiple images and corresponding selection buttons for each image. Users can select one or more images from the multiple images displayed in the image library interface 870 as reference images.

[0196] Alternatively, when the electronic device detects a click on the button 760 corresponding to the reference image, it can display... Figure 8 The image gallery interface 880 shown in (b) can display multiple images categorized by type. For example, the electronic device can display one or more images related to people, one or more images related to locations, and one or more images related to objects, etc. When the electronic device displays images categorized by type, it can also display a button to view more images for each category, allowing users to select the desired image. Users can select one or more images from the multiple images displayed in the image gallery interface 880 as reference images.

[0197] like Figure 8 (a) and Figure 8 As shown in (b), when the electronic device displays multiple images on the gallery interface 870 or 880, it can also display a save button 890. After the user completes the selection of reference images, the user can click the save button 890. For example, as shown in (b), Figure 8 As shown in (a), after the user selects image 87-1, the user can click the corresponding save button 890. When the electronic device detects the click operation on the save button 890, it can save the image selected by the user and determine the image selected by the user as the reference image.

[0198] In determining the reference image (e.g.) Figure 8 After the image shown in (a) 87-1), the electronic device can display Figure 8 The interface shown in (c) allows the electronic device to return to displaying the first image. While displaying the first image, the electronic device can activate an deletion function to remove the person 730 from the first image. After removing the person 730 from the first image, the electronic device can reconstruct the sign 720 obscured by the person 730 based on a user-selected reference image (e.g., image 87-1), resulting in... Figure 8 The second image shown in (d) is shown in the diagram.

[0199] It should be noted that, Figures 5 to 8 The button corresponding to the shown reference image can be used by the user to select a reference image. The selection of the reference image can be optional; that is, the user can select a reference image using the button or choose not to select one. When the user does not select a reference image, the electronic device can... Figure 4The example shown demonstrates image processing whereby the electronic device can determine a reference image on its own and, based on the determined reference image, reconstruct objects (e.g., sign 720) occluded by the object to be removed in the first image to obtain a detailed and accurate second image.

[0200] based on Figure 4 The user interface shown is Figure 9 This application illustrates a schematic flowchart of the image processing method provided in an embodiment. Figure 1 This method can be applied to the electronic devices described above. For example... Figure 9 As shown, the method may include:

[0201] S901, The electronic device displays the first image.

[0202] It should be noted that the first image can be any image selected by the user. That is, the user can select the image (e.g., the first image) that needs image processing on the electronic device. The electronic device can acquire the first image selected by the user and can display the first image. Image processing can include image removal, i.e., deleting objects from the image. It should be understood that the first image can be an image captured by the electronic device through its camera, or an image acquired by the electronic device from another device, or an image downloaded by the electronic device from the network, etc.

[0203] For example, when the image processing method provided in this application is integrated into an image processing application, the electronic device can display the first image through the window of the image processing application. That is, when a user wants to perform image processing on the first image, the user can open the image processing application and select the first image within the application. The electronic device can then acquire the first image selected by the user and display it in the window of the image processing application.

[0204] For example, when the image processing method provided in this application embodiment is built into the gallery application of an electronic device, the electronic device can display the first image through the window of the gallery application. That is, when a user wants to perform image processing on the first image, the user can open the gallery application and select the first image in the gallery application. The electronic device can obtain the first image selected by the user and display the first image in the window of the gallery application.

[0205] S902, The electronic device detects the first user input, which triggers the electronic device to delete the first target object in the first image.

[0206] It should be understood that the first target object in the first image can refer to the image of the first target object in the first image. Deleting the first target object in the first image can refer to deleting the image of the first target object in the first image. Similarly, the second target object in the first image mentioned later can refer to the image of the second target object in the first image. Reconstructing the second target object in the first image can refer to reconstructing the image of the second target object in the first image. The second target object contained in the reference image mentioned later can refer to the reference image containing the image of the second target object.

[0207] It should be noted that the first target object in the first image may include one or more objects in the first image. The first target object may be an object selected by the user. The first user input may refer to the input that triggers the electronic device to delete the first target object in the first image. For example, the first user input may be used to trigger the electronic device to activate the deletion function, trigger the electronic device to determine the first target object in the first image, and trigger the electronic device to delete the first target object based on the deletion function. That is, when the electronic device displays the first image, the electronic device can detect the first user input. When the first user input is detected, the electronic device can activate the deletion function and determine the first target object in the first image based on the first user input. After activating the deletion function and determining the first target object in the first image, the electronic device can delete the first target object in the first image based on the deletion function.

[0208] In some embodiments, the first user input may include one or more user inputs. For example, the first user input may include user input to activate the elimination function (e.g., referred to as user input A) and user input to select a first target object (e.g., referred to as user input B), and the electronic device may delete the first target object from the first image based on user input A and user input B. Alternatively, the first user input may include user input A to activate the elimination function, user input B to select a first target object, and user input to start elimination (e.g., referred to as user input C), and the electronic device may delete the first target object from the first image based on user input A, user input B, and user input C.

[0209] In one embodiment, the first user input can be voice input, meaning the first user input may include a preset voice message. The preset voice message may include one or more keywords. For example, the preset voice message may include a keyword for activating the elimination function (e.g., keyword A) and a keyword for selecting the first target object (e.g., keyword B). That is, when displaying the first image, the electronic device can activate a sound acquisition device such as a microphone to acquire voice messages. When the acquired voice message contains keyword A, the electronic device can determine that user input A has been detected and can activate the elimination function based on user input A. When the acquired voice message also contains keyword B, the electronic device can further determine that user input B has been detected and can determine the first target object based on user input B, thereby eliminating the first target object based on user input A and user input B.

[0210] It should be understood that keywords A and B can be determined based on the actual scenario, and this application embodiment does not impose any limitations on this. For example, keyword A can be determined as "delete," "eliminate," "remove," "erase," "wipe," or "remove," etc., depending on the actual scenario. For example, keyword B can be determined as the category or name of any object, etc., depending on the actual scenario.

[0211] For example, when displaying the first image, the electronic device can activate the microphone to capture speech. When a captured speech (e.g., speech A) includes "delete the stone on the far right of the image," the electronic device can determine that speech A contains the keyword A (i.e., delete). At this point, the electronic device can determine that user input A has been detected and can activate the deletion function based on user input A. Additionally, the electronic device can also determine that speech A contains the keyword B (i.e., stone). At this point, the electronic device can also determine that user input B has been detected and can determine the first target object (i.e., the stone) to be deleted based on user input B. Thus, the rightmost stone in the first image can be deleted based on user input A and user input B.

[0212] For example, when displaying the first image, the electronic device can activate its microphone to capture speech. When a captured speech (e.g., speech B) includes "eliminate the person in front of the sign," the electronic device can determine that speech B contains the keyword A (i.e., eliminate). At this point, the electronic device can determine that user input A has been detected and can activate the elimination function based on user input A. Alternatively, the electronic device can also determine that speech B contains the keyword B (i.e., person). At this point, the electronic device can also determine that user input B has been detected and can determine the first target object (i.e., the person) to be deleted based on user input B. Thus, the person in front of the sign in the first image can be deleted based on user input A and user input B.

[0213] In another embodiment, the first user input can be an operation input, that is, the first user input can include a preset operation. The preset operation can include one or more operations. For example, the preset operation can include an operation to activate the elimination function (e.g., operation A) and an operation to select a first target object (e.g., operation B). For example, the preset operation can include operation A to activate the elimination function, operation B to select the first target object, and an operation to start elimination (e.g., operation C). That is, when displaying the first image, when operation A is detected, the electronic device can determine that user input A has been detected and can activate the elimination function based on user input A. After activating the elimination function, when operation B is detected, the electronic device can determine that user input B has been detected and can determine the first target object based on user input B. After determining the first target object, the electronic device can delete the first target object based on the elimination function. Alternatively, when displaying the first image, when operation A is detected, the electronic device can determine that user input A has been detected and can activate the elimination function based on user input A. After activating the elimination function, when operation B is detected, the electronic device can determine that user input B has been detected and can determine the first target object based on user input B. Upon detecting operation C, the electronic device can determine that user input C has been detected and can begin deleting the identified first target object based on user input C.

[0214] It should be understood that operations A, B, and C can be specifically determined according to the actual scenario, and the embodiments of this application do not impose any limitations on this. For example, operation A can be determined as a click operation, touch operation, long press operation, double-click operation, or tapping operation (such as a single-knuckle tapping operation, a two-knuckle tapping operation, or a three-knuckle tapping operation), etc., depending on the actual scenario. For example, operation B can be determined as a click operation, touch operation, long press operation, double-click operation, tapping operation, or selection operation, etc., depending on the actual scenario. For example, operation C can be determined as a click operation, touch operation, long press operation, double-click operation, or tapping operation, etc., depending on the actual scenario.

[0215] For example, such as Figure 4 As shown in (a), when displaying the first image, the electronic device can also display buttons corresponding to functions such as editing. When the user wants to delete the first target object in the first image, the user can click the corresponding editing button 410. Figure 4 As shown in (b), when the electronic device detects a click on the edit button 410, it can display the erase function button 440. The user can click the erase function button 440 to activate the erase function. Figure 4 (b) or Figure 4As shown in (c), after activating the elimination function, the user can select or click the area corresponding to the first target object in the first image to delete the first target object. When the electronic device detects a click operation on the button 440 corresponding to the elimination function, it can determine that user input A has been detected and can activate the elimination function based on user input A. After activating the elimination function, when a selection or click operation on the area corresponding to the first target object is detected, the electronic device can determine that user input B has been detected and can determine the first target object based on user input B. After determining the first target object, the electronic device can delete the first target object.

[0216] For example, when displaying the first image, the electronic device can also display buttons corresponding to functions such as editing. When a user wants to delete the first target object in the first image, the user can click the corresponding editing button. When the electronic device detects a click on the editing button, it can display the button corresponding to the elimination function. The user can click the button corresponding to the elimination function to start the elimination function. After the elimination function is started, the electronic device can display a "Start Elimination" button. After the elimination function is started, the user can select or click on the area corresponding to the first target object in the first image. After selecting or clicking on the area corresponding to the first target object in the first image, the user can also click the "Start Elimination" button to delete the first target object. When the electronic device detects a click on the button corresponding to the elimination function, it can determine that user input A has been detected and can start the elimination function based on user input A. After the elimination function is started, when a selection or click on the area corresponding to the first target object is detected, the electronic device can determine that user input B has been detected and can determine the first target object based on user input B. After the elimination function is started, when a click on the "Start Elimination" button is detected, the electronic device can also determine that user input C has been detected and can start deleting the determined first target object based on user input C.

[0217] For example, when an electronic device displays a first image, if a user wants to eliminate a first target object from the first image, the user can activate the elimination function by tapping the display interface of the electronic device with their knuckles (such as a single knuckle, two knuckles, or three knuckles), tapping the back of the electronic device, or tapping the side of the electronic device. After activating the elimination function, the user can also select or click on the area corresponding to the first target object in the first image to delete the first target object. When the electronic device detects a knuckle tap, it can determine that it has detected user input A and can activate the elimination function based on user input A. After activating the elimination function, when it detects a selection or click operation on the area corresponding to the first target object, the electronic device can determine that it has detected user input B and can identify the first target object based on user input B. After identifying the first target object, the electronic device can delete the first target object.

[0218] For example, when an electronic device displays a first image, if a user wants to eliminate a first target object from the first image, the user can activate the elimination function by tapping the display interface of the electronic device with their knuckles (such as a single knuckle, two knuckles, or three knuckles), tapping the back of the electronic device, or tapping the side of the electronic device. After activating the elimination function, the user can also select or click on the area corresponding to the first target object in the first image. After selecting or clicking on the area corresponding to the first target object in the first image, the user can continue to tap the display interface of the electronic device with their knuckles (such as a single knuckle, two knuckles, or three knuckles), tapping the back of the electronic device, or tapping the side of the electronic device to delete the first target object. Alternatively, after activating the elimination function, the electronic device can display a "Start Elimination" button. After selecting or clicking on the area corresponding to the first target object in the first image, the user can click the "Start Elimination" button to delete the first target object. When the electronic device detects a knuckle tapping operation, it can determine that it has detected user input A and can activate the elimination function based on user input A. After activating the elimination function, when a selection or click operation is detected on the area corresponding to the first target object, the electronic device can determine that user input B has been detected and can determine the first target object based on user input B. After activating the elimination function, when a knuckle tap operation or a click operation on the start elimination button is detected again, the electronic device can determine that user input C has been detected and can begin deleting the determined first target object based on user input C.

[0219] It should be noted that the first user input described above, which may be voice input containing preset voice or operation input containing preset operation, is only an illustrative explanation and should not be construed as a limitation on the embodiments of this application. In the embodiments of this application, the first user input may also be other types of input.

[0220] In one embodiment, the first user input can be gesture input, that is, the first user input can include preset gestures. The preset gestures can include one or more gestures.

[0221] For example, preset gestures may include a gesture to activate the deletion function (e.g., gesture A) and a gesture to select a first target object (e.g., gesture B). That is, when displaying the first image, if gesture A is detected, the electronic device can determine that user input A has been detected and can activate the deletion function based on user input A. After activating the deletion function, if gesture B is detected, the electronic device can determine that user input B has been detected and can determine the first target object based on user input B. After determining the first target object, the electronic device can delete the first target object. It should be understood that gesture A and gesture B can be determined according to the actual scenario, and this application embodiment does not impose any limitations on this.

[0222] In another embodiment, the first user input may include operation input and voice input, that is, the first user input may include preset operation and preset voice.

[0223] For example, the preset operation may include an operation to activate the elimination function (e.g., operation A), and the preset voice may be voice containing a keyword for selecting the first target object (e.g., keyword B). That is, when the first image is displayed, if operation A is detected, the electronic device can determine that user input A has been detected and can activate the elimination function based on user input A. After activating the elimination function, the electronic device can capture voice through a microphone. When the captured voice contains keyword B, the electronic device can determine that user input B has been detected and can determine the first target object based on user input B. After determining the first target object, the electronic device can delete the first target object.

[0224] For example, the preset voice message can be a message containing a keyword (e.g., keyword A) to initiate the elimination function, and the preset operations can be selecting a first target object (e.g., operation B) and starting elimination (e.g., operation C). That is, when displaying the first image, the electronic device can capture voice messages via a microphone. When the captured voice message contains keyword A, the electronic device can determine that user input A has been detected and can initiate the elimination function based on user input A. After initiating the elimination function, when operation B is detected, the electronic device can determine that user input B has been detected and can determine the first target object based on user input B. After initiating the elimination function, when operation C is detected, the electronic device can determine that user input C has been detected and can begin deleting the determined first target object based on user input C.

[0225] In another embodiment, the first user input may include operation input and gesture input, that is, the first user input may include preset operation and preset gesture generation.

[0226] For example, the preset operation could be activating the elimination function (e.g., operation A), and the preset gesture could be selecting a first target object (e.g., gesture B). That is, when displaying the first image, if operation A is detected, the electronic device can determine that user input A has been detected and can activate the elimination function based on user input A. After activating the elimination function, if gesture B is detected, the electronic device can determine that user input B has been detected and can determine the first target object based on user input B. After determining the first target object, the electronic device can delete the first target object.

[0227] For example, the preset gesture can be a gesture to activate the elimination function (e.g., gesture A). Preset operations can include selecting a first target object (e.g., operation B) and starting elimination (e.g., operation C). That is, when the first image is displayed, if gesture A is detected, the electronic device can determine that user input A has been detected and can activate the elimination function based on user input A. After activating the elimination function, if operation B is detected, the electronic device can determine that user input B has been detected and can determine the first target object based on user input B. After activating the elimination function, if operation C is detected, the electronic device can determine that user input C has been detected and can begin deleting the determined first target object based on user input C.

[0228] In another embodiment, the first user input may include voice input, operation input, and gesture input; that is, the first user input may include preset voice, preset gesture, and preset operation.

[0229] For example, the preset voice can be a voice containing a keyword that initiates the elimination function (e.g., keyword A), the preset gesture can be a gesture for selecting the first target object (e.g., gesture B), and the preset operation can be an operation to start elimination (e.g., operation C). That is, when displaying the first image, the electronic device can capture voice through a microphone. When the captured voice contains keyword A, the electronic device can determine that user input A has been detected and can initiate the elimination function based on user input A. After initiating the elimination function, when gesture B is detected, the electronic device can determine that user input B has been detected and can determine the first target object based on user input B. After initiating the elimination function, when operation C is detected, the electronic device can determine that user input C has been detected and can begin deleting the determined first target object based on user input C.

[0230] For example, the preset gesture can be a gesture to activate the elimination function (e.g., gesture A). The preset voice can be voice containing a keyword for selecting the first target object (e.g., keyword B), and the preset operation can be an operation to start elimination (e.g., operation C). That is, when the first image is displayed, if gesture A is detected, the electronic device can determine that user input A has been detected and can activate the elimination function based on user input A. After the elimination function is activated, the electronic device can capture voice through a microphone. When the captured voice contains keyword B, the electronic device can determine that user input B has been detected and can determine the first target object based on user input B. After the elimination function is activated, if operation C is detected, the electronic device can determine that user input C has been detected and can begin deleting the determined first target object based on user input C.

[0231] S903, The electronic device deletes the first target object based on the first user input.

[0232] In this embodiment, after detecting the first user input, the electronic device can activate the elimination function and determine the first target object based on the first user input, and can delete the first target object based on the elimination function. It should be noted that the electronic device determining the first target object based on the first user input can mean determining the region of the first target object in the first image based on the first user input. Deleting the first target object can mean clearing the image data (e.g., pixel values) corresponding to the first target object.

[0233] For example, after determining the first target object in the first image, that is, after determining the region of the first target object in the first image, the electronic device can set the pixel values ​​of each pixel corresponding to the first target object (i.e., each pixel of the first target object within the region of the first image) to default values. The default values ​​can be determined according to the actual scenario, and this embodiment does not impose any restrictions on this. For example, the default value can be determined to be 0 or 255 according to the actual scenario.

[0234] The process by which an electronic device determines a first target object based on input from a first user will be described in detail below.

[0235] In some embodiments, after acquiring the first image, the electronic device can perform object recognition and segmentation on the first image to obtain regions corresponding to each object contained in the first image, or to obtain regions corresponding to each object contained in the first image and the category of each object. That is, after acquiring the first image, the electronic device can identify the category of each object contained in the first image through image recognition and segmentation technology, and can segment the region corresponding to each object. The object category can include one or more, such as people, signs, animals, trees, or lawns, etc.

[0236] Upon detecting the first user input, the electronic device can determine the first target object based on the first user input and the regions corresponding to the objects contained in the first image. Alternatively, the electronic device can determine the first target object based on the first user input, the regions corresponding to the objects contained in the first image, and the categories of the objects.

[0237] In one embodiment, when the first user input is an operation input, that is, the first user input includes a preset operation, the preset operation includes operation B for selecting a first target object, and operation B is a click operation, touch operation, long press operation, double-click operation, or tap operation, etc., when the first user input is detected, the electronic device can determine the position of the click operation, touch operation, long press operation, double-click operation, or tap operation in the first image (for example, it can be called position A), and can determine the first target object based on position A and the corresponding regions of each object in the first image.

[0238] It should be understood that location A can include one or more, and multiple locations A can correspond to one or more first target objects. That is, after the elimination function is activated, the user can select the first target object through one operation B, or through two or more operations B. The following will illustrate this by assuming that the user selects the first target object through one operation B, i.e., location A includes one object.

[0239] For example, please see Figure 10 , Figure 10 A schematic diagram of image recognition and segmentation provided in an embodiment of this application is shown.

[0240] On electronic device display Figure 10 When the first image shown in (a) is displayed, the electronic device can perform object recognition and segmentation on the first image to obtain, as shown in the figure. Figure 10The recognition and segmentation results shown in (b) indicate that the first image includes person 1010, sign 1020, person 1030, tree 1040, and tree 1050, etc. Furthermore, when recognizing and segmenting objects in the first image, the electronic device can determine that person 1010 is located in region A, sign 1020 in region B, person 1030 in region C, tree 1040 in region D, and tree 1050 in region E, etc. At this point, if the electronic device determines that operation B is located in region A, it can determine that the first target object the user wants to delete is person 1010. If the electronic device determines that operation B is located in region B, it can determine that the first target object the user wants to delete is sign 1020. If the electronic device determines that operation B is located in region C, it can determine that the first target object the user wants to delete is person 1030, and so on.

[0241] In another embodiment, when the first user input is an operation input, that is, the first user input includes a preset operation, the preset operation includes operation B for selecting the first target object, and operation B is a box selection operation, when the first user input is detected, the electronic device can determine the area selected by the box selection operation in the first image (for example, it can be called the box selection area), and can determine the first target object according to the box selection area and the areas corresponding to each object contained in the first image.

[0242] It should be understood that the selection area can include one or more objects. One selection area can correspond to one or more first target objects, and multiple selection areas can also correspond to one or more first target objects. That is, after activating the elimination function, the user can select the first target object through a single selection operation, or through two or more selection operations. The following will illustrate this by assuming that the user selects the first target object through a single selection operation, i.e., the selection area includes one object.

[0243] For example, when the first image includes objects A, B, and C, and object recognition and segmentation are performed on the first image to determine that the region corresponding to object A is region A, the region corresponding to object B is region B, and the region corresponding to object C is region C, if the electronic device determines that the bounding box operation in the first image includes region A, the electronic device can determine that the first target object the user wants to delete includes object A. If the electronic device determines that the bounding box operation in the first image includes region B, the electronic device can determine that the first target object the user wants to delete includes object B. If the electronic device determines that the bounding box operation in the first image includes region C, the electronic device can determine that the first target object the user wants to delete includes object C. If the electronic device determines that the bounding box operation in the first image includes both regions A and B, the electronic device can determine that the first target object the user wants to delete includes both objects A and B. If the electronic device determines that the bounding box operation in the first image includes regions A, B, and C, the electronic device can determine that the first target object the user wants to delete includes objects A, B, and C.

[0244] It should be noted that the selected area including a certain area (e.g., area A) can mean that the selected area includes the entire area of ​​area A, or it can mean that the selected area includes a part of area A. The specific meaning can be determined according to the actual scenario, and this application embodiment does not limit it.

[0245] In another embodiment, when the first user input is voice input, that is, when the first user input includes a preset voice, the electronic device can determine the category of the object that the user wants to delete (e.g., it can be called object category B) based on the preset voice, and determine the first target object that the user wants to delete based on object category B, the category of each object contained in the first image (e.g., it can be called object category A), and the region corresponding to each object.

[0246] For example, when the first image includes objects A, B, and C, and object recognition and segmentation are performed on the first image to determine that the region corresponding to object A is region A, the region corresponding to object B is region B, and the region corresponding to object C is region C, and the category of object A is determined to be a person, the category of object B to be a sign, and the category of object C to be lawn, if the electronic device determines that object category B is a person based on preset voice, the electronic device can determine that the first target object the user wants to delete is the person corresponding to region A. If the electronic device determines that object category B is a sign based on preset voice, the electronic device can determine that the first target object the user wants to delete is the sign corresponding to region B. If the electronic device determines that object category B is lawn based on preset voice, the electronic device can determine that the first target object the user wants to delete is the lawn corresponding to region C.

[0247] In another embodiment, when the first user input is voice input, i.e., the first user input includes a preset voice, the electronic device can determine the object category B and the object's location (e.g., location B) that the user wants to delete based on the preset voice. Additionally, the electronic device can also determine the positional relationships between objects in each of the first images through object recognition. Subsequently, the electronic device can determine the first target object that the user wants to delete based on the object category B and location B, as well as the object category A corresponding to each object in the first image, the positional relationships between objects, and the region corresponding to each object.

[0248] For example, if the first image includes objects A, B, C, and D, and object recognition and segmentation are performed on the first image, determining that object A is in front of object B, object D is in front of object C, and that the region corresponding to object A is region A, the region corresponding to object B is region B, the region corresponding to object C is region C, and the region corresponding to object D is region D, and that the categories of object A and object D are both "person," object B is a sign, and object C is a stone, then if the preset voice message acquired by the electronic device (e.g., preset voice message A) includes "delete the person in front of the sign," the electronic device can determine the object category B as "person" based on preset voice message A, and can determine the location B as in front of the sign (i.e., object B). Therefore, the electronic device can determine that the first target object the user wants to delete is the person corresponding to region A. Similarly, if the preset voice message acquired by the electronic device (e.g., preset voice message B) includes "delete the person in front of the stone," the electronic device can determine the object category B as "person" based on preset voice message B, and can determine the location B as in front of the stone (i.e., object C). Therefore, the electronic device can determine that the first target object the user wants to delete is the person corresponding to region D.

[0249] It should be understood that in a scenario where the first user input is gesture input, meaning the first user input includes a preset gesture, and the preset gesture includes gesture B for selecting a first target object, the electronic device can determine the first target object based on gesture B. The specific details of determining the first target object based on gesture B are similar to those related to determining the first target object based on gesture B, and for the sake of simplicity, will not be repeated here. For example, the first target object can be determined based on the location or area corresponding to the gesture.

[0250] It should be noted that the embodiments of this application do not limit the method by which the electronic device identifies and segments objects in the first image, and can be determined according to the actual scenario.

[0251] For example, an electronic device can use a backpropagation (BP) neural network to identify and segment objects in a first image, obtaining the regions corresponding to each object and the category of each object. Before using the BP neural network for object identification and segmentation, the BP neural network can be optimized using a genetic algorithm (GA). For example, GA can be used to optimize the weights and thresholds of the BP neural network, improving the accuracy of object identification and segmentation. Furthermore, before using the BP neural network for object identification and segmentation, it can be trained by combining multi-scale feature aggregation. This means that by combining features at different scales, more detailed information of the image can be captured, further improving the accuracy of object identification and segmentation.

[0252] It should be understood that the entity that optimizes the BP neural network using GA and the entity that trains the BP neural network by combining multi-scale feature aggregation can be an electronic device; or, it can be other devices, such as a cloud server. That is, the cloud server can optimize the BP neural network using GA and train the BP neural network by combining multi-scale feature aggregation to obtain a trained BP neural network. After obtaining the trained BP neural network, the cloud server can send the trained BP neural network to the electronic device, which can then perform object recognition and segmentation on the first image based on the trained BP neural network.

[0253] For example, electronic devices can use linear discriminant analysis (LDA) and k-nearest neighbor classifier (KNN) to identify and segment objects in a first image, thereby obtaining the regions corresponding to each object in the first image and the categories of each object.

[0254] LDA (Latent Derivative Analysis) is a supervised dimensionality reduction technique that seeks the optimal projection direction to classify data by maximizing between-class scatter and minimizing within-class scatter. Within-class scatter measures the dispersion of data within each class, while between-class scatter measures the dispersion of data across different classes. The goal of LDA is to find a projection direction that minimizes within-class scatter and maximizes between-class scatter. In object recognition and segmentation, LDA can help reduce feature dimensionality while preserving as much class separability information as possible.

[0255] KNN can calculate the distance between the data to be classified and each sample, then select the K nearest samples, and vote based on the categories of these K samples to determine the category of the data to be classified.

[0256] When using LDA and KNN to recognize and segment objects in a first image, the electronic device can first use LDA to reduce the dimensionality of the features in the first image. Then, the electronic device can use KNN to recognize and segment objects. That is, feature dimensionality reduction can be performed first using LDA to reduce the feature dimensions, reduce the amount of computation, and improve the computational efficiency of KNN. Moreover, LDA can select the features most helpful for classification, which can improve the accuracy of KNN classification.

[0257] For example, an electronic device can first use a BP neural network to perform preliminary object recognition and segmentation on a first image, and then use LDA and KNN to perform precise object recognition and segmentation on the first image, so as to improve the accuracy of object recognition and segmentation through BP neural network, LDA and KNN.

[0258] As can be seen from the above, after acquiring the first image, the electronic device can perform object recognition and segmentation on the first image in advance to determine the regions corresponding to each object contained in the first image. This allows the electronic device to quickly and accurately determine the first target object that the user wants to delete based on the first user input and the segmented objects when the first user input is detected.

[0259] In other embodiments, the electronic device may also identify and segment objects in the first image based on the first user input after detecting the first user input, in order to determine the first target object. That is, the electronic device may not perform object identification and segmentation on the first image before acquiring it. Upon detecting the first user input, the electronic device may perform object identification and segmentation on the first image based on the first user input to determine the first target object that the user wants to delete.

[0260] For example, when the first user input is an operation input, that is, the first user input includes a preset operation, the preset operation includes the operation B of selecting the first target object, and the operation B is a click operation, touch operation, long press operation, double-click operation or tap operation, etc., when the first user input is detected, the electronic device can determine the position A of the click operation, touch operation, long press operation, double-click operation or tap operation in the first image, and can perform object recognition and segmentation on the first image according to the position A, and determine the first target object corresponding to the position A.

[0261] For example, when the first user input is an operation input, that is, the first user input includes a preset operation, the preset operation includes operation B of selecting the first target object, and operation B is a box selection operation, when the first user input is detected, the electronic device can determine the box selection area in the first image, and can perform object recognition and segmentation on the first image according to the box selection area to determine the first target object corresponding to the box selection area.

[0262] For example, when the first user input is voice input, that is, when the first user input includes preset voice, the electronic device can determine the object category B and the object location B that the user wants to delete based on the preset voice, and can perform object recognition and segmentation on the first image based on the object category B and the location B, and the first target object corresponding to the object category B and the location B.

[0263] It should be noted that the embodiments of this application do not limit the specific method by which the electronic device identifies and segments the first image based on the position or region of the first target object in the first image, or the specific method by which the electronic device identifies and segments the first image based on the object category B and position B corresponding to the first target object, and can be determined according to the actual scenario.

[0264] In one embodiment, to avoid accidental deletion of objects and improve the accuracy of object removal, after determining a first target object based on first user input, the electronic device can highlight the determined first target object. For example, the electronic device can highlight the area corresponding to the determined first target object. While highlighting the determined first target object, the electronic device can also display a confirm button and a cancel button. The confirm button can be used to confirm that the first target object determined by the electronic device is the object the user wants to delete. The cancel button can be used to cancel the determination of the first target object by the electronic device, that is, to confirm that the first target object determined by the electronic device is not the object the user wants to delete.

[0265] For example, when highlighting the identified first target object, the electronic device may also display an adjustment button. The adjustment button can be used to adjust the first target object, for example, to adjust the area corresponding to the first target object. That is, if it is determined that the first target object displayed by the electronic device is not the object the user wants to delete, the user can trigger the adjustment button to adjust the area corresponding to the first target object.

[0266] S904. The electronic device reconstructs the second target object in the first image based on the reference image to obtain the second image. The second target object is an object occluded by the first target object, and the reference image contains the second target object.

[0267] In this embodiment, after deleting the first target object from the first image, the electronic device can reconstruct the object occluded by the first target object (i.e., the second target object) based on a reference image determined by the electronic device itself, thereby obtaining a second image. This improves the integrity of the objects in the second image, enhances the visual effect of the second image, and improves the user experience. The second image can be the image obtained by deleting the first target object from the first image and reconstructing the second target object. It should be understood that the second target object can be an object completely occluded by the first target object, or an object partially occluded by the first target object. That is, the second target object can include one or more incomplete objects with missing parts in the first image.

[0268] In one embodiment, after determining the reference image, the electronic device can determine the second target object in the reference image based on the first image (e.g., the second target object in the first image and / or the first target object), so as to reconstruct the second target object in the first image based on the second target object in the reference image to obtain the second image.

[0269] For example, an electronic device can identify and segment objects in a reference image, determine each object contained in the reference image, and determine the similarity between each object in the reference image and a second target object in a first image, thereby determining the second target object in the reference image based on the similarity. It should be understood that the similarity between the second target object in the reference image and the second target object in the first image is greater than or equal to a preset similarity. The preset similarity can be determined based on the actual scenario, and this application embodiment does not impose any limitations on it.

[0270] For example, an electronic device can identify and segment objects in a reference image, determine each object in the reference image and the environmental information surrounding each object, and determine the second target object in the reference image based on the environmental information surrounding each object in the reference image and the environmental information surrounding the second target object in the first image (or the environmental information surrounding the first target object). The position of the second target object in the reference image in the actual environment can be the same as the position of the second target object in the first image in the actual environment; therefore, the environmental information surrounding the second target object in the reference image is similar to the environmental information surrounding the second target object (or the first target object) in the first image.

[0271] It should be noted that the embodiments of this application do not limit the specific method by which the electronic device determines the second target object in the reference image based on the first image, and can be determined according to the actual scenario.

[0272] In some embodiments, after deleting the first target object in the first image, the electronic device can directly fuse the image data corresponding to the second target object in the reference image with the first image to reconstruct the second target object, thereby obtaining a second image after reconstructing the second target object.

[0273] For example, after deleting the first target object from the first image, the electronic device can extract feature points from both the reference image and the first image, and match the feature points in the reference image with those in the first image. Subsequently, based on the feature point matching results, the electronic device can align the reference image and the first image, for example, aligning the second target object in the reference image with its corresponding position in the first image. After aligning the reference image and the second image, the electronic device can fuse the image data corresponding to the second target object in the reference image with the first image to reconstruct the second target object, thereby obtaining the second image.

[0274] It should be noted that the above-described method of fusing the image data corresponding to the second target object in the reference image with the first image is merely illustrative and should not be construed as a limitation on the embodiments of this application. In the embodiments of this application, the electronic device may also fuse the image data corresponding to the second target object in the reference image with the first image in other ways. Furthermore, the embodiments of this application do not limit the method of extracting feature points, which can be determined according to the actual scenario. For example, feature points can be extracted using the scale-invariant feature transform (SIFT) algorithm. The embodiments of this application do not limit the method of feature point matching, which can be determined according to the actual scenario. For example, feature point matching can be performed using feature matching algorithms such as fast library for approximate nearest neighbors (FLANN) or brute force matcher (BFMatcher). Similarly, the embodiments of this application do not limit the method of image alignment, which can be determined according to the actual scenario. For example, image alignment can be performed using the random sample consensus (RANSAC) algorithm.

[0275] In other embodiments, after deleting the first target object from the first image, the electronic device can repair the second target object using an image inpainting model to obtain a complete second target object. Subsequently, the electronic device can fuse the complete second target object with the first image based on a reference image to obtain a second image after reconstructing the second target object.

[0276] In one embodiment, the image inpainting model can perform inpainting of the second target object based on the supervision of a reference image, thereby obtaining a complete second target object. Alternatively, the image inpainting model can first learn the features of a reference image and then use those features to inpaint the second target object, resulting in a complete second target object.

[0277] For example, after deleting a first target object from a first image, the electronic device can input the first image (with the target object removed) into an image inpainting model for processing. The image inpainting model can identify a second target object in the first image and extract its features (such as edges, focus, and texture). Subsequently, based on these features, the image inpainting model can repair the second target object, obtaining a complete second target object. Specifically, when repairing the second target object based on its features, the image inpainting model can do so either under the supervision of a reference image or based on features learned from the reference image by the image inpainting model.

[0278] It should be noted that the image inpainting model can be determined based on the actual scenario, and this application embodiment does not impose any limitations on it. For example, the image inpainting model can be determined as a cascaded modulation-generative adversarial network (CM-GAN) model based on the actual scenario. CM-GAN can include an encoder with Fourier convolutional blocks, which can extract multi-scale features from images containing missing regions. Additionally, CM-GAN can include a dual-stream decoder, which can perform global modulation and spatial modulation. Global modulation can be responsible for coarse but semantically aware structural synthesis, while spatial modulation can refine the feature maps in a spatially adaptive manner, ensuring that the synthesized local details are consistent with global details.

[0279] It should be understood that the embodiments of this application do not limit the specific method by which the electronic device fuses the complete second target object with the first image based on the reference image to obtain the second image after reconstructing the second target object, and can be determined according to the actual scenario. For example, after deleting the first target object from the first image, the electronic device can extract feature points from both the reference image and the first image, and can match the feature points in the reference image with the feature points in the first image. Subsequently, the electronic device can align the reference image and the first image according to the matching results of the feature points. After aligning the reference image and the first image, the electronic device can fuse the image data corresponding to the second target object obtained based on the reference image with the first image to reconstruct the second target object, thereby obtaining the second image after reconstructing the second target object.

[0280] For example, in deleting Figure 4 After person 430 is shown in (a) above, the electronic device can use a reference image determined by itself (e.g., Figure 1 The reference image shown in (a) is used to reconstruct the sign 420 that is partially obscured by person 430, in order to obtain Figure 4 The second image shown in (d) is based on the reference image, which can accurately reconstruct the details of the sign 420, improve the accuracy of the sign 420 reconstruction, enhance the visual effect of the image, and improve the user experience.

[0281] In some embodiments, the second target object is generally an object surrounding the first target object. When deleting the first target object from the first image, the electronic device can save the position of the first target object in the first image, so that when repairing the second target object, the electronic device or image repair model can determine the objects surrounding the first target object based on the position of the first target object in the first image, and can determine the second target object based on the objects surrounding the first target object.

[0282] It should be understood that the position of the first target object in the first image may include the positions of each pixel in the region corresponding to the first target object (hereinafter referred to as the target region for ease of understanding) in the first image, or it may include the positions of key pixels in the target region in the first image. The key pixels can be determined according to the actual scene, and this embodiment of the application does not impose any limitations on this. For example, the key pixels may be determined according to the actual scene, including pixels at the edges of the target region.

[0283] The following section will provide a detailed explanation of how electronic devices determine reference images.

[0284] In some embodiments, the reference image may include one or more images determined by the electronic device.

[0285] In one embodiment, the electronic device can determine a reference image based on the first image. That is, after acquiring the first image, the electronic device can determine one or more reference images based on the first image.

[0286] For example, an electronic device can identify a first image, determine the location corresponding to the first image (e.g., the location of the content contained in the first image in the actual environment), and determine one or more images containing that location as reference images based on the location of the first image. Alternatively, after determining one or more images containing that location, the electronic device can determine the similarity between the images containing that location and the first image, and determine one or more images from the images containing that location as reference images based on the similarity. Alternatively, after determining one or more images containing that location, the electronic device can determine the object contained in the images containing that location, and determine an image containing the second target object from the images containing that location as a reference image based on the object contained in the image containing that location and the second target object in the first image. Here, the image containing the second target object can refer to an image containing all of the second target object, or it can be an image containing a portion of the second target object.

[0287] For example, an electronic device can determine one or more candidate images based on the objects contained in a first image. Each candidate image may include one or more objects from the first image; that is, each candidate image may have one or more identical objects to the first image. Furthermore, the object included in each candidate image (e.g., object A) may be the complete object A or a part of object A. When a candidate image includes a part of object A, the electronic device can determine multiple candidate images containing object A, and these multiple candidate images containing object A can be merged to form the complete object A. After determining the candidate images, the electronic device can determine an image (i.e., a reference image) containing the second target object from the candidate images, based on the objects contained in each candidate image and the second target object in the first image.

[0288] For example, an electronic device can determine objects occluded by other objects (e.g., occluded objects) based on the relationships between objects in a first image, and can determine one or more candidate images based on the occluded objects. Each candidate image can include one or more occluded objects, and the occluded object (e.g., occluded object B) included in each candidate image can be the complete occluded object B or a part of the occluded object B. When a candidate image includes a part of the occluded object B, the electronic device can determine multiple candidate images of the occluded object B, and these multiple candidate images containing the occluded object B can be merged to form the complete occluded object B. After determining the candidate images, the electronic device can determine an image (i.e., a reference image) containing the second target object from the candidate images based on the objects contained in each candidate image and the second target object in the first image.

[0289] In another embodiment, the electronic device can directly determine the reference image based on the second target object in the first image. That is, after the user selects the first target object in the first image, the electronic device can determine the second target object in the first image, thereby identifying the object occluded by the first target object. Based on the second target object, one or more reference images containing the second target object can be determined. Directly determining the reference image based on the second target object can improve the speed and efficiency of reference image determination and improve the accuracy of the reference image, thereby enabling the rapid and accurate reconstruction of the second target object based on the reference image.

[0290] Each reference image may include a second target object. The second target object included in each reference image may be the complete second target object or a part of a second target object. When a reference image includes a part of a second target object, the electronic device can determine multiple reference images containing the second target object, and these multiple reference images containing the second target object can be merged to form the complete second target object.

[0291] It should be noted that the reference image can be an image determined by the electronic device from a gallery application, an image obtained by the electronic device from the network, or an image from one or more other applications of the electronic device (such as a media library), etc. That is, the electronic device can determine the reference image from one or more of the following: a gallery application, the network, or other applications of the electronic device (such as a media library). This application does not impose specific limitations on the method by which the electronic device determines the reference image; it can be determined according to the actual scenario.

[0292] In some embodiments, after obtaining the second image, the method may further include: S905, the electronic device performs smooth fusion of the second image. That is, after reconstructing the second target object, in order to ensure a natural transition of the image and improve the visual effect of the image, the electronic device may perform smooth fusion of the edges of the second target object.

[0293] It should be noted that the embodiments of this application do not limit the method by which the electronic device smoothly blends the edges of the second target object, and can be determined according to the actual scenario. For example, the electronic device can use a KNN classifier to smoothly blend the edges of the second target object. This smoothing process using a KNN classifier may include edge detection, smoothing, and edge blending.

[0294] Edge detection: After reconstructing the second target object, the electronic device can use edge detection algorithms (such as the Canny edge detector) to identify the edges of the second target object, and can extract key features of the edges based on algorithms such as SIFT (scale invariant feature transform). These features may include, but are not limited to: color features (such as RGB values), texture features (such as local binary patterns), shape features (such as the direction and length of the edge), etc.

[0295] Smoothing Processing: After extracting the features of each edge of the second target object, the electronic device can input these features into a trained KNN classifier for processing. The KNN classifier can determine whether the edges of the second target object are smooth based on the input edge features. When the KNN classifier determines that a certain edge of the second target object is not smooth, the electronic device can use local smoothing techniques (such as Gaussian blur or median filtering) to smooth that edge. Additionally, the electronic device can use morphological operations (such as dilation and erosion) to adjust the edges of the second target object, making them smoother and more natural. The KNN classifier is trained using training samples. Each training sample can be labeled as "smooth" or "non-smooth".

[0296] Edge blending: After smoothing the edges of the second target object, the electronic device can use image fusion techniques (such as Poisson fusion and / or multi-band fusion) to blend the smoothed edges to reduce the color difference between the reconstructed second target object and the first image, and ensure a natural transition of the edges of the second target object.

[0297] based on Figures 5 to 8 The user interface shown is Figure 11 This application illustrates a schematic flowchart of the image processing method provided in an embodiment. Figure 2 This method can be applied to the electronic devices described above. For example... Figure 11 As shown, the method may include:

[0298] S1101, The electronic device displays the first image and the corresponding editing button.

[0299] It should be noted that the first image can be any image selected by the user. That is, the user can select the image (e.g., the first image) that needs image processing on the electronic device. The electronic device can acquire the first image selected by the user and can display the first image. Image processing can include image removal, i.e., deleting objects from the image. It should be understood that the first image can be an image captured by the electronic device through its camera, or an image acquired by the electronic device from another device, or an image downloaded by the electronic device from the network, etc.

[0300] The following will be an exemplary description of the image processing method provided in the embodiments of this application, which is built into a gallery application.

[0301] like Figure 5 As shown in (a) or as in Figure 7 As shown in (a), the user can launch the gallery application and open the first image through the gallery application. That is, the electronic device can display the first image. When displaying the first image, the electronic device can also display buttons corresponding to functions such as editing, allowing the user to click the corresponding editing button to activate the deletion function and delete a certain object (e.g., the first target object) in the first image.

[0302] S1102. The electronic device detects a click operation on the button corresponding to the edit function and displays the button corresponding to the erase function and the button corresponding to the reference image.

[0303] like Figure 5 As shown in (a) or as in Figure 7 As shown in (a), when a user wants to delete the first target object in the first image, the user can click the corresponding edit button. Figure 5 As shown in (b) in the figure, or as shown in the figure ... Figure 7 As shown in (b), when the electronic device detects a click on the button corresponding to the edit function, it can display a button corresponding to the erase function and a button corresponding to the reference image. The user can click the button corresponding to the erase function to activate the erase function and delete the first target object in the first image. Additionally, the user can click the button corresponding to the reference image to select a reference image, allowing the electronic device to reconstruct objects occluded by the first target object (e.g., a second target object) based on the user-selected reference image.

[0304] It should be noted that when the user clicks the button corresponding to the reference image, that is, when a click operation on the button corresponding to the reference image is detected, the electronic device can execute S1103 to S1105, and S1109 to S1111. When the user clicks the button corresponding to the elimination function, that is, when a click operation on the button corresponding to the elimination function is detected, the electronic device can execute S1106 to S1108, and S1109 to S1111.

[0305] S1103. The electronic device detects a click operation on the button corresponding to the reference image and displays the first image library interface or the second image library interface.

[0306] When a user wants to input a reference image so that the electronic device can reconstruct a second target object after deleting a first target object, the user can click the button corresponding to the reference image. For example... Figure 5 (c) and Figure 5 As shown in (d), when a click operation on the button corresponding to the reference image is detected, the electronic device can display a first gallery interface (e.g., Figure 5 The gallery interface shown in (c) is 570), or the electronic device can display a second gallery interface (e.g., Figure 5 The image library interface shown in (d) is 580. That is, the first image library interface can be an image library interface without image classification, and the second image library interface can be an image library interface with image classification.

[0307] It should be noted that the specific method of image classification in this application embodiment is not limited and can be determined according to the actual scenario.

[0308] In some embodiments, the first gallery interface and the second gallery interface may include images native to the electronic device, that is, images stored on the electronic device. For example, they may include images from the electronic device's gallery application.

[0309] In other embodiments, the first and second gallery interfaces may include images native to the electronic device and images acquired by the electronic device from a network. For example, they may include images acquired by the electronic device from a network based on a first image, containing one or more objects from the first image. Alternatively, they may include images acquired by the electronic device from a network based on a second target object from the first image, containing all or part of the second target object.

[0310] In other embodiments, the first and second gallery interfaces may include images from other applications of the electronic device (e.g., a media library). For example, they may include images obtained by the electronic device from a media library based on a first image, containing one or more objects from the first image. Alternatively, they may include images obtained by the electronic device from a media library based on a second target object in the first image, containing all or part of the second target object.

[0311] It should be noted that the order in which images are displayed in the first and second image library interfaces is not limited in this embodiment and can be determined according to the actual scenario. For example, images can be displayed based on their acquisition time, such that images closer to the current time are displayed earlier and images farther away are displayed later. Alternatively, images can be displayed based on their content, such that images containing more objects from the first image are displayed earlier and images containing fewer objects from the first image are displayed later; or images containing more of the second target object are displayed earlier and images containing less of the second target object are displayed later. For example, images can also be displayed by combining their acquisition time and content, such that images closer to the current time and containing more objects from the first image are displayed earlier and images farther away and containing fewer objects from the first image are displayed later, and so on.

[0312] S1104. The electronic device detects a selection operation on a third image in the first or second image library interface and saves the third image as a reference image.

[0313] The third image may include one or more images displayed in the first image gallery interface, or it may include one or more images displayed in the second image gallery interface. For example, such as Figure 5 As shown in (c), the user can select one image from multiple images displayed in the first image library interface as a reference image. The electronic device can save the image selected by the user and designate it as the reference image.

[0314] S1105. After saving the third image as a reference image, the electronic device detects a click operation on the button corresponding to the elimination function, and detects a click operation or selection operation on the area corresponding to the first target object, and determines the first target object.

[0315] like Figure 6 As shown in (a), after completing the reference image (e.g.) Figure 5 After selecting image 57-1 as shown in (c), the electronic device can return to displaying the first image and may also display the button corresponding to the elimination function. At this time, the user can click the button corresponding to the elimination function to activate the elimination function and select the first target object to be deleted. For example, as... Figure 6 In (b), the user can select the area corresponding to the first target object. Or, Figure 6As shown in (c), the user can click on the area corresponding to the first target object. After detecting a click operation corresponding to the elimination function, the electronic device can activate the elimination function and determine the first target object based on the user's selection operation or click operation on the area corresponding to the first target object.

[0316] It should be noted that the above-described method of determining the first target object based on the user's selection or clicking operation of the area corresponding to the first target object is merely an illustrative explanation and should not be construed as a limitation on the embodiments of this application. In the embodiments of this application, the electronic device may also determine the first target object based on other user instructions or operations. The specific details of how the electronic device determines the first target object can be found in the aforementioned description in "The process of the electronic device determining the first target object based on the first user input will be described in detail below," and will not be repeated here.

[0317] S1106. The electronic device detects a click operation on the button corresponding to the elimination function, and detects a click operation or selection operation on the area corresponding to the first target object, and determines the first target object.

[0318] like Figure 7 (b) or Figure 7 As shown in (c), when a user wants to delete the first target object in the first image, the user can click the button corresponding to the deletion function, and after clicking the button, click or select the area corresponding to the first target object. After the electronic device detects the click or selection operation on the area corresponding to the first target object, it can determine the first target object that the user wants to delete. The specific content of the electronic device determining the first target object can be referred to the relevant description in the above "The process of the electronic device determining the first target object based on the first user input will be described in detail below", and will not be repeated here.

[0319] S1107. The electronic device detects a click operation on the button corresponding to the reference image and displays a first image library interface or a second image library interface.

[0320] like Figure 8 (a) or Figure 8 As shown in (b), when a user wants to input a reference image so that the electronic device can reconstruct the second target object after deleting the first target object based on the user-input reference image, the user can click the button corresponding to the reference image. When the electronic device detects the click operation on the button corresponding to the reference image, it can display a first image library interface (i.e., an image library interface without image classification) or a second image library interface (i.e., an image library interface with image classification), allowing the user to select the desired reference image through either the first or second image library interface.

[0321] The specific details regarding the first and second image library interfaces can be found in the descriptions of the first and second image library interfaces in the aforementioned S1103.

[0322] S1108. The electronic device detects a selection operation on a third image in the first or second image library interface and saves the third image as a reference image.

[0323] like Figure 8 As shown in (a), when the electronic device detects a selection operation on image 87-1 and a click operation on the corresponding save button 890, the electronic device can save image 87-1 and determine image 87-1 as the reference image selected by the user. Subsequently, the electronic device can reconstruct the second target object from the first image after deleting the first target object based on the reference image selected by the user (e.g., 87-1).

[0324] The third image may include one or more images displayed in the first image library interface, or it may include one or more images displayed in the second image library interface.

[0325] S1109. The electronic device deletes the first target object from the first image.

[0326] It should be noted that deleting the first target object in the first image can mean clearing the image data (e.g., pixel values) corresponding to the first target object. For example, after determining the first target object, i.e., after determining the area of ​​the first target object in the first image, the electronic device can set the pixel values ​​of each pixel point corresponding to the first target object (i.e., each pixel point within the area of ​​the first target object in the first image) to default values. The default values ​​can be determined according to the actual scenario, and this embodiment does not impose any limitations on this. For example, the default value can be determined to be 0 or 255 depending on the actual scenario.

[0327] S1110. The electronic device reconstructs the second target object in the first image based on the reference image to obtain the second image.

[0328] It should be understood that the second target object can be an object in the first image that is occluded by the first target object. A reference image can include the second target object. For example, a single reference image can contain the complete second target object, or multiple reference images can be merged to obtain the complete second target object.

[0329] After deleting the first target object from the first image, the electronic device can reconstruct the second target object from the first image based on a user-selected reference image, obtaining an image with the reconstructed second target object. Alternatively, the electronic device can determine a reference image based on the first image, and then reconstruct the second target object from the first image based on both the user-selected reference image and the reference image determined by the electronic device, obtaining an image with the reconstructed second target object. That is, even when the user selects a reference image, the electronic device can also automatically determine the reference image based on the first image, improving the accuracy of the reference image, thereby improving the accuracy of the reconstructed second target image, enhancing the visual effect of the image after deleting the first target object, and improving the user experience.

[0330] The specific content of the reference image determined by the electronic device can be referred to in the aforementioned "Detailed Explanation of the Method by which the Electronic Device Determines the Reference Image" section, and will not be repeated here. The specific content of the electronic device reconstructing the second target object in the first image based on the reference image can be referred to in the aforementioned S904 section, "The Electronic Device Reconstructs the Second Target Object in the First Image Based on the Reference Image to Obtain the Second Image," and will not be repeated here.

[0331] S1111, The electronic device performs smooth fusion of the second image.

[0332] After reconstructing the second target object, to ensure a natural transition of the object's edges in the second image and improve the image's visual effect, the electronic device can perform smooth fusion on the second image. The specific details of the electronic device performing smooth fusion on the second image can be found in the aforementioned description of S905 and the electronic device performing smooth fusion on the second image, and will not be repeated here.

[0333] It should be understood that the above-described selection of a reference image by the user before or after activating the elimination function is merely illustrative and should not be construed as a limitation on the embodiments of this application. The embodiments of this application do not impose a limitation on the time at which the user selects the reference image, and this can be determined according to the actual scenario.

[0334] The steps and order involved in S1101 to S1111 above are merely examples and should not be construed as limitations on the embodiments of this application. In the embodiments of this application, the order of the above steps can be adjusted, and not all steps are mandatory. In actual scenarios, some steps can be omitted as needed.

[0335] For example, in a real-world scenario, the content of displaying the edit button in S1101 can be omitted. That is, when displaying the first image, the electronic device can directly display the button corresponding to the elimination function and / or the button corresponding to the reference image. The user can directly activate the elimination function based on the button corresponding to the elimination function, or directly select the reference image based on the button corresponding to the reference image. It is not necessary for the user to click the button corresponding to the edit function before displaying the button corresponding to the elimination function and / or the button corresponding to the reference image.

[0336] For example, in a real-world scenario, the click operation on the button corresponding to the elimination function in S1105 can be omitted. That is, the user does not need to click the button corresponding to the elimination function, but can directly select the first target object to be deleted. In other words, the user can directly perform a click operation or a box selection operation on the area corresponding to the first target object to select the first target object. After the user selects the first target object, the electronic device can directly delete the first target object and perform subsequent operations.

[0337] It should be noted that the image processing method provided in this application embodiment, executed by an electronic device, is for illustrative purposes only and should not be construed as a limitation of the embodiments of this application. In this application embodiment, the image processing method provided in this application embodiment may also be executed collaboratively by other devices and electronic devices.

[0338] For example, the image processing method provided in this application embodiment can be executed collaboratively by a cloud server and an electronic device. For instance, when the electronic device displays a first image, if a first user input to delete a first target object in the first image is detected, the electronic device can send the first image to the cloud server. The cloud server can obtain the first image and determine a reference image corresponding to it. Furthermore, the cloud server can delete the first target object from the first image and reconstruct a second target object occluded by the first target object based on the determined reference image to obtain a second image. After obtaining the second image, the cloud server can send it to the electronic device.

[0339] Alternatively, when the electronic device displays the first image, if it detects user input to delete the first target object from the first image, the electronic device can send the first image to the cloud server. Additionally, when the user selects one or more reference images on the electronic device, the device can send these selected reference images to the cloud server. The cloud server can acquire the first image and the reference images. The cloud server can delete the first target object from the first image and reconstruct a second target object occluded by the first target object based on the reference images sent by the electronic device, obtaining a second image, which can then be sent to the electronic device. Furthermore, after acquiring the first image, the cloud server can determine the reference images based on them and reconstruct the second target object using both the reference images sent by the electronic device and the reference images determined by the cloud server. This improves the accuracy of the reference images and the accuracy of the reconstructed second target object, enhancing the visual effect of the image.

[0340] The cloud server determines the specific content of the reference image based on the first image. This can be referred to in the relevant description in the above-mentioned "The method of determining the reference image by electronic devices will be explained in detail below", and will not be repeated here.

[0341] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0342] Corresponding to the image processing method described in the above embodiments, this application also provides an image processing apparatus, the various modules of which can correspondingly implement the various steps of the image processing method.

[0343] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0344] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0345] This application also provides an electronic device, which includes at least one memory, at least one processor, and a computer program stored in the at least one memory and executable on the at least one processor. When the processor executes the computer program, it causes the electronic device to perform the steps in any of the above-described method embodiments. Exemplarily, the structure of the electronic device can be as follows: Figure 2 As shown.

[0346] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a computer, causes the computer to perform the steps in any of the above method embodiments.

[0347] This application provides a computer program product that, when run on an electronic device, causes the electronic device to perform the steps in any of the above-described method embodiments.

[0348] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate form. The computer-readable storage medium can include at least: any entity or device capable of carrying computer program code to a device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0349] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0350] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0351] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0352] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0353] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An image processing method, characterized in that, Applied to electronic devices, the method includes: The electronic device displays a first image, which includes a first target object and a second target object, wherein the second target object is occluded by the first target object in the first image; The electronic device acquires a first user input, which triggers the electronic device to delete a first target object in the first image. The electronic device deletes the first target object based on the first user input, and reconstructs the second target object in the first image based on the reference image to obtain a second image, wherein the reference image contains the second target object.

2. The method according to claim 1, characterized in that, The reference image includes an image selected by the user.

3. The method according to claim 1 or 2, characterized in that, The reference image includes an image determined by the electronic device based on the first image.

4. The method according to claim 3, characterized in that, The reference image includes an image determined by the electronic device based on the second target object in the first image.

5. The method according to any one of claims 2 to 4, characterized in that, The reference image includes at least one of an image from a stock photo library, an image obtained from the internet, or an image from a media library.

6. The method according to any one of claims 1 to 5, characterized in that, Before the electronic device deletes the first target object based on the first user input, the method further includes: The electronic device performs recognition processing on the first image to determine the first region corresponding to the first target object; The electronic device determines the first target object based on the first user input and the first region corresponding to the first target object.

7. The method according to claim 6, characterized in that, The first user input includes a first preset operation; The electronic device determines the first target object based on the first user input and the first region corresponding to the first target object, including: The electronic device determines the first position of the first preset operation in the first image; The electronic device determines the first target object based on the first location and the first region corresponding to the first target object.

8. The method according to claim 7, characterized in that, The first preset operation includes a click operation, a long press operation, or a double-click operation.

9. The method according to claim 6, characterized in that, The first user input includes a second preset operation, which includes a selection box operation; The electronic device determines the first target object based on the first user input and the first region corresponding to the first target object, including: The electronic device determines the second region selected by the second preset operation in the first image; The electronic device determines the first target object based on the second region and the first region corresponding to the first target object.

10. The method according to claim 6, characterized in that, The first user input includes a preset voice, which includes a first keyword, or the preset voice includes the first keyword and a second keyword; the first keyword is used to indicate a first target object to be deleted, and the second keyword is used to indicate a second position of the first target object to be deleted in the first image; The electronic device determines the first target object based on the first user input and the first region corresponding to the first target object, including: The electronic device determines the first target object based on the first keyword and the first region corresponding to the first target object; Alternatively, the electronic device determines the first target object based on the first keyword, the second keyword, and the first region corresponding to the first target object.

11. The method according to any one of claims 6 to 10, characterized in that, After the electronic device determines the first target object based on the first user input and the first region corresponding to the first target object, the method further includes: The electronic device displays the first area corresponding to the first target object; In response to an adjustment operation on a first region corresponding to the first target object, the electronic device adjusts the first region corresponding to the first target object.

12. The method according to any one of claims 1 to 11, characterized in that, The electronic device reconstructs the second target object in the first image based on the reference image to obtain a second image, including: The electronic device acquires image data corresponding to the second target object in the reference image; The electronic device reconstructs the second target object in the first image based on the image data to obtain the second image.

13. The method according to any one of claims 1 to 12, characterized in that, The electronic device is equipped with an image restoration model, which is trained based on the reference image. The electronic device reconstructs the second target object in the first image based on the reference image to obtain a second image, including: The electronic device reconstructs the second target object in the first image using the image restoration model to obtain a complete second target object; The electronic device fuses the complete second target object with the first image based on the reference image to obtain the second image.

14. The method according to any one of claims 1 to 13, characterized in that, After the electronic device reconstructs the second target object in the first image based on the reference image to obtain the second image, the method further includes: The electronic device performs a smooth fusion process on the second image to obtain a smoothed fused second image.

15. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the electronic device to implement the image processing method as described in any one of claims 1 to 14.

16. A computer program product, the computer program product comprising a computer program, characterized in that, When the computer program is run by the electronic device, the electronic device performs the image processing method as described in any one of claims 1 to 14.

17. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the computer, it causes the computer to implement the image processing method as described in any one of claims 1 to 14.

Images