Image processing method and related apparatus

Abstract

Embodiments of the present application provide an image processing method and related device, and relate to the technical field of terminals. The method can include: identifying a shooting object in a video recording process, identifying an irrelevant person or object when the irrelevant person or object blocks the shooting object, and eliminating the identified person or object when the recording is finished, to obtain a video without blocking of the shooting object. In this way, the shooting experience of a user can be improved. Alternatively, the method can include: identifying a shooting object in a video recording process, and eliminating an irrelevant person or object when the irrelevant person or object blocks the shooting object. In this way, the irrelevant person or object is eliminated in real time during the video recording process, and the shooting experience of a user can be improved.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an image processing method and related apparatus. Background Technology

[0002] With the development of terminal technology, electronic devices can support functions such as taking photos and recording videos.

[0003] Currently, during video recording, images captured by electronic devices may contain people or objects that obscure the subject, affecting the user's recording experience. Summary of the Invention

[0004] This application provides an image processing method and related apparatus, which are applied in the field of terminal technology. During the recording process, people or objects that obstruct the shooting object can be identified and removed from the recorded video, which helps to improve the user's shooting experience.

[0005] In a first aspect, embodiments of this application propose an image processing method applicable to an electronic device including a first camera. The method includes: in response to an operation to start recording, displaying a first interface, the first interface displaying a first image captured by the first camera, the first image including a first object; during a first duration of recording, a second object does not obscure the first object, the first image including the first object and a second object; during a second duration of recording, the second object partially obscures the first object, the first image including the second object and a portion of the first object not obscured by the second object, the second object being associated with a first identifier, the first identifier indicating the position of the second object, the start time of the second duration being later than the end time of the first duration; during a third duration of recording, the second object does not obscure the first object, the first image including the first object and the second object, the second object not being associated with the first identifier, the start time of the third duration being later than the end time of the second duration; and in response to an operation to stop recording, obtaining a first video; wherein, during the first duration of recording the first video, the first video includes the first object and the second object; during the second duration of recording the first video, the first video includes the first object but does not include the second object; and during the third duration of recording the first video, the first video includes both the first object and the second object.

[0006] The image processing method provided in this application embodiment allows for the association of a first identifier with the second object during video recording if a second object obscures the first object. If the second object no longer obscures the first object, the electronic device can remove the first identifier associated with the second object. This identification of the second object with the first identifier helps the user determine if the second object is a person or object unrelated to the recording. At the end of recording, the recorded video will not contain any image of the second object obscuring the first object. This eliminates the need for the user to manually remove the second object, reducing user operations and improving the user's recording experience.

[0007] In one possible implementation, when the second object partially obscures the first object, the method further includes: displaying a first prompt message and a first control, the first prompt message being used to prompt whether the second object should be identified; and in response to an operation that triggers the first control, associating the second object with a first identifier.

[0008] This approach, by prompting the user with a first message whether to annotate, and then annotating the second object only if the user agrees, helps reduce the probability of annotation errors.

[0009] In one possible implementation, in response to an operation to initiate recording, the method further includes: identifying a first object in a first image and associating the first object with a second identifier, the second identifier being used to identify the range in which the first object is located.

[0010] In this way, by using a second identifier to indicate to the user that the first object is the identified object that the user wants to photograph, the user's recording experience can be improved.

[0011] In one possible implementation, the first image further includes a third object; in response to an operation to initiate recording, the method further includes: identifying the first object and the third object in the first image, and associating the first object and the third object with a third identifier, the third identifier being used to identify the range in which the first object and the second object are located; in response to an operation to adjust the third identifier, obtaining a second identifier, the second identifier being used to identify the range in which the first object is located. This allows users to adjust the identifier to determine the object they want to capture, providing greater flexibility.

[0012] In one possible implementation, within the first time period, the method further includes: if the second object is in motion, calculating the first direction of motion and the first speed of the second object; determining the first probability that the second object occludes the first object based on the first direction of motion and the first speed; if the first probability is greater than the first preset probability, extracting the contour information of the second object; if the second object occludes part of the first object, the method further includes: associating the second object with a first identifier based on the contour information of the second object.

[0013] Within the first time period, the second object does not obscure the first object, but the second object is a moving object. The electronic device can determine the first probability that the second object obscures the first object. If the probability is large, that is, the first probability is greater than the first preset probability, the outline information of the second object is extracted so that the second object can be quickly associated with the first identifier when the second object partially obscures the first object, which is beneficial to improving the display rate of the first identifier.

[0014] In one possible implementation, the electronic device further includes a second camera with a shooting angle greater than that of the first camera; in response to an operation to initiate recording, the method further includes: acquiring a second image captured by the second camera, the second image including all objects in the first image, and the second image also including a fourth object; if the fourth object is in motion, calculating a second direction of motion and a second speed of motion of the fourth object; determining a second probability that the fourth object enters the first image based on the second direction of motion and the second speed of motion; if the second probability is greater than a second preset probability, extracting the contour information of the fourth object; and if the fourth object partially occludes the first object, associating a fourth identifier with the fourth object based on the contour information of the fourth object, the fourth identifier being used to identify the position of the fourth object.

[0015] If the fourth object is in motion and the second probability of the fourth object entering the first image is greater than the second preset probability, it indicates that the fourth object has a higher probability of occluding the first object. In this case, the electronic device can extract the contour information of the fourth object so that when the fourth object partially occludes the first object, it can quickly associate the fourth object with a fourth identifier based on the contour information of the fourth object, which helps to improve the display speed of the fourth identifier.

[0016] In one possible implementation, the method further includes: when the depth information of the second object is less than that of the first object, and the second object and the first object have partially overlapping areas in the first image, determining that the second object occludes a portion of the first object. This facilitates the identification of the portion of the first object occluded by the second object, enabling subsequent annotation and removal.

[0017] In one possible implementation, obtaining a first video in response to an operation to stop recording includes: obtaining a second video in response to the operation to stop recording, wherein during a second duration of recording the second video, the second video includes a portion of a first object not occluded by a second object and a second object, the second object being associated with a first identifier; and during the second duration of recording the second video, eliminating the second object associated with the first identifier and filling the eliminated pixels to obtain the first video.

[0018] In this way, eliminating the second object that obscures the first object helps to obtain an unobstructed video of the first object, which improves the user's recording experience.

[0019] Secondly, embodiments of this application propose an image processing method that can be applied to an electronic device including a first camera. The method includes: in response to an operation for recording video, displaying a first interface, the first interface displaying a first image captured by the first camera, the first image including a first object and a second object, the first object and the second object not obscuring each other; at a first moment, a third object does not obscure the first object and does not obscure the second object, the first image including the first object, the second object, and the third object; at a second moment, the third object does not obscure the first object and obscures part of the second object, the first image including the first object, the portion of the second object not obscured by the third object, and the third object; at a third moment, the third object obscures part of the first object but does not obscure the second object, the first image including the first object and the second object, but not including the third object; at a fourth moment, the third object obscures part of the first object and part of the second object, the first image including the portions of the first object and the second object not obscured by the third object, but not including the third object.

[0020] In this way, electronic devices can eliminate third objects that obscure the first object in real time, eliminating the need for users to manually remove third objects from the recorded video, which improves the user's recording experience.

[0021] In one possible implementation, when a third object partially obscures a first object, the method further includes: displaying a second prompt message and a second control, the second prompt message indicating whether to eliminate the third object; and eliminating the third object in response to triggering the operation of the second control, such that the first image does not include the third object.

[0022] This implementation method prompts the user with a second message to indicate whether to eliminate the object. If the user agrees to eliminate the object, the second object is then eliminated, which helps reduce the probability of elimination errors.

[0023] In one possible implementation, in response to an operation to initiate recording, the method further includes: identifying a first object in a first image and associating the first object with a second identifier, the second identifier being used to identify the range in which the first object is located.

[0024] In this way, by using a second identifier to indicate to the user that the first object is the identified object that the user wants to photograph, the user's recording experience can be improved.

[0025] In one possible implementation, in response to an operation to initiate recording, the method further includes: identifying a first object and a third object in a first image, and associating the first object and the third object with a third identifier, the third identifier being used to identify the range in which the first object and the second object are located; and in response to an operation to adjust the third identifier, obtaining a second identifier, the second identifier being used to identify the range in which the first object is located. This allows users to adjust the identifier to determine the object they want to capture, providing greater flexibility.

[0026] In one possible implementation, the method further includes: if the second object is in motion, calculating the first direction of motion and the first speed of the second object; determining the first probability that the second object occludes the first object based on the first direction of motion and the first speed; if the first probability is greater than the first preset probability, extracting the contour information of the second object; and, if the second object occludes part of the first object, eliminating the second object based on the contour information of the second object, so that the first image includes the first object but does not include the second object.

[0027] The second object does not obscure the first object, but the second object is a moving object. The electronic device can determine the first probability that the second object obscures the first object. If the probability is large, that is, the first probability is greater than the first preset probability, the outline information of the second object is extracted. This is so that when the second object partially obscures the first object, the first identifier can be quickly associated with the second object, which is beneficial to improving the display speed of the first identifier.

[0028] In one possible implementation, the electronic device further includes a second camera with a shooting angle greater than that of the first camera; in response to an operation to initiate recording, the method further includes: acquiring a second image captured by the second camera, the second image including all objects in the first image, and the second image also including a fourth object; if the fourth object is in motion, calculating a second direction of motion and a second speed of motion of the fourth object; determining a second probability that the fourth object enters the first image based on the second direction of motion and the second speed of motion; if the second probability is greater than a second preset probability, extracting the contour information of the fourth object; and, if the fourth object partially occludes the first object, eliminating the fourth object based on the contour information of the fourth object, so that the first image includes the first object but does not include the fourth object.

[0029] If the fourth object is in motion and the second probability of the fourth object entering the first image is greater than the second preset probability, it indicates that the fourth object has a higher probability of occluding the first object. In this case, the electronic device can extract the contour information of the fourth object so that when the fourth object partially occludes the first object, it can quickly associate the fourth object with a fourth identifier based on the contour information of the fourth object, which helps to improve the display speed of the fourth identifier.

[0030] In one possible implementation, the method further includes: determining that the third object occludes a portion of the first object when the depth information of the third object is less than that of the first object, and the third object and the first object have a partially overlapping area in the first image. This facilitates the identification of the portion of the first object occluded by the third object, making it easier to remove.

[0031] In one possible implementation, the method further includes: in response to an operation to eliminate a second object, eliminating the second object such that the first image does not include the second object. This allows the electronic device to support the user in eliminating any object in an image, providing greater flexibility and improving the user's recording experience.

[0032] Thirdly, embodiments of this application provide an image processing apparatus, which may be an electronic device, a chip, or a chip system within an electronic device. The image processing apparatus may include a display unit and a processing unit. When the image processing apparatus is an electronic device, the display unit may be a display screen. The display unit is used to perform display steps to cause the electronic device to implement an image processing method described in the first aspect or any possible implementation of the first aspect. When the image processing apparatus is an electronic device, the processing unit may be a processor. The image processing apparatus may further include a storage unit, which may be a memory. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to cause the electronic device to implement an image processing method described in the first aspect or any possible implementation of the first aspect. When the image processing apparatus is a chip or a chip system within an electronic device, the processing unit may be a processor. The processing unit executes the instructions stored in the storage unit to cause the electronic device to implement an image processing method described in the first aspect or any possible implementation of the first aspect. The storage unit can be a storage unit within the chip (e.g., a register, cache, etc.) or a storage unit located outside the chip within the electronic device (e.g., a read-only memory, random access memory, etc.).

[0033] For example, a display unit is configured to display a first interface in response to an operation to start recording. The first interface displays a first image captured by a first camera, the first image including a first object. During a first duration of recording, a second object does not obscure the first object, and the first image includes the first object and a second object. During a second duration of recording, the second object partially obscures the first object, the first image includes the second object and the portion of the first object not obscured by the second object, the second object is associated with a first identifier used to identify the position of the second object, and the start time of the second duration is later than the end time of the first duration. During a third duration of recording, the second object does not obscure the first object, the first image includes the first object and the second object, the second object is not associated with the first identifier, and the start time of the third duration is later than the end time of the second duration. A processing unit is configured to obtain a first video in response to an operation to stop recording. During the first duration of recording the first video, the first video includes the first object and the second object. During the second duration of recording the first video, the first video includes the first object but does not include the second object. During the third duration of recording the first video, the first video includes both the first object and the second object.

[0034] In one possible implementation, when the second object partially obscures the first object, the method further includes: displaying a first prompt message and a first control, the first prompt message being used to prompt whether the second object should be identified; and in response to an operation that triggers the first control, associating the second object with a first identifier.

[0035] In one possible implementation, in response to an operation to initiate recording, the method further includes: identifying a first object in a first image and associating the first object with a second identifier, the second identifier being used to identify the range in which the first object is located.

[0036] In one possible implementation, the first image further includes a third object; in response to an operation to initiate recording, the method further includes: identifying the first object and the third object in the first image, and associating the first object and the third object with a third identifier, the third identifier being used to identify the range in which the first object and the second object are located; in response to an operation to adjust the third identifier, obtaining a second identifier, the second identifier being used to identify the range in which the first object is located.

[0037] In one possible implementation, within the first time period, the method further includes: if the second object is in motion, calculating the first direction of motion and the first speed of the second object; determining the first probability that the second object occludes the first object based on the first direction of motion and the first speed; if the first probability is greater than the first preset probability, extracting the contour information of the second object; if the second object occludes part of the first object, the method further includes: associating the second object with a first identifier based on the contour information of the second object.

[0038] In one possible implementation, the electronic device further includes a second camera with a shooting angle greater than that of the first camera; in response to an operation to initiate recording, the method further includes: acquiring a second image captured by the second camera, the second image including all objects in the first image, and the second image also including a fourth object; if the fourth object is in motion, calculating a second direction of motion and a second speed of motion of the fourth object; determining a second probability that the fourth object enters the first image based on the second direction of motion and the second speed of motion; if the second probability is greater than a second preset probability, extracting the contour information of the fourth object; and if the fourth object partially occludes the first object, associating a fourth identifier with the fourth object based on the contour information of the fourth object, the fourth identifier being used to identify the position of the fourth object.

[0039] In one possible implementation, the method further includes: determining that the second object partially occludes the first object when the depth information of the second object is less than that of the first object and the second object and the first object have a partially overlapping area in the first image.

[0040] In one possible implementation, obtaining a first video in response to an operation to stop recording includes: obtaining a second video in response to the operation to stop recording, wherein during a second duration of recording the second video, the second video includes a portion of a first object not occluded by a second object and a second object, the second object being associated with a first identifier; and during the second duration of recording the second video, eliminating the second object associated with the first identifier and filling the eliminated pixels to obtain the first video.

[0041] For example, a display unit is configured to respond to a recording operation by displaying a first interface. The first interface displays a first image captured by a first camera. The first image includes a first object and a second object, which do not obstruct each other. At a first moment, a third object does not obstruct the first object or the second object, and the first image includes the first object, the second object, and the third object. At a second moment, the third object does not obstruct the first object but partially obstructs the second object, and the first image includes the first object, the portion of the second object not obstructed by the third object, and the third object. At a third moment, the third object partially obstructs the first object but does not obstruct the second object, and the first image includes the first object and the second object but does not include the third object. At a fourth moment, the third object partially obstructs the first object and partially obstructs the second object, and the first image includes the portions of the first and second objects not obstructed by the third object but does not include the third object.

[0042] In one possible implementation, the display unit is further configured to: display a second prompt message and a second control, the second prompt message being used to prompt whether to eliminate the third object; the processing unit is configured to: eliminate the third object in response to the operation of triggering the second control, such that the first image does not include the third object.

[0043] In one possible implementation, the processing unit is further configured to: identify a first object in the first image and associate a second identifier with the first object, the second identifier being used to identify the range in which the first object is located.

[0044] In one possible implementation, the processing unit is further configured to: identify a first object and a third object in the first image, and associate a third identifier with the first object and the third object, the third identifier being used to identify the range in which the first object and the second object are located; and in response to an operation for adjusting the third identifier, obtain a second identifier, the second identifier being used to identify the range in which the first object is located.

[0045] In one possible implementation, the processing unit is further configured to: if the second object is in motion, calculate the first direction of motion and the first speed of the second object; determine the first probability that the second object occludes the first object based on the first direction of motion and the first speed; if the first probability is greater than the first preset probability, extract the contour information of the second object; and if the second object occludes part of the first object, eliminate the second object based on the contour information of the second object, so that the first image includes the first object but does not include the second object.

[0046] In one possible implementation, the electronic device further includes a second camera with a shooting angle greater than that of the first camera; the processing unit is further configured to: acquire a second image captured by the second camera, the second image including all objects in the first image, and the second image also including a fourth object; if the fourth object is in motion, calculate a second direction of motion and a second speed of motion of the fourth object; determine a second probability that the fourth object enters the first image based on the second direction of motion and the second speed of motion; if the second probability is greater than a second preset probability, extract the contour information of the fourth object; if the fourth object partially occludes the first object, eliminate the fourth object based on the contour information of the fourth object, so that the first image includes the first object but does not include the fourth object.

[0047] In one possible implementation, the processing unit is further configured to: determine that the third object occludes a portion of the first object when the depth information of the third object is less than the depth information of the first object, and the third object and the first object have a partially overlapping area in the first image.

[0048] In one possible implementation, the processing unit is further configured to: eliminate the second object in response to the operation for eliminating the second object, such that the first image does not include the second object.

[0049] Fourthly, embodiments of this application provide an electronic device including a processor and a memory, the memory for storing code instructions, and the processor for executing the code instructions to perform the methods described in any aspect or any possible implementation of any aspect.

[0050] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program or instructions that, when executed on a computer, cause the computer to perform the methods described in any aspect or any possible implementation of any aspect.

[0051] Sixthly, embodiments of this application provide a computer program product including a computer program, which, when run on a computer, causes the computer to perform the methods described in any aspect or any possible implementation of any aspect.

[0052] Seventhly, this application provides a chip or chip system including at least one processor and a communication interface, wherein the communication interface and at least one processor are interconnected via a circuit, and the at least one processor is used to run computer programs or instructions to perform the methods described in any aspect or any possible implementation of any aspect. The communication interface in the chip can be an input / output interface, pins, or circuits, etc.

[0053] In one possible implementation, the chip or chip system described above in this application further includes at least one memory storing instructions. The memory can be an internal storage unit of the chip, such as a register or cache, or it can be a storage unit of the chip itself (e.g., read-only memory, random access memory, etc.).

[0054] It should be understood that the third to seventh aspects of this application correspond to the technical solutions of any aspect of this application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation are similar, and will not be repeated here. Attached Figure Description

[0055] Figure 1 This is a schematic diagram of a recording interface provided in an embodiment of this application;

[0056] Figure 2 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0057] Figure 3This is a schematic diagram of the hardware and software architecture of an electronic device provided in an embodiment of this application;

[0058] Figures 4 to 6 This is a schematic diagram of the interface of an electronic device entering recording mode according to an embodiment of this application;

[0059] Figure 7 and Figure 8 This is a schematic diagram of an interface for adjusting a selected indicator provided in an embodiment of this application;

[0060] Figure 9 and Figure 10 This is a schematic diagram of a character elimination interface provided in an embodiment of this application;

[0061] Figure 11 This is a schematic diagram of an interface for displaying prompt information provided in an embodiment of this application;

[0062] Figure 12 This is a schematic diagram of an interface for a character identifier provided in an embodiment of this application;

[0063] Figure 13 This is a schematic diagram of another interface for displaying prompt information provided in an embodiment of this application;

[0064] Figure 14 This is a flowchart illustrating the module interaction of an image processing method provided in an embodiment of this application.

[0065] Figure 15 This is a schematic diagram of a shooting scene provided in an embodiment of this application;

[0066] Figure 16 This is a flowchart illustrating the module interaction of another image processing method provided in this application embodiment;

[0067] Figure 17 This is a schematic diagram of another character elimination interface provided in an embodiment of this application;

[0068] Figure 18 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0069] Figure 19 This is a schematic diagram of the structure of a chip provided in an embodiment of this application. Detailed Implementation

[0070] To facilitate a clear description of the technical solutions in the embodiments of this application, some terms and technologies involved in the embodiments of this application will be briefly introduced below:

[0071] 1. 1x shooting mode

[0072] 1x shooting mode refers to shooting using the standard focal length. The field of view obtained in this mode is similar to that of the user's naked eye.

[0073] 2. Telephoto shooting mode

[0074] Telephoto shooting mode uses a lens with a longer focal length. Telephoto lenses have a narrower angle of view, which brings distant objects closer, making the subject stand out more in the frame.

[0075] This shooting method is suitable for capturing distant scenery or people, and it has a significant advantage, especially when shooting scenes such as wildlife and sporting events.

[0076] 3. Ultra-wide-angle shooting mode

[0077] Ultra-wide-angle shooting mode refers to shooting with an ultra-wide-angle lens. The field of view of an ultra-wide-angle lens is much greater than that of a standard lens, allowing you to capture a much wider scene.

[0078] This shooting style is suitable for landscape photography, large events, or occasions requiring a wide field of view. Ultra-wide-angle shooting can present a strong sense of perspective, making the picture more visually impactful.

[0079] 4. Other terms

[0080] In the embodiments of this application, terms such as "first" and "second" are used to distinguish identical or similar items with substantially the same function and purpose. For example, "first chip" and "second chip" are used only to distinguish different chips and do not limit their order of execution. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply that they are different.

[0081] It should be noted that, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0082] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, and c can be single or multiple.

[0083] Currently, during video recording, images captured by electronic devices may contain people or objects that obscure the subject, affecting the user's recording experience.

[0084] For example, Figure 1 A schematic diagram of a recording interface is shown. (For example...) Figure 1 As shown, the electronic device recorded for 00:06, and the recorded footage includes person 110, person 120, mountain range 130, and person 140. However, the footage the user wanted to record did not include person 140, as person 140 obstructed the view of people 110 and 120. Person 140 is marked with a black border to indicate that person 140 was the one obstructing the view.

[0085] If the shooting angle is fixed or the shooting range is limited, and it is impossible to eliminate the influence of person 140 on people 110 and 120 by changing the recording angle, it will affect the user's recording experience.

[0086] If users try to remove the influence of person 140 on people 110 and 120 using the removal tool after the video recording is finished, it will be time-consuming and laborious.

[0087] In view of this, embodiments of this application provide an image processing method and related apparatus that can eliminate people or objects obstructing the subject during video recording, reducing the impact of irrelevant people or objects on the subject and improving the recording experience. Alternatively, people or objects obstructing the subject can be identified during video recording and then removed from the recorded video, further improving the recording experience. It is understood that embodiments of this application refer to people or objects obstructing the subject as irrelevant people or objects.

[0088] The embodiments of this application can be applied to electronic devices that include recording functions. The electronic devices in the embodiments of this application may include handheld devices with cameras and vehicle-mounted devices that include cameras, etc. For example, some electronic devices include: mobile phones, tablets, PDAs, laptops, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, in-vehicle devices, wearable devices, terminal devices in 5G networks, or future evolution of public land mobile communication networks. Terminal devices in a network (PLMN), etc., are not limited to this in the embodiments of this application.

[0089] The electronic devices in the embodiments of this application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent, or user device, etc.

[0090] To facilitate understanding of the embodiments of this application, the hardware structure of the electronic device provided in the embodiments of this application will be described below.

[0091] Figure 2 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application is shown. For example... Figure 2As shown, the electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, and a display screen 194, etc.

[0092] Optionally, the aforementioned sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0093] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0094] Camera 193 may include one or more cameras, all of which can be used to capture images. In one example, camera 193 may include a main camera, a telephoto camera, and an ultra-wide-angle camera. The main camera can capture images in 1x shooting mode. The telephoto camera can capture images in telephoto mode. The ultra-wide-angle camera can capture images in ultra-wide-angle shooting mode.

[0095] The processor 110 can execute the methods provided in the embodiments of this application to eliminate irrelevant people or objects.

[0096] In one possible implementation, during video recording, the processor 110 can remove irrelevant people or objects in real time. When video recording ends, a video without irrelevant people or objects is obtained.

[0097] For example, in response to an operation for recording video, processor 110 can acquire an image captured by camera 193 and identify the object the user wants to photograph in the image. Processor 110 can detect whether there are irrelevant people or objects obstructing the object the user wants to photograph. If there are irrelevant people or objects obstructing the image, the processor 110 can remove the pixels of the irrelevant people or objects from the image and fill the removed area with background to obtain a new image, which is then displayed on display screen 194.

[0098] In this way, irrelevant people or objects that obstruct the shooting subject are eliminated in real time, and neither the displayed image nor the user's recorded image contains any irrelevant people or objects that obstruct the shooting subject, which helps to improve the user's recording experience.

[0099] In some examples, after identifying the object the user wants to photograph in the image, the processor 110 can also display a selection marker in the image to identify the object the user wants to photograph. The processor 110 detects the user's adjustment of the selection marker, redetermines the object the user wants to photograph, and detects whether the redetermined object is obstructed by irrelevant people or objects. If irrelevant people or objects are obstructing the image, the processor 110 can remove pixels of the irrelevant people or objects from the image, fill the removed area with background, obtain a new image, and display it on the display screen 194.

[0100] In response to an operation to end video recording, processor 110 may store the recorded video in internal memory 121. In response to an operation to view the recorded video, processor 110 may retrieve the video from internal memory 121 and display it on display screen 194.

[0101] It is understandable that the recorded video does not include any irrelevant people or objects that obstruct the subject of the recording. This eliminates the need to manually remove such objects, thus improving the user's recording experience.

[0102] In another possible implementation, during video recording, the processor 110 can annotate irrelevant people or objects in real time. When video recording ends, the processor 110 removes the annotated people or objects, obtaining the recorded video.

[0103] For example, in response to an operation for recording video, processor 110 can acquire an image captured by camera 193 and identify the object the user wants to photograph in the image. Processor 110 can detect whether there are any irrelevant people or objects obstructing the object the user wants to photograph. If there are irrelevant people or objects obstructing the image, the irrelevant people or objects can be marked in the image and the image marked with irrelevant people or objects can be displayed on display screen 194.

[0104] In this way, marking irrelevant people or objects can remind users that these irrelevant people or objects will not appear in the recorded video, eliminating the need for manual removal and improving the user's recording experience.

[0105] In some examples, after identifying the object the user wants to photograph in the image, the processor 110 can also display a selection marker in the image to identify the object the user wants to photograph. The processor 110 detects the user's adjustment of the selection marker, redetermines the object the user wants to photograph, and detects whether the redetermined object is obstructed by any irrelevant person or object. If an irrelevant person or object is obstructing the image, the processor 110 can mark the irrelevant person or object in the image and display the image with the marked irrelevant person or object on the display screen 194.

[0106] In response to an operation to end video recording, processor 110 can remove marked people or objects, obtain the recorded video, and store the recorded video in internal memory 121. In response to an operation to view the recorded video, processor 110 can retrieve the video from internal memory 121 and display it on display screen 194.

[0107] It is understandable that the recorded video does not include any irrelevant people or objects that obstruct the subject of the recording. This eliminates the need to manually remove such objects, thus improving the user's recording experience.

[0108] The software system of an electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. A layered architecture can use the Android system, the Apple iOS system, or other operating systems; this application embodiment does not limit this. The following uses a layered Android system as an example to exemplify the software architecture of the electronic device provided in this application embodiment.

[0109] Figure 3 This is a schematic diagram of the hardware and software architecture of an electronic device provided in an embodiment of this application. Figure 3 As shown, a layered architecture can divide the software system of an electronic device into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system can be divided into five layers, from top to bottom: applications, application framework, hardware abstraction layer (HAL), kernel, and hardware layer.

[0110] The application layer can include a series of application packages. The application layer runs applications by calling the application programming interface (API) provided by the application framework layer. For example... Figure 3As shown, the application package can include applications such as camera and gallery.

[0111] The application framework layer provides APIs and a programming framework for applications within the application layer. The application framework layer includes predefined functions. For example... Figure 3 As shown, the application framework layer may include a camera access interface and a view system. The camera access interface provides an application programming interface and programming framework for camera applications. The view system includes visual controls, such as controls for recording video, controls for ending video recording, and controls for viewing recorded video.

[0112] like Figure 3 As shown, the HAL layer may include a camera hardware abstraction layer and a camera algorithm library. The camera hardware abstraction layer can provide virtual hardware for the camera device. The camera algorithm library may include runtime code and data implementing the image processing methods provided in the embodiments of this application. In one example, the camera algorithm library may include a target recognition module, an irrelevant person / object intelligent recognition module, an artificial intelligence (AI) real-time removal module, an image fusion processing module, and a post-processing module. The post-processing module is optional.

[0113] The kernel layer is the layer between hardware and software. For example... Figure 3 As shown, this kernel layer may include one or more of the following: camera device driver, digital signal processor driver, and image processor driver. The camera device driver is used to drive the camera's sensor to acquire images. The digital signal processor driver is used to drive the digital signal processor to process images. The image processor driver is used to drive the graphics processor to process images.

[0114] The hardware layer may include hardware such as cameras, digital signal processors, and image processors.

[0115] It should be understood that in some embodiments, layers that perform the same function may be called by other names, or layers that can perform the functions of multiple layers may be considered as one layer, or layers that can perform the functions of multiple layers may be divided into multiple layers. This application does not impose any limitations on this.

[0116] The following is in conjunction with the above. Figure 3 The software structure shown below provides a detailed description of the image processing method in this application embodiment:

[0117] In response to a user's action of opening the camera application, such as clicking the camera application icon, the camera application calls the camera access interface in the application framework layer to launch the camera application, and then sends a command to start the camera by calling the camera hardware abstraction layer. The camera hardware abstraction layer can then send this command to the camera device driver in the kernel layer through the camera device. The camera device driver can then start the corresponding camera to capture images.

[0118] The camera can transmit captured images to the camera hardware abstraction layer via the camera device driver. The camera hardware abstraction layer can then transmit the images to the camera algorithm library.

[0119] In one possible implementation, in response to an operation for recording video, the camera algorithm library can acquire images captured by the camera, identify the main object the user wants to record in the image using a target recognition module, and detect whether any irrelevant people or objects are obstructing the main object being recorded using an irrelevant person / object intelligent recognition module. If irrelevant people or objects are present, the camera algorithm library can remove pixels of irrelevant people or objects from the image using an AI real-time removal module, and then fill the removed area with background using an image fusion processing module to obtain a new image. To reduce the probability of the new image being blurry or mismatches between the filled pixels and the original pixels, the camera algorithm library can process the new image using a post-processing module to improve the image quality.

[0120] The camera algorithm library can transmit images output by the image fusion processing module or the post-processing module to the camera hardware abstraction layer. The camera hardware abstraction layer can then display this image.

[0121] In this way, removing irrelevant people or objects during the recording process, compared to removing them after recording, helps improve the efficiency of generating the recorded video.

[0122] In another possible implementation, in response to an operation to record video, the camera algorithm library can acquire images captured by the camera, identify the main object the user wants to record in the image using a target recognition module, and detect whether any irrelevant people or objects are obstructing the main object being recorded using an irrelevant person / object intelligent recognition module. If irrelevant people or objects are obstructing the image, the irrelevant person / object intelligent recognition module can identify the irrelevant person or object and transmit the identified image to the camera hardware abstraction layer. The camera hardware abstraction layer can then display the image. In response to an operation to end video recording, the camera algorithm library can remove pixels of irrelevant people or objects from the video using an AI real-time removal module, and fill the removed areas with background using an image fusion processing module to obtain a new video.

[0123] To reduce the probability of new videos being blurry or having mismatches between filled pixels and pixels that were not filled, the camera algorithm library can process the new video through a post-processing module to improve the image quality in the video.

[0124] In this way, marking irrelevant people / or objects during the recording process and removing them at the end of the recording reduces the computational demands on electronic devices compared to removing irrelevant people / or objects during the recording process.

[0125] Optionally, before using the target recognition module to identify the main object the user wants to record in the image, in order to improve the accuracy of subsequent processing, the camera algorithm library can drive and control the digital signal processor to perform operations such as noise reduction and contrast enhancement on the image captured by the camera. The camera algorithm library can also perform image filtering, image segmentation, and image compression operations through the digital signal processor.

[0126] To better understand the embodiments of this application, the following describes the shooting scenarios provided by the embodiments of this application in detail with reference to a set of user interface diagrams.

[0127] It is understood that the terms "interface" and "user interface" in the specification, claims, and drawings of this application refer to the medium through which an application or operating system interacts and exchanges information with the user, realizing the conversion between the internal form of information and a form acceptable to the user. A commonly used form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0128] 1. Enter recording mode ( Figures 4 to 6 )

[0129] like Figure 4 As shown, the electronic device displays its main interface. The main interface may include multiple application icons, such as clock, calendar, gallery, notes, file manager, email, music, and calculator icons, etc. The main interface may also include a page indicator, which indicates the positional relationship between the currently displayed page and other pages. This page indicator is located below the aforementioned application icons on the main interface. Below this page indicator may also be multiple application icons, including camera (401), contacts, phone, and messaging icons, etc.

[0130] In response to an operation that triggers the camera application icon 401, the electronic device can launch the camera application and display... Figure 5 The interface shown. Figure 5 The interface shown is the default shooting interface of the camera application, where users can view the preview image and take the photo.

[0131] It should be noted that the operation used to trigger the camera application icon 401 in this application embodiment may include, but is not limited to, touch (e.g., click), voice control, gesture, etc., and this application embodiment does not limit this.

[0132] like Figure 5 As shown, the electronic device displays the shooting interface of the camera application's default shooting mode. The shooting interface displays a preview image, which can be an image captured by the electronic device's camera based on the field of view. The preview image includes a person 510, a person 520, a mountain range 530, and plants 540.

[0133] The shooting interface also displays album shortcut controls, shutter controls, and camera flip controls. The album shortcut controls can be used to launch the photo album application. After triggering the photo album application on the electronic device using the album shortcut controls, users can view the captured images and videos. Additionally, the album shortcut controls can also display thumbnails of the captured images or videos.

[0134] The shooting interface also displays one or more shooting mode options. These shooting mode options may include: aperture mode, night mode, portrait mode, still image mode, video mode, movie mode, and pro mode. It should be noted that the camera application may contain more or fewer shooting mode options; other shooting mode options may not be displayed due to limited interface space.

[0135] like Figure 5 As shown, the electronic device can display, in response to the operation of the recording mode option 501. Figure 6 The interface shown. Figure 6 The interface shown is the preview interface in recording mode.

[0136] Figure 6 The interface shown is Figure 5 The interfaces shown are largely the same, except that... Figure 6 The interface shown includes shooting mode options, including more options, but does not include aperture mode options. Figure 6 The interface shown includes a start recording control 601. The electronic device can begin recording in response to an operation performed on the start recording control 601.

[0137] 2. User adjusts the selected indicator ( Figure 7 and Figure 8 )

[0138] For example, Figure 7 A schematic diagram of an interface for a user to adjust a selected indicator is shown. The electronic device, in response to an operation on the start recording control 601, can display... Figure 7 The interface shown. (As shown) Figure 7 As shown, the recording duration is 00:01. The recorded footage includes people 510, people 520, mountains 530, and plants 540. The shooting mode is 1x shooting mode. The dotted line marker 701 in the image is the selection marker displayed on the electronic device. This selection marker 701 is used to identify the object the user wants to photograph.

[0139] The electronic device can display a message in response to the user's adjustment of the selected indicator 701. Figure 8 The interface shown. (As shown) Figure 8 As shown, the dotted line marking 702 in the image represents the adjusted mark 701. The user can adjust the selected mark 701 by dragging the dotted line. Alternatively, the user can manually draw the selected mark, and the electronic device will identify the object marked by the user's drawing as the object the user wants to photograph.

[0140] The user's operation of adjusting the selected identifier can also be other operations, and this application embodiment does not limit this.

[0141] 3. Elimination of irrelevant persons or objects ( Figure 9 and Figure 10 )

[0142] If an unrelated person or object obstructs the subject the user wants to film during recording, the electronic device can remove the unrelated person or object.

[0143] For example, if an unrelated person or object obstructs the object the user wants to film during video recording by an electronic device, the electronic device can remove that unrelated person or object.

[0144] For example, Figure 9 A schematic diagram of a recording interface is shown. For example... Figure 9 As shown, the object marked by the dashed selection mark 702 is the object the user wants to photograph. When the recording time is 00:04, a person 550 appears in the recording screen. At this time, the person 550 is not within the selection mark 702, that is, it does not obstruct the object the user wants to photograph, and the electronic device does not need to process the image.

[0145] Figure 10A schematic diagram of an interface for eliminating irrelevant people or objects is shown. For example... Figure 10 As shown, when the recording duration is 00:06, person 550 appears within the selected marker 702, meaning they are obscuring the object the user wants to photograph. The electronic device can then remove person 550. Figure 10 In the interface shown, character 550 is represented by a dashed line to indicate that it has been eliminated.

[0146] Optionally, to reduce the probability of mistakenly eliminating a person or object, the electronic device can output a prompt message if a person or object enters the selected area 702, i.e., obstructs the object the user wants to photograph. This prompt message asks the user whether to eliminate the identified person or object. In response to the user's confirmation, the identified person or object can be eliminated.

[0147] For example, Figure 11 A schematic diagram of an interface for displaying prompt information is shown. For example... Figure 11 As shown above, in the above Figure 10 In the example shown, person 550 appears within the selected marker 702, obscuring the object the user wants to photograph. The electronic device can output a prompt message asking whether to remove person 550, and display a "Yes" control for removing person 550 and a "No" control for not removing person 550. If the electronic device detects that the user has triggered the "Yes" control, the above will be displayed. Figure 10 The interface described above refers to removing character 550. If the electronic device detects that the user has triggered the "No" control, character 550 may not be processed.

[0148] 4. Identification of unrelated persons or objects

[0149] During recording, if an unrelated person or object obstructs the object the user wants to film, the electronic device can identify that unrelated person or object.

[0150] For example, Figure 12 A schematic diagram of an interface for identifying unrelated people or objects is shown. For example... Figure 12 As shown, the object marked by the dashed selection mark 702 is the object the user wants to photograph. When the recording time is 00:06, the person 550 appears within the selection mark 702, thus obscuring the object the user wants to photograph. The electronic device can use mark 1210 to mark the person 550.

[0151] Optionally, to reduce the probability of misidentifying a person or object, the electronic device can output a prompt message if a person or object enters the selected identifier 702, i.e., obstructs the object the user wants to photograph. This prompt message asks the user whether to identify the identified person or object. In response to the user's confirmation, the electronic device can identify the identified person or object.

[0152] For example, Figure 13 A schematic diagram of an interface for displaying prompt information is shown. For example... Figure 13 As shown above, in the above Figure 10 In the example shown, person 550 appears within the selected marker 702, obscuring the object the user wants to photograph. The electronic device can output a prompt message asking whether to mark person 550, and display a "Yes" control for marking person 550 and a "No" control for not marking person 550. If the electronic device detects that the user has triggered the "Yes" control, the above will be displayed. Figure 12 The interface shown uses identifier 1210 to identify person 550. If the electronic device detects that the user has triggered the "No" control, it may not process person 550.

[0153] It should be noted that, Figure 11 and Figure 13 The display format of the prompts and controls is merely an example, and this application does not limit it.

[0154] The above combination Figures 4 to 13 The interface display involved in the embodiments of this application has been described. The following will be combined with... Figure 14 and Figure 15 This application introduces an image processing method provided in its embodiments.

[0155] Electronic devices may be equipped with an AI erasure function. When recording video, if the AI ​​erasure function is enabled, the electronic device can execute the image processing method provided in this application embodiment. The AI ​​erasure function can be enabled or disabled in the settings application, or in the camera application; this application embodiment does not limit this.

[0156] As described above, with the AI ​​removal function enabled, this application embodiment includes two image processing methods. One image processing method is to remove irrelevant people or objects from the recorded screen in real time during the recording process. After recording ends, the recorded video contains no irrelevant people or objects. This application embodiment will combine... Figure 14 This method will be explained in detail.

[0157] Another image processing method is to mark irrelevant people or objects in the recorded frame during the recording process, and remove the marked people or objects when recording ends. After recording ends, the recorded video contains no irrelevant people or objects. This application embodiment will combine... Figure 15 This method will be explained in detail.

[0158] For example, Figure 14This diagram illustrates the module interaction flowchart of an image processing method provided in an embodiment of this application. The method can be executed by a software module or a hardware module in an electronic device. The hardware architecture of the electronic device can be as described above. Figure 2 As shown, the hardware and software architecture of electronic devices can be as described above. Figure 3 As shown, the embodiments of this application are not limited thereto.

[0159] like Figure 14 As shown, the method may include the following steps:

[0160] S1401, The camera application detected an operation to start recording.

[0161] In the above Figure 6 In the interface shown, when the user triggers the start recording control 601, the camera application 401 can detect the operation used to start recording.

[0162] S1402, The camera application responds to the operation used to start recording and determines whether the AI ​​erasure function is turned on.

[0163] If the AI ​​removal feature is enabled, the camera app can proceed with the subsequent steps. If the AI ​​removal feature is disabled, the camera app will not proceed with the subsequent steps; that is, it will not remove irrelevant people or objects from the recorded footage, nor will it save videos without irrelevant people or objects.

[0164] S1403. If the AI ​​elimination function is enabled, the camera application can transmit information that the AI ​​elimination function is enabled to the target recognition module.

[0165] In the above Figure 3 In the example shown, the information indicating that AI removal is enabled can be transmitted from the application layer through the application framework layer to the camera algorithm library in the hardware abstraction layer. This allows the target recognition module to obtain the information that AI removal is enabled.

[0166] S1404 The target recognition module can obtain image A captured by the camera based on the information of the AI ​​elimination function being turned on. Image A can be the first frame image captured by the camera after recording is started. The target object i is identified in image A and circled with the selection mark 1.

[0167] In the above Figure 7 In the interface shown, the subject i can include people 510, people 520, mountains 530, and plants 540. The selected identifier 1 can be an identifier 701 formed by dashed lines. The selected identifier 1 can be any polygon, primarily including the identified subject i.

[0168] In one example, the target recognition module may include an object recognition model. The target recognition module can use the object recognition model to identify the captured object i in image A.

[0169] The training process of this object recognition model may include: inputting images of various scenes into the source model, which then identifies the objects being photographed. A loss function is calculated based on the identified objects and ground truth (or labels). When the loss function converges, the object recognition model is obtained. The ground truth can be objects manually identified by the user in the images of each scene.

[0170] Optionally, image A can be an image that has undergone one or more of the following processing steps: noise reduction, sharpening, color correction, or white balance adjustment. This helps improve the accuracy of identifying the subject i.

[0171] S1405 The target recognition module can transmit image B to the display screen for display. Image B includes the selected identifier 1 and image A.

[0172] The target recognition module can transmit the selected identifier 1 and image A to the display screen. Marking image A with the selected identifier 1 results in image B, which is merely an example. Therefore, transmitting the selected identifier 1 and image A together helps reduce the probability that image A and the selected identifier 1 will not be displayed simultaneously.

[0173] In another example, while the electronic device transmits image A to the display screen, the target recognition module can simultaneously transmit selected identifier 1 to the display screen. This parallel execution improves display efficiency.

[0174] S1406. In response to the user's operation of adjusting the selected marker 1, the target recognition module can redetermine the shooting object j. The shooting object j is the shooting object circled by the adjusted selected marker 1.

[0175] If the user adjusts the selection marker 1, it indicates a difference between the intelligently identified shooting object i and the object the user wants to photograph. The user adjusts the selection marker 1 so that the adjusted selection marker 1 can pinpoint the object the user wants to photograph. Therefore, in response to the user's operation of adjusting the selection marker 1, the target recognition module can redetermine the shooting object j based on the adjusted selection marker 1. Here, the shooting object j is the shooting object pinpointed by the adjusted selection marker 1.

[0176] In this way, the electronic device allows the user to adjust the selection marker 1 so that the adjusted selection marker 1 can select the object that the user wants to photograph, thus meeting the user's needs and improving the user experience.

[0177] For example, in the above Figure 7 and Figure 8 In the example shown, Figure 7In the interface shown, the dotted line forming marker 701 can represent the selected marker 1, and the subject i can include all objects within the selected marker 1. The electronic device responds to the user's adjustment of the selected marker 1 by displaying... Figure 8 The interface shown. (As shown) Figure 8 As shown, the adjusted selection marker 1 is marker 702 formed by dashed lines. The subject circled by the adjusted selection marker 1 can be subject j. Subject j can be any object within the adjusted selection marker 1.

[0178] S1407 The target recognition module acquires the image C captured by the camera, identifies the shooting object j from the image C, and circles the shooting object j with the selection mark 2.

[0179] Image C can be an image captured by the camera after image A is captured. Image C and image A can be adjacent to each other.

[0180] For example, image C can be the above. Figure 9 The image shown.

[0181] This step can be referred to in S1404 above, and will not be repeated here.

[0182] S1408, The target recognition module will transmit the selected identifier 2 to the unrelated person / object intelligent recognition module.

[0183] The target recognition module can transmit the location information of the selected identifier 2 in image C to the unrelated person / object intelligent recognition module.

[0184] S1409, The unrelated person / object intelligent recognition module determines whether there is an object k unrelated to the subject j within the range circled by the selected marker 2.

[0185] Object k is an object that obstructs the shooting of object j, thus affecting the shooting of object j. Therefore, the irrelevant person / object intelligent recognition module determines whether there is an object k unrelated to the shooting object j within the area circled by the selected marker 2. This indicates that the irrelevant person / object intelligent recognition module determines whether there is an object obstructing the shooting object j.

[0186] The unrelated person / object intelligent recognition module determines whether there is an object k unrelated to the subject j within the area circled by the selected marker 2, which may include one or more of the following methods.

[0187] In the first method, the unrelated person / object intelligent recognition module determines whether there is an object k unrelated to the subject j within the area circled by the selected marker 2 based on the depth information of the pixels.

[0188] For example, the unrelated person / object intelligent recognition module can identify objects other than the subject j being photographed. If the depth information of an object other than the subject j is less than the depth information of the subject j, and there is an overlapping area, then it can be determined that there is an object k unrelated to the subject j. Otherwise, the unrelated person / object intelligent recognition module can determine that there is no object k unrelated to the subject j.

[0189] For example, in the above Figure 1 In the example shown, the subject j is the stage, and the subject k is person 520. The depth information of person 520 is less than that of the stage, and there is an overlapping area. The irrelevant person / object intelligent recognition module can determine that person 520 is an irrelevant person or object.

[0190] This implementation method is simple and helps improve recognition efficiency.

[0191] In the second method, the unrelated person / object intelligent recognition module can determine whether there is an object k unrelated to the subject j within the area circled by the selected marker 2 based on the contour information.

[0192] For example, the unrelated person / object intelligent recognition module can identify objects other than the shooting object j. If the outline of an object other than the shooting object j covers part or all of the selected marker 2, it can be determined that there is an object k unrelated to the shooting object j within the range circled by the selected marker 2. Otherwise, the unrelated person / object intelligent recognition module can determine that there is no object k unrelated to the shooting object j.

[0193] The third method involves the electronic device simultaneously recording video in 1x shooting mode (excluding ultra-wide-angle mode) or telephoto shooting mode. This allows the device to capture more information from the ultra-wide-angle video. It's important to note that while both 1x and ultra-wide-angle modes are being used simultaneously, or telephoto and ultra-wide-angle modes are being used concurrently, the electronic device will only display the footage captured in 1x or telephoto mode, not the footage captured in ultra-wide-angle mode.

[0194] The irrelevant person / object intelligent recognition module can acquire video captured in ultra-wide-angle mode, with the last frame of the video being image C. This module can detect whether there are moving people or objects in the video besides the subject j. If moving people or objects are present, their direction and speed can be calculated. If there is a high probability that the moving person / object will enter the image captured in 1x or telephoto shooting mode, its contour information is extracted, and based on this contour information, it is determined whether the moving person / object appears within the area circled by the selected marker 2.

[0195] It is understandable that if a moving person / object appears within the area circled by the selected marker 2, it indicates the existence of an object k that is unrelated to the subject j being photographed.

[0196] For example, Figure 15 A schematic diagram of a shooting scene is shown. For example... Figure 15 As shown, the shooting scene includes person 510, person 520, mountain range 530, and person 550. Person 550 is moving, moving in the direction indicated by the arrow. The electronic device can shoot in 1x shooting mode. Figure 15 The scene shown.

[0197] The shooting range in 1x mode is shown as 1501, and the shooting range in ultra-wide-angle mode is shown as 1502. Videos shot in ultra-wide-angle mode include more information than those shot in 1x mode. During video shooting in 1x mode, the electronic device can obtain both the video shot in 1x mode and the video shot in ultra-wide-angle mode.

[0198] When shooting video, the selected area can be as shown in 1503. During the video shooting process, the irrelevant person / object intelligent recognition module can determine that person 550 is moving and has a high probability of entering the shooting range of 1x shooting mode through the video shot in ultra-wide-angle mode. Then, it extracts the outline information of person 550 and determines whether person 550 is within the range circled by the selected indicator based on the outline information of person 550.

[0199] S1410. If there is an object k that is unrelated to the subject j, the unrelated person / object intelligent recognition module can transmit the area where object k is located to the AI ​​real-time elimination module.

[0200] In some examples, if there is an object k that is unrelated to the subject j, the unrelated person / object intelligent recognition module can identify the region where object k is located through image segmentation and transmit the region where object k is located to the AI ​​real-time elimination module.

[0201] S1411, the AI ​​real-time removal module can remove pixels from the region where object k is located in image C to obtain image D.

[0202] The AI ​​real-time removal module removes pixels from the region containing object k in image C, indicating that object k has been removed from image C.

[0203] S1412, the AI ​​real-time elimination module can send image D to the image fusion processing module.

[0204] S1413 The image fusion processing module can fill in the pixels removed from image D to obtain image E.

[0205] In some examples, the image fusion processing module can acquire images captured by the camera before acquiring image C, filter out images similar to image C from these images, determine the background region where object k is located from these similar images, and fill the image D with the pixels in the background region to obtain image E.

[0206] S1414 The image fusion processing module can transmit image E to the display screen for display.

[0207] The image processing method provided in this application can identify the subject being filmed and eliminate objects unrelated to the subject during video recording, thereby improving the user's shooting experience. Furthermore, it supports user adjustment of the subject, offering greater flexibility.

[0208] In the image processing method described above, irrelevant people and objects are not displayed during video recording. After recording ends, irrelevant people and objects will not appear in the recorded video. This results in an unobstructed and interference-free video focused on the object the user wants to capture. The object the user wants to capture can also be referred to as the target subject.

[0209] Optionally, in the above Figure 14 In the method shown, S1406 can be optional, meaning the user does not need to adjust the selected marker 1, and the target recognition module does not need to redetermine the subject. In this implementation, the target recognition module can transmit the selected marker 1 to the irrelevant person / object intelligent recognition module. The irrelevant person / object intelligent recognition module determines whether there is an object unrelated to the subject i within the area circled by the selected marker 1. If so, the AI ​​real-time elimination module can eliminate the object unrelated to the subject i.

[0210] Optionally, in the above Figure 14 In the method shown, S1407 can be optional, in which case the target recognition module can respond to the operation of adjusting the selected marker 1 to obtain the selected marker 2, and the irrelevant person / object intelligent recognition module determines whether there is an object k unrelated to the subject j within the area circled by the selected marker 2 in image B. If there is an object k unrelated to the subject j, the irrelevant person / object intelligent recognition module can transmit the area where the object k is located to the AI ​​real-time removal module. The AI ​​real-time removal module can remove the pixels in the area where the object k is located from image B to obtain image D. The image fusion processing module can fill the removed pixels in image D to obtain image E, and transmit image E to the display screen for display.

[0211] Optionally, before transmitting image E to the display screen, the image fusion processing module can transmit image E to the post-refinement processing module. The post-refinement processing module can perform a smoothing process on image E to obtain a smoothed image, making the area of ​​the filled pixels in the fused image E more consistent with the area before the filled pixels.

[0212] For example, the post-refinement module can analyze the colors in image E using methods such as mean, mode, or color distribution to ensure that the color of the area to be filled matches the color of the area preceding the filled pixel. The post-refinement module can also analyze the texture features of image E to ensure that the texture of the area to be filled matches the texture of the area preceding the filled pixel. The post-refinement module can also perform edge detection on image E to ensure that the edges of the area to be filled match the edges of the area preceding the filled pixel.

[0213] In this way, the fused image E is more refined, realistic, and seamless, which helps to improve image quality.

[0214] In the method described above, the AI ​​real-time removal module and the image fusion processing model can be a single module. In this case, the module may include a machine learning model that can be used to remove irrelevant people or objects without affecting the display of the subject being photographed. The irrelevant people or objects can be object k, and the subject being photographed can be object j.

[0215] In one example, this model could be a Generative Adversarial Network (GAN), which includes a generator and a discriminator. The generator produces new images, while the discriminator determines whether the generated images are realistic. Through continuous adversarial training between the generator and the discriminator, the generator can learn how to generate realistic images, thereby achieving the effect of removing irrelevant people or objects.

[0216] Alternatively, the above model may also include the functionality of a post-refinement module, which simplifies implementation.

[0217] The image processing method described above allows the electronic device to remove irrelevant people or objects that obstruct the user's intended subject during video recording. The following embodiments of this application will introduce another image processing method.

[0218] For example, Figure 16 This diagram illustrates the module interaction flowchart of an image processing method provided in an embodiment of this application. The method can be executed by a software module or a hardware module in an electronic device. The hardware architecture of the electronic device can be as described above. Figure 2As shown, the hardware and software architecture of electronic devices can be as described above. Figure 3 As shown, the embodiments of this application are not limited thereto.

[0219] like Figure 16 As shown, the method may include S1401 to S1409 as described above. After executing S1409, the method may further include the following steps:

[0220] S1601. If there is an object k that is unrelated to the subject j, the unrelated person / object intelligent recognition module can use the selected identifier 3 to identify the object k.

[0221] For example, in the above Figure 12 In the interface shown, object k, which is unrelated to the subject j, can be person 550, and selecting identifier 3 can be identifier 1210.

[0222] The unrelated person / object intelligent recognition module uses the selected identifier 3 to identify object k without eliminating it, which helps save computing power and reduces the probability of stuttering during video recording.

[0223] S1602, the unrelated person / object intelligent recognition module can transmit image F to the display screen for display. Image F includes selection marker 3 and image C.

[0224] The target recognition module can transmit the selected identifier 3 and image C to the display screen. The image F obtained after identifying image C with the selected identifier 3 is just one example. Therefore, transmitting the selected identifier 3 and image C together helps reduce the probability that image C and the selected identifier 3 will not be displayed simultaneously.

[0225] In another example, while the electronic device transmits image C to the display screen, the target recognition module can simultaneously transmit the selected identifier 3 to the display screen. This parallel execution improves display efficiency.

[0226] During recording, the target recognition module and the irrelevant person / object intelligent recognition module can repeatedly execute the above steps. Specifically, the target recognition module can identify the subject j in the image captured by the camera and circle it with selection marker 2. The target recognition module can also determine whether there is an object k unrelated to the subject j within the range of selection marker 2. If it exists, it uses selection marker 3 to mark object k and displays selection marker 3 along with the image captured by the camera. If it does not exist, the image captured by the camera is displayed.

[0227] S1603, The camera application detected an operation to end recording.

[0228] In the above Figures 7 to 13In any of the interfaces shown, the control to end recording is a black square icon. When the user triggers this icon, the camera application can detect the action to end recording.

[0229] S1604. In response to an operation to end recording, the camera application can transmit information indicating the end of recording to the unrelated person / object intelligent recognition module.

[0230] In the above Figure 3 In the example shown, the information indicating the end of recording can be transmitted from the application layer through the application framework layer to the camera algorithm library in the hardware abstraction layer.

[0231] S1605, the unrelated person / object intelligent recognition module acquires the recorded video 1 and determines whether there is an object m unrelated to the subject being filmed in the video 1.

[0232] Object m can include object k. It should be noted that during the recording process, objects other than object k may obstruct the view of the subject, which are not all shown.

[0233] In one example, the unrelated person / object intelligent recognition module can determine whether there is an identified object in video 1. If it exists, then there is an object m that is unrelated to the subject being filmed; if it does not exist, then there is no object m that is unrelated to the subject being filmed.

[0234] For example, if the unrelated person / object intelligent recognition module detects that there is an object k identified by selected identifier 3 in video 1, then there is an object m that is unrelated to the subject being filmed.

[0235] In some examples, the object m unrelated to the subject being filmed can also include objects other than the subject that enter and exit the frame. For example, in video 1, there is an object other than the subject that appears discontinuously; this object could be an object m unrelated to the subject being filmed.

[0236] S1606. If there is an object m that is unrelated to the subject being photographed, the unrelated person / object intelligent recognition module can transmit the location of object m to the AI ​​real-time elimination module.

[0237] The location of object m can include the video frame in which object m is located and the region within that video frame.

[0238] S1607, the AI ​​real-time removal module can remove the pixels at the location of object m from video 1 to obtain video 2.

[0239] The pixels at the location of the object m to be eliminated by the AI ​​real-time elimination module can be referenced in S1411 above, and will not be repeated here.

[0240] S1608, the AI ​​real-time elimination module can send video 2 to the image fusion processing module.

[0241] S1609 The image fusion processing module can fill in the pixels removed from video 2 to obtain video 3.

[0242] The image fusion processing module can perform image fusion on video frames with removed pixels in video 2, video frames before that video frame without removed pixels, and video frames after that video frame without removed pixels, thereby filling in the removed pixels in video 2 to obtain video 3.

[0243] Video 3 does not contain any object m that is unrelated to the subject being filmed. This allows us to identify objects unrelated to the subject during video recording and remove them when recording ends, thus improving the user's shooting experience.

[0244] Optionally, after obtaining video 3, the image fusion processing module can transmit video 3 to the post-processing module. The post-processing module can perform a smoothing process on video 3 to obtain a smoothed video 3, making the area where the filled pixels are located in the fused video 3 more consistent with the area before the filled pixels. In this way, the fused video 3 is more refined, realistic, and seamless, which helps to improve image quality.

[0245] In addition to the above Figure 14 and Figure 16 In addition to the image processing method shown, embodiments of this application may also provide an image processing method in which, during video recording, in response to a user's selection of a person or object, the electronic device may also remove the selected person or object. It is understood that user selection indicates that the user wants to remove the selected object. In some examples, the user may manually circle the object to be removed; embodiments of this application do not limit this to this.

[0246] For example, during video recording, in response to a user selecting an object, the electronic device can eliminate the selected object. Figure 17 A schematic diagram of an interface for eliminating irrelevant people or objects is shown. For example... Figure 17 As shown, the recorded screen includes a person 510, a person 520, a mountain range 530, and a plant 540. If the electronic device detects that the user has selected the plant 540, the electronic device can remove the plant 540 in response to the user's selection of the plant 540.

[0247] In this way, removing the people or things that the user wants to eliminate can improve the user's shooting experience.

[0248] Based on the methods provided in the embodiments of this application, the following two scenarios can be constructed. These two scenarios can be deployed in the same electronic device or in different electronic devices, and the embodiments of this application do not limit this.

[0249] The following is combined with Figure 18 The first scenario constructed according to the embodiments of this application is introduced.

[0250] For example, Figure 18 A schematic flowchart illustrating a communication method provided in an embodiment of this application is shown. This method can be executed by an electronic device including a camera. Figure 18 As shown, the method may include the following steps:

[0251] S1801. In response to an operation to start recording, a first interface is displayed, the first interface displaying a first image captured by a first camera, the first image including a first object.

[0252] The operation for initiating recording can also be referred to as the operation for starting recording, the operation for starting video recording, or the operation for starting video recording; this application embodiment does not limit this. The operation for initiating recording can be a click operation, a touch operation, or a long press operation, etc.; this application embodiment does not limit this.

[0253] The first interface is the interface displayed on the electronic device, which may include one or more controls in addition to the first image.

[0254] The first camera can be a standard camera, a telephoto camera, a wide-angle camera, an ultra-wide-angle camera, or an ultra-telephoto camera, etc., and this application embodiment does not limit it.

[0255] The first camera can capture a raw image. The first image can be a raw image that has undergone one or more of the following processing steps: noise reduction, sharpening, color correction, or white balance adjustment. This application embodiment does not limit this.

[0256] The first object can be one or a combination of people, plants, animals, objects, or landscapes, and the embodiments of this application do not limit this.

[0257] In some examples, the first interface can be as described above. Figure 7 As shown, it includes images captured by the camera, as well as controls for stopping or pausing recording. The first camera can be a standard or ordinary camera, and can support 1x shooting mode. The first object can include a person 510, a person 520, a mountain 530, and a plant 540.

[0258] S1802. During the first duration of the recording process, the second object does not obscure the first object, and the first image includes the first object and the second object.

[0259] The starting time of the first duration can be the moment recording begins, or it can be any time after recording begins; this application embodiment does not limit this. The first duration can be 2 seconds, 2 minutes, or 300 milliseconds, etc.; this application embodiment does not limit this.

[0260] If the second object does not occlude the first object within the first time period, it can be concluded that the first object and the second object do not overlap in the first image. Therefore, the first image includes the first object and the second object.

[0261] In some examples, the second object may refer to the above. Figure 9 The first object, represented by figure 550, may include all objects defined by identifier 702. The second object does not obscure the first object, and the first image includes both the second and first objects.

[0262] S1803. During the second duration of the recording process, the second object partially occludes the first object. The first image includes the second object and the portion of the first object that is not occluded by the second object. The second object is associated with a first identifier, which is used to identify the position of the second object. The start time of the second duration is later than the end time of the first duration.

[0263] The second duration can be the same as or different from the first duration; this application does not limit this. If the second object occludes the first object, it indicates that the first object and the second object overlap in the first image. Therefore, the first image includes the second object and the portion of the first object not occluded by the second object.

[0264] The shape of the first identifier can be regular or irregular, and this application embodiment does not limit this. The size of the first identifier is also not limited in this application embodiment.

[0265] In some examples, the second object may refer to the above. Figure 12 In the image, character 550, the first object can include all objects defined by identifier 702. A second object occludes the first object; the first image includes the second object and the portion of the first object not occluded by the second object. The first identifier can be referenced... Figure 12 The identifier 1210 in the text.

[0266] S1804. During the third duration of the recording process, the second object does not obscure the first object, the first image includes the first object and the second object, the second object is not associated with the first identifier, and the start time of the third duration is later than the end time of the second duration.

[0267] The third duration, the second duration, and the first duration can have the same duration or different durations; this application embodiment does not limit this. If the second object does not obscure the first object, then the first identifier associated with the second object disappears.

[0268] S1805. In response to an operation to stop recording, a first video is obtained; wherein, during a first duration of recording the first video, the first video includes a first object and a second object; during a second duration of recording the first video, the first video includes the first object but does not include the second object; during a third duration of recording the first video, the first video includes the first object and the second object.

[0269] The operation used to stop recording can also be called the operation to end recording; however, this application's embodiments do not limit this. In the above... Figures 7 to 13 In any of the interfaces shown, the control to stop recording can be a black square icon. The operation to stop recording can be a click, a touch, or a long press, etc., and this application embodiment does not limit this.

[0270] The image processing method provided in this application embodiment allows for the association of a first identifier with the second object during video recording if a second object obscures the first object. If the second object no longer obscures the first object, the electronic device can remove the first identifier associated with the second object. This identification of the second object with the first identifier helps the user determine if the second object is a person or object unrelated to the recording. At the end of recording, the recorded video will not contain any image of the second object obscuring the first object. This eliminates the need for the user to manually remove the second object, reducing user operations and improving the user's recording experience.

[0271] Optionally, when the second object partially obscures the first object, the above method further includes: displaying a first prompt message and a first control, wherein the first prompt message is used to prompt whether the second object is marked; and in response to the operation of triggering the first control, associating the second object with a first identifier.

[0272] The first prompt information can be found above. Figure 13 The prompt message is as described above. The first control can be referenced from the above. Figure 13 The control in the text is "Yes".

[0273] This approach, by prompting the user with a first message whether to annotate, and then annotating the second object only if the user agrees, helps reduce the probability of annotation errors.

[0274] Optionally, in response to the operation of initiating recording, the method further includes: identifying a first object in the first image and associating the first object with a second identifier, the second identifier being used to identify the range in which the first object is located.

[0275] The first object can be associated with a second identifier, which is used to identify the area where the first object is located. If the second object occludes the first object, it will enter the area identified by the second identifier.

[0276] In some examples, the second identifier may refer to the above. Figure 8 The identifier 702 in the text.

[0277] In this way, by using a second identifier to indicate to the user that the first object is the identified object that the user wants to photograph, the user's recording experience can be improved.

[0278] Optionally, the first image further includes a third object; in response to the operation of initiating recording, the method further includes: identifying the first object and the third object in the first image, and associating the first object and the third object with a third identifier, the third identifier being used to identify the range of the first object and the second object; in response to the operation of adjusting the third identifier, obtaining a second identifier, the second identifier being used to identify the range of the first object.

[0279] The first and third objects are the objects that the electronic device recognizes as the objects the user wants to photograph. The third identifier is displayed to indicate that the first and third objects are the objects that the electronic device recognizes as the objects the user wants to photograph. This helps to prompt the user that the first and third objects are the objects that the electronic device recognizes as the objects they want to photograph.

[0280] The operation of adjusting the third identifier can be a drag operation or a slide operation, etc., and this application embodiment does not limit it.

[0281] The electronic device allows users to manually adjust the marker to redefine the object they want to photograph. The marker adjusted by the user is the second marker, which is used to identify the range of the first object, and the first object is the object that the user has determined to photograph.

[0282] In some examples, the third identifier may refer to the above. Figure 7 The identifier 701 is shown above. The second identifier can be referenced from the above. Figure 8 The identifier 702 in the text.

[0283] This allows users to adjust the markers to determine the subject they want to photograph, providing greater flexibility.

[0284] Understandably, the third object is not the object the user intended to film. If, during recording, the third object obscures the first object, the electronic device can mark the third object. After recording, the resulting video will not contain any image of the third object obscuring the first object.

[0285] Optionally, within the first time period, the method further includes: if the second object is in motion, calculating the first direction of motion and the first speed of the second object; determining the first probability that the second object occludes the first object based on the first direction of motion and the first speed; if the first probability is greater than the first preset probability, extracting the contour information of the second object; if the second object occludes part of the first object, the method further includes: associating the second object with a first identifier based on the contour information of the second object.

[0286] Within the first time period, the second object does not obscure the first object, but the second object is a moving object. The electronic device can determine the first probability that the second object obscures the first object. If the probability is large, that is, the first probability is greater than the first preset probability, the outline information of the second object is extracted so that the second object can be quickly associated with the first identifier when the second object partially obscures the first object, which is beneficial to improving the display rate of the first identifier.

[0287] Optionally, the electronic device further includes a second camera, the second camera having a shooting angle greater than that of the first camera; in response to the operation of starting recording, the method further includes: acquiring a second image captured by the second camera, the second image including all objects in the first image, the second image also including a fourth object; if the fourth object is in motion, calculating a second direction of motion and a second speed of motion of the fourth object; determining a second probability that the fourth object enters the first image based on the second direction of motion and the second speed of motion; if the second probability is greater than a second preset probability, extracting the contour information of the fourth object; and if the fourth object partially occludes the first object, associating a fourth identifier with the fourth object based on the contour information of the fourth object, the fourth identifier being used to identify the position of the fourth object.

[0288] In this method, the first camera can be a standard camera or a telephoto camera. The second camera can be an ultra-wide-angle camera. The first and second cameras can acquire images at the same time. The second image contains more pixels than the first image. The second image includes a fourth object, while the first image does not.

[0289] In some examples, the field of view of the first camera may be as shown in 1501, and the field of view of the second camera may be as shown in 1502. The first object may include the object circled by identifier 1503. The fourth object may be a person 550.

[0290] The second preset probability may be the same as or different from the first preset probability, and this application embodiment does not limit this. The second motion direction may be the same as or different from the first motion direction, and this application embodiment does not limit this. The second motion speed may be the same as or different from the first motion speed, and this application embodiment does not limit this.

[0291] If the fourth object is in motion and the second probability of the fourth object entering the first image is greater than the second preset probability, it indicates that the fourth object has a higher probability of occluding the first object. In this case, the electronic device can extract the contour information of the fourth object so that when the fourth object partially occludes the first object, it can quickly associate the fourth object with a fourth identifier based on the contour information of the fourth object, which helps to improve the display speed of the fourth identifier.

[0292] Optionally, the method may further include: when the depth information of the second object is less than that of the first object, and the second object and the first object have a partially overlapping area in the first image, determining that the second object occludes a portion of the first object. This facilitates the identification of the portion of the first object occluded by the second object, enabling subsequent annotation and removal.

[0293] Optionally, in response to an operation to stop recording, obtaining a first video includes: in response to an operation to stop recording, obtaining a second video, wherein during a second duration of recording the second video, the second video includes a portion of a first object not occluded by a second object and a second object, the second object being associated with a first identifier; during the second duration of recording the second video, eliminating the second object associated with the first identifier and filling the eliminated pixels to obtain the first video.

[0294] In some examples, the second video may refer to the above. Figure 16 Video 1 in S1605, the first video can be referenced above. Figure 16 Video 3 in S1609. The process of obtaining the first video from the second video can be referred to S1605 to S1609 above, and will not be repeated here.

[0295] In this way, eliminating the second object that obscures the first object helps to obtain an unobstructed video of the first object, which improves the user's recording experience.

[0296] Optionally, the electronic device can also support user-selected objects for annotation. For example, the method further includes: when the second object does not obscure the first object, in response to the operation of annotating the second object, the electronic device can associate a second identifier with the second object. When recording stops, the electronic device can remove the association of the second identifier with the second object. This allows users to identify the objects they want to identify, providing greater flexibility and improving the user's recording experience.

[0297] The first scenario constructed by the embodiments of this application has been introduced above. The second scenario constructed by the embodiments of this application will be introduced below.

[0298] For example, embodiments of this application also provide an image processing method that can be applied to an electronic device including a first camera. The method may include: in response to an operation for recording video, displaying a first interface, the first interface displaying a first image captured by the first camera, the first image including a first object and a second object, the first object and the second object not obscuring each other; at a first moment, a third object does not obscure the first object and does not obscure the second object, the first image including the first object, the second object, and the third object; at a second moment, the third object does not obscure the first object and obscures part of the second object, the first image including the first object, the portion of the second object not obscured by the third object, and the third object; at a third moment, the third object obscures part of the first object but does not obscure the second object, the first image including the first object and the second object, but not including the third object; at a fourth moment, the third object obscures part of the first object and part of the second object, the first image including the portions of the first object and the second object not obscured by the third object, but not including the third object.

[0299] The first object is the object to be photographed, and the second object is a non-object to be photographed. If the first object and the second object do not obstruct each other, the first image may include both the first and second objects. If the third object does not obstruct the first object, the first image will include the third object regardless of whether the third object obstructs the second object. If the third object obstructs the first object, the first image will not include the third object regardless of whether the third object obstructs the second object.

[0300] In this way, electronic devices can eliminate third objects that obscure the first object in real time, eliminating the need for users to manually remove third objects from the recorded video, which improves the user's recording experience.

[0301] Optionally, if a third object partially obscures a first object, the method further includes: displaying a second prompt message and a second control, the second prompt message being used to prompt whether to eliminate the third object; and eliminating the third object in response to triggering the operation of the second control, such that the first image does not include the third object.

[0302] The second prompt message can be found above. Figure 11 The prompt message in the text. The second control can be referenced as described above. Figure 11 The control in the text is "Yes".

[0303] This implementation method prompts the user with a second message to indicate whether to eliminate the object. If the user agrees to eliminate the object, the second object is then eliminated, which helps reduce the probability of elimination errors.

[0304] Optionally, in response to the operation of initiating recording, the method further includes: identifying a first object in the first image and associating the first object with a second identifier, the second identifier being used to identify the range in which the first object is located.

[0305] In this way, by using a second identifier to indicate to the user that the first object is the identified object that the user wants to photograph, the user's recording experience can be improved.

[0306] Optionally, in response to an operation to initiate recording, the method further includes: identifying a first object and a third object in the first image, and associating the first object and the third object with a third identifier, the third identifier being used to identify the range in which the first object and the second object are located; and in response to an operation to adjust the third identifier, obtaining a second identifier, the second identifier being used to identify the range in which the first object is located. This allows users to adjust the identifier to determine the object they want to capture, providing greater flexibility.

[0307] Optionally, the above method further includes: if the second object is in motion, calculating the first direction of motion and the first speed of the second object; determining the first probability that the second object occludes the first object based on the first direction of motion and the first speed; if the first probability is greater than the first preset probability, extracting the contour information of the second object; and, if the second object occludes part of the first object, eliminating the second object based on the contour information of the second object, so that the first image includes the first object but does not include the second object.

[0308] The second object does not obscure the first object, but the second object is a moving object. The electronic device can determine the first probability that the second object obscures the first object. If the probability is large, that is, the first probability is greater than the first preset probability, the outline information of the second object is extracted. This is so that when the second object partially obscures the first object, the first identifier can be quickly associated with the second object, which is beneficial to improving the display speed of the first identifier.

[0309] Optionally, the aforementioned electronic device further includes a second camera, the second camera having a shooting angle greater than that of the first camera; in response to the operation of initiating recording, the method further includes: acquiring a second image captured by the second camera, the second image including all objects in the first image, the second image also including a fourth object; if the fourth object is in motion, calculating a second direction of motion and a second speed of motion of the fourth object; determining a second probability that the fourth object enters the first image based on the second direction of motion and the second speed of motion; if the second probability is greater than a second preset probability, extracting the contour information of the fourth object; and, in the case where the fourth object partially occludes the first object, eliminating the fourth object based on the contour information of the fourth object, so that the first image includes the first object but does not include the fourth object.

[0310] If the fourth object is in motion and the second probability of the fourth object entering the first image is greater than the second preset probability, it indicates that the fourth object has a higher probability of occluding the first object. In this case, the electronic device can extract the contour information of the fourth object so that when the fourth object partially occludes the first object, it can quickly associate the fourth object with a fourth identifier based on the contour information of the fourth object, which helps to improve the display speed of the fourth identifier.

[0311] Optionally, the method further includes: determining the portion of the first object occluded by the third object when the depth information of the third object is less than that of the first object, and the third object and the first object have a partially overlapping area in the first image. This facilitates the identification of the portion of the first object occluded by the third object, making it easier to remove.

[0312] Optionally, the method further includes: in response to an operation for eliminating a second object, eliminating the second object such that the first image does not include the second object.

[0313] In some examples, the second object may refer to the above. Figure 17 Plant 540 in the middle, the operation to eliminate the second object can be the operation of selecting the second object.

[0314] In this way, electronic devices allow users to remove any object from an image, offering greater flexibility and improving the user's recording experience.

[0315] It should be noted that the module names involved in the embodiments of this application can all be defined as other names, as long as they can achieve the function of each module, and no specific restrictions are placed on the module names.

[0316] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0317] The image processing method of the present application embodiments has been described above. The apparatus for performing the above method provided in the present application embodiments is described below. Those skilled in the art will understand that the methods and apparatus can be combined with and referenced by each other, and the related apparatus provided in the present application embodiments can perform the steps in the above list sorting method.

[0318] Figure 19 This is a schematic diagram of a chip structure provided in an embodiment of this application. Figure 19As shown, chip 190 includes one or more processors 1901, communication lines 1902, communication interfaces 1903, and memory 1904.

[0319] In some implementations, memory 1904 stores elements such as executable modules or data structures, or subsets thereof, or extended sets thereof.

[0320] The image processing method described in the embodiments of this application can be applied to, or implemented by, processor 1901. Processor 1901 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above image processing method can be completed by integrated logic circuits in the hardware of processor 1901 or by instructions in software form. Processor 1901 may be a general-purpose processor (e.g., a microprocessor or conventional processor), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates, transistor logic devices, or discrete hardware components. Processor 1901 can implement or execute the various processing-related methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0321] The steps of the image processing method disclosed in the embodiments of this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can be located in mature storage media in the art, such as random access memory, read-only memory, programmable read-only memory, or electrically erasable programmable read-only memory (EEPROM). This storage medium is located in memory 1904, and processor 1901 reads the information in memory 1904 and, in conjunction with its hardware, completes the steps of the above method.

[0322] The processor 1901, memory 1904 and communication interface 1903 can communicate with each other via communication line 1902.

[0323] In the above embodiments, the instructions stored in the memory for execution by the processor can be implemented in the form of a computer program product. This computer program product can be pre-written into the memory, or it can be downloaded and installed into the memory as software.

[0324] The image processing method provided in this application can be applied to electronic devices with video recording functions. The electronic device includes a terminal device; the specific device form of the terminal device can be referred to the above-described related information, and will not be repeated here.

[0325] This application provides a terminal device, which includes a processor and a memory; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, causing the terminal device to perform the above-described method.

[0326] This application provides a chip. The chip includes a processor, which is used to call a computer program in memory to execute the technical solutions in the above embodiments. Its implementation principle and technical effects are similar to those in the related embodiments described above, and will not be repeated here.

[0327] This application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, it implements the methods described above. The methods described in the above embodiments can be implemented wholly or partially by software, hardware, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions or code on or transmitted over the computer-readable medium. The computer-readable medium can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium accessible by a computer.

[0328] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage or other magnetic storage devices, or any other medium targeted to carry or to store the required program code in the form of instructions or data structures, and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. As used herein, disks and optical discs include optical discs, laser discs, optical discs, Digital Versatile Discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs optically reproduce data using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0329] This application provides a computer program product, which includes a computer program that, when run, causes a computer to perform the above-described method.

[0330] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0331] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image processing method, characterized by, Applied to an electronic device including a first camera, the method includes: In response to an operation to start recording, a first interface is displayed, the first interface displaying a first image captured by the first camera, the first image including a first object; During the first duration of the recording process, the second object does not obscure the first object, and the first image includes the first object and the second object; During the second duration of the recording process, the second object partially occludes the first object. The first image includes the second object and the portion of the first object that is not occluded by the second object. The second object is associated with a first identifier, which is used to identify the position of the second object. The start time of the second duration is later than the end time of the first duration. During the third duration of the recording process, the second object does not obscure the first object, the first image includes the first object and the second object, the second object is not associated with the first identifier, and the start time of the third duration is later than the end time of the second duration; in response to an operation to stop recording, a first video is obtained; Wherein, within the first duration of the first video recording, the first video includes the first object and the second object; within the second duration of the first video recording, the first video includes the first object but does not include the second object; within the third duration of the first video recording, the first video includes the first object and the second object; Within the first time period, the method further includes: if the second object is in motion, calculating the first direction of motion and the first speed of the second object; and determining the first probability that the second object occludes the first object based on the first direction of motion and the first speed of motion. If the first probability is greater than the first preset probability, then the contour information of the second object is extracted; In the case where the second object partially occludes the first object, the method further includes: associating the second object with the first identifier based on the outline information of the second object.

2. The method of claim 1, wherein, When the second object partially occludes the first object, the method further includes: Display a first prompt message and a first control, wherein the first prompt message is used to prompt whether to identify the second object; In response to the operation that triggers the first control, the first identifier is associated with the second object.

3. The method according to claim 1 or 2, characterized in that, In response to the operation of initiating recording, the method further includes: The first object is identified in the first image, and a second identifier is associated with the first object, the second identifier being used to identify the range in which the first object is located.

4. The method according to claim 1 or 2, characterized in that, The first image also includes a third object; In response to the operation of initiating recording, the method further includes: The first object and the third object are identified in the first image, and a third identifier is associated with the first object and the third object, the third identifier being used to identify the range of the first object and the second object; In response to an operation to adjust the third identifier, a second identifier is obtained, which identifies the range in which the first object is located.

5. The method according to claim 1 or 2, characterized in that, The electronic device also includes a second camera, the second camera having a wider field of view than the first camera; In response to the operation of initiating recording, the method further includes: Acquire a second image captured by the second camera. The second image includes all objects in the first image and also includes a fourth object. If the fourth object is in motion, then calculate the second direction of motion and the second speed of motion of the fourth object; Based on the second direction of motion and the second running speed, determine the second probability that the fourth object enters the first image; If the second probability is greater than the second preset probability, then the contour information of the fourth object is extracted; When the fourth object partially occludes the first object, a fourth identifier is associated with the fourth object based on the outline information of the fourth object, and the fourth identifier is used to identify the position of the fourth object.

6. The method according to claim 1 or 2, characterized in that, The method further includes: If the depth information of the second object is less than that of the first object, and the second object and the first object have a partially overlapping area in the first image, it is determined that the second object occludes part of the first object.

7. The method according to claim 1 or 2, characterized in that, The process of obtaining the first video in response to an operation to stop recording includes: In response to an operation to stop recording, a second video is obtained, which, during the second duration of recording, includes the portion of the first object not obscured by the second object and the second object, the second object being associated with the first identifier; During the second duration of the second video recording, the second object associated with the first identifier is eliminated, and the eliminated pixels are filled in to obtain the first video.

8. An image processing method, characterized in that, Applied to an electronic device including a first camera, the method includes: In response to an operation to start recording, a first interface is displayed, which displays a first image captured by the first camera. The first image includes a first object and a second object, and the first object and the second object do not obstruct each other. At the first moment, the third object does not obscure the first object, nor does it obscure the second object; therefore, the first image includes the first object, the second object, and the third object. At the second moment, the third object does not obscure the first object, but partially obscures the second object. The first image includes the first object, the portion of the second object not obscured by the third object, and the third object. At the third moment, the third object partially obscures the first object but does not obscure the second object; the first image includes the first object and the second object, but does not include the third object. At the fourth moment, the third object partially occludes the first object and partially occludes the second object. The first image includes the portions of the first object and the second object that are not occluded by the third object, but does not include the third object.

9. The method according to claim 8, characterized in that, When the third object partially obscures the first object, the method further includes: Display a second prompt message and a second control, the second prompt message being used to prompt whether to eliminate the third object; In response to the operation that triggers the second control, the third object is eliminated such that the first image does not include the third object.

10. The method according to claim 8 or 9, characterized in that, In response to the operation of initiating recording, the method further includes: The first object is identified in the first image, and a second identifier is associated with the first object, the second identifier being used to identify the range in which the first object is located.

11. The method according to claim 8 or 9, characterized in that, In response to the operation of initiating recording, the method further includes: The first object and the third object are identified in the first image, and a third identifier is associated with the first object and the third object, the third identifier being used to identify the range of the first object and the second object; In response to an operation to adjust the third identifier, a second identifier is obtained, which identifies the range in which the first object is located.

12. The method according to claim 8 or 9, characterized in that, The method further includes: If the second object is in motion, then calculate the first direction of motion and the first speed of motion of the second object; Based on the first direction of movement and the first running speed, determine the first probability that the second object will occlude the first object; If the first probability is greater than the first preset probability, then the contour information of the second object is extracted; When the second object partially occludes the first object, the second object is eliminated based on its contour information, so that the first image includes the first object but does not include the second object.

13. The method according to claim 8 or 9, characterized in that, The electronic device also includes a second camera, the second camera having a wider field of view than the first camera; In response to the operation for initiating recording, the method further includes: Acquire a second image captured by the second camera. The second image includes all objects in the first image and also includes a fourth object. If the fourth object is in motion, then calculate the second direction of motion and the second speed of motion of the fourth object; Based on the second direction of motion and the second running speed, determine the second probability that the fourth object enters the first image; If the second probability is greater than the second preset probability, then the contour information of the fourth object is extracted; When the fourth object partially occludes the first object, the fourth object is eliminated based on its contour information, so that the first image includes the first object but does not include the fourth object.

14. The method according to claim 8 or 9, characterized in that, The method further includes: If the depth information of the third object is less than the depth information of the first object, and the third object and the first object have a partially overlapping area in the first image, it is determined that the third object occludes part of the first object.

15. The method according to claim 8 or 9, characterized in that, The method further includes: In response to the operation for eliminating the second object, the second object is eliminated such that the first image does not include the second object.

16. An electronic device, characterized in that, The electronic device includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 7, or to perform the method as described in any one of claims 8 to 15.

17. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the one or more processors being configured to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 7, or to perform the method as described in any one of claims 8 to 15.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as claimed in any one of claims 1 to 7, or to perform the method as claimed in any one of claims 8 to 15.

19. A computer program product, characterized in that, The computer program product includes computer program code that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 1 to 7, or to perform the method as described in any one of claims 8 to 15.

Images