An image interference elimination method, device and computer readable storage medium

Abstract

The application discloses an image interference elimination method and device and a computer readable storage medium, wherein the method comprises the following steps: detecting all shooting objects contained in an image in real time during shooting, determining at least one shooting object as a foreground object among all the shooting objects, and taking the remaining shooting objects as background objects; determining at least one background object as an interference object; acquiring the motion state of the interference object in real time, and determining the interference region where the interference object is located in the image according to the motion state; removing all pixels within the interference region range, and restoring the pixels within the interference region range according to the pixels of the peripheral region of the interference region. A humanized image interference elimination scheme is realized, the interference elimination steps are simplified, the use difficulty is reduced, the elimination effect is improved, and the user experience is enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of mobile communication, and in particular to an image interference elimination method, device and computer readable storage medium. BACKGROUND

[0002] In the prior art, with the continuous development of intelligent terminal devices, users have higher and higher requirements for the shooting of devices. In particular, when shooting in a complex dynamic scene, other objects in the range of viewfinder may appear to interfere with the shooting subject. With the current image processing technology, if the other irrelevant shooting objects need to be eliminated, the user needs to call the corresponding image processing program and perform relatively complex image processing steps to achieve a satisfactory processing effect, which undoubtedly brings higher learning cost and difficulty to ordinary users. Therefore, there is an urgent need for a convenient, accurate and more widely applicable image interference adaptive elimination scheme. SUMMARY

[0003] In order to solve the above technical defects in the prior art, the present application provides an image interference elimination method, which comprises:

[0004] In the shooting process, all shooting objects contained in the image are detected in real time, at least one of all the shooting objects is determined as a foreground object, and the remaining shooting objects are determined as background objects.

[0005] According to the relative motion relationship of the foreground object and all the background objects, at least one of all the background objects is determined as an interference object, and the remaining background objects are determined as non-interference objects.

[0006] The motion state of the interference object is obtained in real time, and the interference region where the interference object is located in the image is determined according to the motion state.

[0007] All pixels in the interference region are removed, and the pixels in the interference region are restored according to the pixels of the surrounding region of the interference region.

[0008] Optionally, the method for detecting all shooting objects contained in the image in real time in the shooting process, determining at least one of all the shooting objects as a foreground object, and determining the remaining shooting objects as background objects comprises:

[0009] Detect the current shooting type, wherein the shooting type includes image shooting and video shooting.

[0010] All shooting objects contained in the image are detected and marked in real time in the preview interface of image shooting or in the viewfinder interface of video shooting.

[0011] Optionally, the real-time detection of all the photographed objects contained in the image during the photographing process, the determination of at least one of the photographed objects as a foreground object, and the remaining photographed objects as background objects further comprise:

[0012] The determination of the foreground object, wherein the determination comprises a user-triggered selection of the mark or a pre-set detection model.

[0013] Real-time marking of the foreground object and the background object in a preview interface of image shooting or a viewfinder interface of video shooting, and tracking of the foreground object and the background object, respectively.

[0014] Optionally, the determination of at least one of the background objects as a disturbance object and the remaining background objects as non-disturbance objects according to the relative motion relationship between the foreground object and all the background objects further comprises:

[0015] Real-time receiving of a selection instruction of the background object in the preview interface of image shooting or the viewfinder interface of video shooting.

[0016] Determination of at least one of the background objects as the disturbance object according to the selection instruction.

[0017] Optionally, the determination of at least one of the background objects as a disturbance object and the remaining background objects as non-disturbance objects according to the relative motion relationship between the foreground object and all the background objects further comprises:

[0018] Determination of an object attribute of the selected disturbance object, wherein the object attribute comprises an object type and an object size.

[0019] According to the relative motion relationship, a background object having the same object attribute and motion state as the selected disturbance object is determined as the disturbance object.

[0020] Optionally, the real-time acquisition of the motion state of the disturbance object and the determination of a disturbance area in the image where the disturbance object is located according to the motion state further comprises:

[0021] When a photographing instruction is detected in the preview interface of image shooting, the motion state of the disturbance object before the photographing instruction is extracted.

[0022] The determination of a disturbance area in the image where the disturbance object is located according to the motion state.

[0023] Optionally, the real-time motion state of the interference object is acquired, and the interference region where the interference object is located in the image is determined according to the motion state, and the method further comprises:

[0024] When the camera shooting instruction is detected in the taking interface of the video shooting, the first motion state of the foreground object and the second motion state of the interference object are acquired.

[0025] According to the first motion state and the second motion state, the motion region and the interference region are distinguished in the taking interface.

[0026] Optionally, the pixels in the interference region range are removed, and the pixels in the interference region range are restored according to the pixels in the peripheral region of the interference region, and the method further comprises:

[0027] The search range of the pixels in the interference region range is converged through iteration of the image of the peripheral region until the peripheral region corresponding to the interference region is acquired.

[0028] The pixels in the interference region range are restored according to the pixels in the peripheral region.

[0029] The application further provides an image interference elimination device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program realizes the steps of the image interference elimination method according to any one of the above when executed by the processor.

[0030] The application further provides a computer readable storage medium, which stores an image interference elimination program, and the image interference elimination program realizes the steps of the image interference elimination method according to any one of the above when executed by a processor.

[0031] The image interference elimination method, device and computer readable storage medium of the present application are characterized in that all the photographed objects contained in the image are detected in real time during the photographing process, at least one of the photographed objects is determined as a foreground object, and the remaining photographed objects are determined as background objects; according to the relative motion relationship between the foreground object and all the background objects, at least one of the background objects is determined as an interference object, and the remaining background objects are determined as non-interference objects; the motion state of the interference object is acquired in real time, and the interference region where the interference object is located is determined in the image according to the motion state; all the pixels in the interference region are removed, and the pixels in the interference region are restored according to the pixels in the peripheral region of the interference region. A humanized image interference elimination scheme is realized, the interference elimination steps are simplified, the use difficulty is reduced, the elimination effect is improved, and the user experience is enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0032] The present application will be further described below in combination with the drawings and embodiments, wherein:

[0033] Figure 1 is a schematic diagram of a hardware structure of a mobile terminal related to the present application;

[0034] Figure 2 is a communication network system architecture diagram provided by an embodiment of the present application;

[0035] Figure 3 is a flowchart of a first embodiment of the image interference elimination method of the present application;

[0036] Figure 4 is a flowchart of a second embodiment of the image interference elimination method of the present application;

[0037] Figure 5 is a flowchart of a third embodiment of the image interference elimination method of the present application;

[0038] Figure 6 is a flowchart of a fourth embodiment of the image interference elimination method of the present application;

[0039] Figure 7 is a flowchart of a fifth embodiment of the image interference elimination method of the present application;

[0040] Figure 8 is a flowchart of a sixth embodiment of the image interference elimination method of the present application;

[0041] Figure 9 is a flowchart of a seventh embodiment of the image interference elimination method of the present application;

[0042] Figure 10is a flowchart of an eighth embodiment of the image interference elimination method of the present invention.

[0043] Figure 11 is an elimination schematic diagram of a first embodiment of the image interference elimination method of the present invention. DETAILED DESCRIPTION

[0044] It should be understood that the specific embodiments described herein are merely illustrative of the present invention and do not limit the present invention.

[0045] In the following description, the suffixes used for elements, such as "module", "part", or "unit", are merely intended for facilitating a description of the present invention, and do not have specific meanings or roles.

[0046] A terminal can be implemented in various forms. For example, the terminal described in the present invention can include a mobile terminal such as a mobile phone, a tablet, a notebook computer, a palmtop computer, a Personal Digital Assistant (PDA), a Portable Media Player (PMP), a navigation device, a wearable device, a smart band, a pedometer, and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like.

[0047] In the following description, a mobile terminal will be exemplified, and it will be understood by those skilled in the art that the configuration according to the embodiments of the present invention can be applied to a stationary type terminal, except for elements particularly used for mobile purposes.

[0048] Referring to Figure 1 is a hardware structure schematic diagram of a mobile terminal implementing the various embodiments of the present invention. The mobile terminal 100 can include an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an A / V (audio / video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a storage 109, a processor 110, and a power supply 111, and the like. Those skilled in the art will appreciate that the mobile terminal structure shown in Figure 1 The mobile terminal structure shown in FIG. 1 is not intended to limit the mobile terminal, and the mobile terminal can include more or less components than those shown, or combine certain components, or arrange the components in different arrangements.

[0049] The following will be described in detail with reference to Figure 1 The various components of the mobile terminal will be described in detail.

[0050] The radio frequency unit 101 can be used for receiving and transmitting signals in information or communication processes. Specifically, the radio frequency unit 101 receives downlink information from a base station and provides the received information to the processor 110 for processing. In addition, the radio frequency unit 101 transmits uplink data to the base station. Generally, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 101 can communicate with the network and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System for Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution), etc.

[0051] The WiFi belongs to a short-range wireless transmission technology. The WiFi module 102 can help the user to send and receive e-mails, browse web pages, and access streaming media, etc. It provides the user with wireless broadband Internet access. Although Figure 1 The WiFi module 102 is shown, but it is understood that it does not belong to the necessary components of the mobile terminal, and can be omitted as needed without changing the essence of the application.

[0052] The audio output unit 103 can convert audio data, which is received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109, into an audio signal and output the audio signal as sound when the mobile terminal 100 is in a call signal receiving mode, a call mode, a recording mode, a voice recognition mode, a broadcast reception mode, etc. In addition, the audio output unit 103 can provide audio output related to a particular function performed by the mobile terminal 100 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 103 can include a speaker, a buzzer, etc.

[0053] The A / V input unit 104 is configured to receive audio or video signals. The A / V input unit 104 can include a graphics processor (GPU) 1041 and a microphone 1042. The graphics processor 1041 processes image data of a still picture or a video obtained by an image capture device (e.g., a camera) in a video call mode or an image call mode. Processed image frames can be displayed on the display unit 106. Processed image frames can be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in a phone call mode, a recording mode, a voice recognition mode, or the like, and can process such sound into audio data. Processed audio (voice) data can be converted into a format transmittable to a mobile communication base station via the radio frequency unit 101 in the case of the phone call mode. The microphone 1042 can implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated in the process of receiving and transmitting audio signals.

[0054] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of ambient light, and the proximity sensor can turn off the display panel 1061 and / or the backlight when the mobile terminal 100 is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when at rest, and can be used for applications that recognize the posture of the mobile phone (such as switching between landscape and portrait screens, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometers, taps), and the like. As for the fingerprint sensor, pressure sensor, iris sensor, molecular sensor, gyroscope, barometer, hygrometer, thermometer, infrared sensor, and other sensors that can be configured to the mobile phone, they are not described here.

[0055] The display unit 106 is configured to display information input by a user or information provided to the user. The display unit 106 can include a display panel 1061, which can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0056] The user input unit 107 can be used to receive input numerals or character information, and to generate key signal inputs related to user settings of the mobile terminal and control of functions. Specifically, the user input unit 107 can include a touch panel 1071 and other input devices 1072. The touch panel 1071, also called a touch screen, can collect a touch operation of a user on or proximate thereto (such as an operation of the user using a finger, a stylus, or any suitable object or accessory on or proximate to the touch panel 1071), and drive a corresponding connection device according to a pre-set program. The touch panel 1071 can include two parts, a touch detecting device and a touch controller. The touch detecting device detects a user's touch position and detects a signal resulting from a touch operation, and transmits the signal to the touch controller. The touch controller receives touch information from the touch detecting device, converts it into touch coordinates, and transmits the touch coordinates to the processor 110, and can receive commands from the processor 110 and execute them. In addition, the touch panel 1071 can be implemented in various types such as a resistive type, a capacitive type, an infrared type, and a surface acoustic wave type. In addition to the touch panel 1071, the user input unit 107 can include other input devices 1072. Specifically, the other input devices 1072 can include one or more of, but are not limited to, a physical keyboard, function keys (such as a volume control button, a switch button, etc.), a trackball, a mouse, a joystick, etc.

[0057] Further, the touch panel 1071 can cover the display panel 1061, and when the touch panel 1071 detects a touch operation on or proximate thereto, it transmits the same to the processor 110 to determine the type of the touch event, and then the processor 110 provides a corresponding visual output on the display panel 1061 according to the type of the touch event. Although in the above description, the touch panel 1071 and the display panel 1061 are implemented as two separate components to achieve the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to achieve the input and output functions of the mobile terminal, without being limited thereto. Figure 1

[0058] The interface unit 108 serves as an interface through which at least one external device can be connected to the mobile terminal 100. For example, the external device can include a wired or wireless headset port, an external power (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input / output (I / O) port, a video I / O port, an earphone port, etc. The interface unit 108 can be used to receive input (e.g., data information, power, etc.) from the external device and to transmit the received input to one or more elements within the mobile terminal 100, or can be used to transmit data between the mobile terminal 100 and the external device. ​

[0059] The memory 109 can be used to store software programs and various data. The memory 109 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; and the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), and the like. In addition, the memory 109 can include a high-speed random access memory, and can also include a nonvolatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0060] The processor 110 is the control center of the mobile terminal, connects all parts of the mobile terminal through various interfaces and lines, executes various functions of the mobile terminal and processes data by running or executing software programs and / or modules stored in the memory 109 and calling data stored in the memory 109, and thus performs overall monitoring on the mobile terminal. The processor 110 can include one or more processing units; preferably, the processor 110 can integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the like, and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 110.

[0061] The mobile terminal 100 can also include a power supply 111 (such as a battery) for supplying power to various components; preferably, the power supply 111 can be logically connected to the processor 110 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system.

[0062] Although Figure 1 The mobile terminal 100 can also include a Bluetooth module and the like, which are not described herein again.

[0063] In order to facilitate the understanding of the embodiments of the present application, the communication network system based on the mobile terminal of the present application is described below.

[0064] Please refer to Figure 2 , Figure 2 A communication network system architecture diagram provided by the embodiments of the present application, the communication network system is a LTE system of general mobile communication technology, the LTE system includes a UE (User Equipment, user equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network, evolved UMTS terrestrial radio access network) 202, an EPC (Evolved Packet Core, evolved packet core network) 203 and an operator's IP service 204 connected in turn.

[0065] Specifically, the UE 201 can be the terminal 100 described above, which will not be repeated here.

[0066] The E-UTRAN 202 includes eNode-Bs 2021 and other eNode-Bs 2022. The eNode-B 2021 can be connected to the other eNode-Bs 2022 through backhaul (e.g., X2 interface), the eNode-B 2021 is connected to the EPC 203, and the eNode-B 2021 can provide access for the UE 201 to the EPC 203.

[0067] The EPC 203 can include a MME (Mobility Management Entity) 2031, a HSS (Home Subscriber Server) 2032, other MMEs 2033, a SGW (Serving Gate Way) 2034, a PGW (PDN Gate Way) 2035, and a PCRF (Policy and Charging Rules Function) 2036. The MME 2031 is a control node that processes signaling between the UE 201 and the EPC 203, and provides bearer and connection management. The HSS 2032 is configured to provide some registers to manage functions such as a home location register (not shown in the figure), and to store some user-specific information about service features, data rates, etc. All user data can be transmitted through the SGW 2034, the PGW 2035 can provide IP address allocation for the UE 201 and other functions, and the PCRF 2036 is a policy and charging control policy decision point for service data flow and IP bearer resources, which selects and provides available policy and charging control decisions for policy and charging enforcement function units (not shown in the figure).

[0068] The IP service 204 can include the Internet, an intranet, an IMS (IP Multimedia Subsystem), or other IP services.

[0069] Although the above describes the LTE system as an example, those skilled in the art should know that the present application is not only applicable to the LTE system, but also applicable to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, and future new network systems, etc., which are not limited here.

[0070] Based on the above mobile terminal hardware structure and communication network system, various embodiments of the method of the present application are proposed.

[0071] Embodiment One

[0072] Figure 3 is a flow chart of the first embodiment of the image interference elimination method of the present application. An image interference elimination method, the method comprising:

[0073] S1, detecting all shooting objects contained in an image in real time during shooting, determining at least one of the shooting objects as a foreground object, and taking the remaining shooting objects as background objects.

[0074] S2, determining at least one of the background objects as an interference object according to the relative motion relationship of the foreground object and all the background objects, and taking the remaining background objects as non-interference objects.

[0075] S3, acquiring the motion state of the interference object in real time, and determining the interference region where the interference object is located in the image according to the motion state.

[0076] S4, removing all pixels within the interference region, and restoring the pixels within the interference region according to the pixels of the surrounding region of the interference region.

[0077] In this embodiment, an image interference elimination scheme based on user's region of interest is proposed. The scheme is suitable for both video shooting and photographing. Specifically, the scheme predicts whether the background of shooting has interference, and adaptively extracts the information of interference objects. In this process, the user can manually intervene to select and determine whether it is an interference object according to actual needs, and then the scheme removes the interference objects that intrude into the viewfinder range. In this embodiment, the image within the removal range is restored while the interference objects are removed.

[0078] Optionally, in the embodiment, the user actively selects the interference object as an example for illustration. First, before starting to shoot, the image is subjected to object detection by deep learning, the possible foreground target (pedestrian, pet, building) in the field of view is judged and recommended, and the approximate area of the foreground target is determined; the user selects the foreground target (pedestrian) recommended according to the preset algorithm or learning model. The user can select a static object as a foreground target (building), or can select a moving object as a foreground target; then, the user can autonomously select any target as an interference object, or the algorithm can automatically recommend an interference object; then, the tracking state of all foreground targets and interference objects is maintained, and the related information of these objects is acquired in real time; the position information of the interference object is acquired in real time, the area where the interference object is located is determined; finally, the determined area of the interference object is removed from the current image, and after the image removes the interference object, the surrounding pixels are used for restoration, that is, the user has a visual experience that no one has entered the background.

[0079] Optionally, in the embodiment, the preset algorithm used can be an image restoration algorithm patchmatch. Specifically, an image A is taken as patch_A, and a region in the image that is considered to be repaired is determined; then, the patch block most adjacent to patch_A in a related another image B is searched to restore the image A. It can be understood that patch_A is unknown in practice, so the embodiment needs to re-determine the image block near patch_A, and the specific position of the image block near patch_A in the initial state is not clear, so the position of the initial image block near patch_A is randomly initialized in the embodiment. Based on this, in the embodiment, the continuity of the image is used, and the patch block most likely to appear near the nearest neighbor of patch_A is in the nearest neighbor of patch_A, so the continuity of the image is used to greatly reduce the search range, and iteration is used to ensure that most points can converge as soon as possible. Thus, by the method of continuous iteration and propagation, the corresponding patch block near patch_A and the nearest neighbor frame in the B image are obtained, and the nearest neighbor frame is used to repair the image.

[0080] Optionally, in the embodiment, please refer to Figure 11 The elimination schematic diagram of the first embodiment of the image interference elimination method of the application is shown. In the process of image shooting, if there is a dynamic person as an interference object different from the conventional image interference elimination requirement, the corresponding processing mode is to take the left image as the image to be restored, then obtain the middle image as the region to be restored, and finally obtain the image restoration effect of the right image according to the above-mentioned preset algorithm.

[0081] The beneficial effect of the embodiment is that by detecting all the photographed objects contained in the image in real time during the shooting process, determining at least one of the photographed objects as a foreground object among all the photographed objects, and taking the remaining photographed objects as background objects; according to the relative motion relationship of the foreground object and all the background objects, determining at least one of the background objects as an interference object among all the background objects, and taking the remaining background objects as non-interference objects; obtaining the motion state of the interference object in real time, and determining the interference area where the interference object is located in the image according to the motion state; removing all pixels within the interference area range, and restoring the pixels within the interference area range according to the pixels of the peripheral area of the interference area. A humanized image interference elimination scheme is realized, the interference elimination steps are simplified, the use difficulty is reduced, the elimination effect is improved, and the user experience is enhanced.

[0082] Embodiment two

[0083] Figure 4 is a flowchart of the second embodiment of the image interference elimination method of the application, based on the above embodiment, the real-time detection of all the photographed objects contained in the image during the shooting process, the determination of at least one of the photographed objects as a foreground object among all the photographed objects, and the taking of the remaining photographed objects as background objects, comprising:

[0084] S11, detecting the current shooting type, wherein the shooting type includes image shooting and video shooting.

[0085] S12, detecting and marking all the photographed objects contained in the image in real time in the preview interface of image shooting or in the viewfinder interface of video shooting.

[0086] Optionally, in the embodiment, the camera preview is opened, the object detection is performed, the foreground objects possibly contained in the current image are obtained, including pedestrians, pets and buildings, etc.; then, the current camera application type is determined, including photographing and video recording.

[0087] Optionally, in the embodiment, the current is the photographing mode, so the tracking state of all the foreground objects and interference objects is maintained, and the related information of these objects is obtained in real time.

[0088] Optionally, in the embodiment, the current is the video recording mode, so the tracking state of the detected objects is maintained, and the related information is obtained in real time.

[0089] The beneficial effect of the embodiment is that by detecting the current shooting type, wherein the shooting type includes image shooting and video shooting, all shooting objects contained in the image are detected and marked in real time in the preview interface of image shooting or in the viewfinder interface of video shooting. A humanized image interference elimination scheme is realized, the interference elimination steps are simplified, the use difficulty is reduced, the elimination effect is improved, and the user experience is enhanced.

[0090] Embodiment three

[0091] Figure 5 is a flowchart of the third embodiment of the image interference elimination method of the application, based on the above-mentioned embodiment, the all shooting objects contained in the image are detected in real time during the shooting process, at least one of the all shooting objects is determined as a foreground object, and the remaining shooting objects are determined as background objects, and the method further comprises:

[0092] S13, determining the determination mode of the foreground object, wherein the determination mode comprises selecting according to the user's trigger selection of the mark, or selecting according to a preset detection model.

[0093] S14, distinguishing and marking the foreground object and the background object in real time in the preview interface of image shooting or in the viewfinder interface of video shooting, and tracking the foreground object and the background object respectively.

[0094] Optionally, in the embodiment, in the shooting scene, the user can autonomously select any target as the interference object, or the algorithm can automatically recommend the interference object.

[0095] Optionally, in the embodiment, unlike the shooting scene, in the video recording scene, since some backgrounds are moving during video shooting, for example, leaves and waves, if the interference target is directly removed and repaired, the image repair will be wrong. Therefore, in the embodiment, first, the motion area existing in the video shooting area is estimated by establishing a model, wherein the embodiment further limits that the shooting device does not have large-scale shaking; then, the current motion area and the interference target area are distinguished.

[0096] The beneficial effect of the embodiment is that by determining the determination mode of the foreground object, wherein the determination mode comprises selecting according to the user's trigger selection of the mark, or selecting according to a preset detection model; the foreground object and the background object are distinguished and marked in real time in the preview interface of image shooting or in the viewfinder interface of video shooting, and the foreground object and the background object are tracked respectively. A humanized image interference elimination scheme is realized, the interference elimination steps are simplified, the use difficulty is reduced, the elimination effect is improved, and the user experience is enhanced.

[0097] Embodiment Four

[0098] Figure 6 is a flow chart of the fourth embodiment of the image interference elimination method of the present application, based on the above-mentioned embodiments, the relative motion relationship between the foreground object and all the background objects is determined, at least one of the background objects is determined as the interference object, and the remaining background objects are determined as non-interference objects, comprising:

[0099] S21, receiving a selection instruction of the background object in real time in the preview interface of image shooting or in the viewfinder interface of video shooting.

[0100] S22, determining at least one of the background objects as the interference object according to the selection instruction.

[0101] Optionally, in this embodiment, the marked object is selected as the selected interference object from left to right or from top to bottom by the shaking action of the user.

[0102] The beneficial effects of this embodiment are that the selection instruction of the background object is received in real time in the preview interface of image shooting or in the viewfinder interface of video shooting, and at least one of the background objects is determined as the interference object according to the selection instruction. A humanized image interference elimination scheme is realized, the interference elimination steps are simplified, the use difficulty is reduced, the elimination effect is improved, and the user experience is enhanced.

[0103] Embodiment Five

[0104] Figure 7 is a flow chart of the fifth embodiment of the image interference elimination method of the present application, based on the above-mentioned embodiments, the relative motion relationship between the foreground object and all the background objects is determined, at least one of the background objects is determined as the interference object, and the remaining background objects are determined as non-interference objects, further comprising:

[0105] S23, determining the object attribute of the selected interference object, wherein the object attribute includes the object type and the object size.

[0106] S24, according to the relative motion relationship, the background objects with the same object attribute and motion state as the selected interference object are determined as the interference object.

[0107] Optionally, in this embodiment, the object type is human body, animal, tree, vehicle, etc., the object size is the size in the viewfinder range, and the motion state is the speed and / or direction of the motion.

[0108] The embodiment has the beneficial effects that the object attribute of the selected interference object is determined, the object attribute includes an object type and an object size, and a background object with the same object attribute and motion state as the selected interference object is taken as the interference object according to the relative motion relationship, a humanized image interference elimination scheme is implemented, the interference elimination steps are simplified, the use difficulty is reduced, the elimination effect is improved, and the user experience is enhanced.

[0109] Embodiment six

[0110] Figure 8 is a flowchart of the sixth embodiment of the image interference elimination method of the present application, based on the above-mentioned embodiments, the motion state of the interference object is acquired in real time, and the interference region in which the interference object is located in the image is determined according to the motion state, and the method comprises the following steps:

[0111] S31, when a photographing instruction is detected in a preview interface of image shooting, the motion state of the interference object before the photographing instruction is extracted.

[0112] S32, the interference region in which the interference object is located in the image is determined according to the motion state.

[0113] Optionally, in the embodiment, for a photographing scene, when the interference object is determined, the photograph is clicked, all the pixels in the rectangular range in which the interference object is located are deleted, and the target region deleted is restored by an image restoration method.

[0114] The embodiment has the beneficial effects that the motion state of the interference object before the photographing instruction is extracted when a photographing instruction is detected in a preview interface of image shooting, and the interference region in which the interference object is located in the image is determined according to the motion state, a humanized image interference elimination scheme is implemented, the interference elimination steps are simplified, the use difficulty is reduced, the elimination effect is improved, and the user experience is enhanced.

[0115] Embodiment seven

[0116] Figure 9 is a flowchart of the seventh embodiment of the image interference elimination method of the present application, based on the above-mentioned embodiments, the motion state of the interference object is acquired in real time, and the interference region in which the interference object is located in the image is determined according to the motion state, and the method further comprises the following steps:

[0117] S33, when a photographing instruction is detected in a preview interface of image shooting, the motion state of the interference object before the photographing instruction is extracted.

[0118] S34, distinguishing a motion region and the interference region in the viewfinder interface according to the first motion state and the second motion state.

[0119] Optionally, in the embodiment, for a video recording scene, after distinguishing the background motion region and the interference target, the content of the interference region is removed, and the current frame is restored by an image restoration method.

[0120] The embodiment has the beneficial effects that when a camera shooting viewfinder interface detects a shooting instruction, the first motion state of the foreground object and the second motion state of the interference object are acquired; and a motion region and the interference region are distinguished in the viewfinder interface according to the first motion state and the second motion state. A humanized image interference elimination scheme is implemented, interference elimination steps are simplified, the use difficulty is reduced, the elimination effect is improved, and the user experience is enhanced.

[0121] Embodiment Eight

[0122] Figure 10 is a flowchart of the eighth embodiment of the image interference elimination method of the application, based on the above-mentioned embodiments, the removal of all pixels in the interference region range and the restoration of the pixels in the interference region range according to the pixels of the peripheral region of the interference region range, comprising:

[0123] S41, converging the pixel search range in the interference region range by iterating the image of the peripheral region until the peripheral region corresponding to the interference region is acquired.

[0124] S42, restoring the pixels in the interference region range according to the pixels in the peripheral region.

[0125] Optionally, in the embodiment, the corresponding relationship of the three of the current shooting scene, the foreground object and the interference object is recorded.

[0126] Optionally, in the embodiment, in the next shooting or video recording scene, according to the selection result of the foreground object or the interference object, the corresponding corresponding relationship is called to determine the corresponding interference object or foreground object, thereby providing the user with a more convenient image interference selection and elimination method.

[0127] The embodiment has the beneficial effects that by iterating the image of the peripheral region, the pixel search range in the interference region range is converged until the peripheral region corresponding to the interference region is acquired; and the pixels in the interference region range are restored according to the pixels in the peripheral region. A humanized image interference elimination scheme is implemented, interference elimination steps are simplified, the use difficulty is reduced, the elimination effect is improved, and the user experience is enhanced.

[0128] Embodiment Nine

[0129] Based on the above embodiments, the present application further provides an image interference elimination device, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the image interference elimination method according to any one of the above embodiments.

[0130] It should be noted that the above device embodiments and method embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, and the technical features in the method embodiments are all applicable to the device embodiments, which will not be repeated here.

[0131] Embodiment Ten

[0132] Based on the above embodiments, the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores an image interference elimination program, and the image interference elimination program, when executed by a processor, implements the steps of the image interference elimination method according to any one of the above embodiments.

[0133] It should be noted that the above medium embodiments and method embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, and the technical features in the method embodiments are all applicable to the medium embodiments, which will not be repeated here.

[0134] It should be noted that in this document, the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles or devices including a series of elements not only include those elements, but also include other elements not explicitly listed, or further include elements inherent in such processes, methods, articles or devices. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0135] The above embodiment numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0136] Those skilled in the art can clearly understand the above-mentioned embodiment method can be realized by means of software and necessary general hardware platform, of course, also can be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application essentially or say the part which contributes to the prior art can be embodied in the form of software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a plurality of instructions to make a terminal (may be a mobile phone, computer, server, air conditioner, or network equipment, etc.) execute the method described in various embodiments of the present application.

[0137] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-mentioned specific embodiments, and the above-mentioned specific embodiments are only illustrative, not limiting, and those skilled in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the scope protected by the claims.

Claims

1. A method for eliminating image interference, characterized in that, The method includes: During the shooting process, all shooting objects contained in the image are detected in real time, at least one shooting object is identified as a foreground object among all shooting objects, and the remaining shooting objects are identified as background objects. Based on the relative motion relationship between the foreground object and all the background objects, at least one background object is identified as an interfering object among all the background objects, and the remaining background objects are identified as non-interfering objects. The motion state of the interfering object is acquired in real time, and the interference area where the interfering object is located is determined in the image based on the motion state; Remove all pixels within the interference area, and restore the pixels within the interference area based on the pixels in the surrounding area of ​​the interference area; The step of determining at least one background object as an interfering object and the remaining background objects as non-interfering objects based on the relative motion relationship between the foreground object and all the background objects includes: The background object selection command is received in real time in the preview interface of image capture or in the viewfinder of video capture. At least one of the background objects determined according to the selected instruction is the interference object; Determine the object attributes of the selected interference object, wherein the object attributes include object type and object size; Based on the relative motion relationship, background objects that have the same object attributes and motion states as the selected interference objects are included together as the interference objects; The step of acquiring the motion state of the interfering object in real time and determining the interference area where the interfering object is located in the image based on the motion state includes: When a photo capture command is detected in the image capture preview interface, the motion state of the interfering object prior to the photo capture command is extracted; The interference area where the interfering object is located is determined in the image based on the motion state; When a camera command is detected in the viewfinder of the video recording, the first motion state of the foreground object and the second motion state of the interfering object are acquired. Based on the first motion state and the second motion state, the motion area and the interference area are distinguished within the viewfinder; The step of removing all pixels within the interference region and restoring the pixels within the interference region based on the pixels in the surrounding area of ​​the interference region includes: By iterating over the image of the surrounding area, the pixel search range within the interference area is narrowed until the surrounding area corresponding to the interference area is obtained. The pixels within the interference area are restored based on the pixels in the surrounding area.

2. The image interference cancellation method according to claim 1, characterized in that, The step of detecting all objects in the image during real-time shooting, identifying at least one object as a foreground object, and designating the remaining objects as background objects, includes: Detect the current shooting type, wherein the shooting type includes image shooting and video shooting; All subjects captured in the image are detected and marked in real time in the preview interface of the image capture or in the viewfinder of the video capture.

3. The image interference cancellation method according to claim 2, characterized in that, The step of detecting all objects in the image in real time during the shooting process, identifying at least one of the objects as a foreground object, and designating the remaining objects as background objects, further includes: The method for determining the foreground object is determined, wherein the method includes selection based on user triggering of the marker, or selection based on a preset detection model; In the preview interface of image capture or the viewfinder interface of video capture, the foreground object and the background object are distinguished and marked in real time, and the foreground object and the background object are tracked respectively.

4. An image interference cancellation device, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the image interference cancellation method as described in any one of claims 1 to 3.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an image interference cancellation program, which, when executed by a processor, implements the steps of the image interference cancellation method as described in any one of claims 1 to 3.

Images