Image processing method and device, electronic device and computer readable storage medium

Abstract

The present disclosure is about an image processing method and device. The image processing method is applied to electronic devices, including: using the electronic device to shoot or select a video from an album, the video includes a current frame image, and one or more previous frame images before the current frame image; determining the subject in the video; based on the determined subject, using a semantic segmentation algorithm, obtaining the subject area of ​​the current frame image, and the background area of ​​the current frame image and the previous frame image; based on the background area of ​​the current frame image and the previous frame image, obtaining the afterimage image of the current frame image; fusing the subject area of ​​the current frame image with the afterimage image of the current frame image to obtain the output image of the current frame image. The subject area and the background area are obtained by a semantic segmentation algorithm, and the afterimage image formed by inter-frame fusion is fused with the subject area of ​​the current frame image to obtain the output image, which can be shot and form high-quality pictures or videos with afterimage effects in any environment and conditions.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image processing method, an image processing device, an electronic device, and a computer-readable storage medium. Background Art

[0002] At present, in some shooting needs, time-lapse photography is used to show the image of the trailing effect of moving objects, or afterimage. Time-lapse photography is to extend the exposure time of the image sensor. When the shooting picture changes, the light reflected from the shooting object will continue to be exposed (imaged) on the image sensor to show a continuous moving track. However, due to the limitation of the photosensitivity of the camera equipment, time-lapse photography cannot be used directly under sufficient light conditions, or the image effect is not ideal. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides an image processing method, an image processing device, an electronic device and a computer-readable storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, which is applied to an electronic device, and the image processing comprises: shooting a video using a camera of the electronic device or selecting a video from an album of the electronic device, wherein the video comprises a current frame image and one or more preceding frame images before the current frame image; determining a subject in the video; based on the determined subject, obtaining a subject area of ​​the current frame image and background areas of the current frame image and the preceding frame image through a semantic segmentation algorithm; obtaining an afterimage image of the current frame image based on the background area of ​​the current frame image and the preceding frame image; and fusing the subject area of ​​the current frame image and the afterimage image of the current frame image to obtain an output image of the current frame image.

[0005] In one embodiment, determining a subject in a video includes: determining a subject in any frame in the video in response to an external instruction; and determining the subject in the video through a target tracking algorithm based on the subject in any frame in the video.

[0006] In one embodiment, based on the background areas of the current frame image and the previous frame image, an afterimage image of the current frame image is obtained, including: convolving the background area of ​​the current frame image and the background areas of the current frame image with a preset number of previous frame images in sequence to obtain the afterimage image of the current frame image.

[0007] In one embodiment, based on the background area of ​​the current frame image and the previous frame image, an afterimage image of the current frame image is obtained, including: weighted fusion of the background area of ​​the current frame image and the afterimage image of the previous frame image adjacent to the current frame image to obtain the afterimage image of the current frame image.

[0008] In one embodiment, the main area of ​​the current frame image and the afterimage image of the current frame image are fused to obtain an output image of the current frame image, including: edge fusion of the edge of the main area of ​​the current frame image and the afterimage image of the current frame image.

[0009] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing device, which is applied to an electronic device, and the image processing device includes: an acquisition unit, which is used to shoot a video using a camera of the electronic device or select a video from an album of the electronic device, wherein the video includes a current frame image and one or more preceding frame images before the current frame image; a confirmation unit, which is used to determine a subject in a video frame; a segmentation unit, which is used to obtain a subject area of ​​the current frame image and a background area of ​​the current frame image and the preceding frame image through a semantic segmentation algorithm based on the determined subject; an inter-frame fusion unit, which is used to obtain an afterimage image of the current frame image based on the background area of ​​the current frame image and the preceding frame image; and a current frame image fusion unit, which is used to fuse the subject area of ​​the current frame image with the afterimage image of the current frame image to obtain an output image of the current frame image.

[0010] In one embodiment, the confirmation unit is used to: determine the subject of any frame in the video in response to an external instruction; determine the subject in the video through a target tracking algorithm based on the subject of any frame in the video.

[0011] In one embodiment, the inter-frame fusion unit is used to convolve the background area of ​​the current frame image and the background areas of the current frame image with a preset number of previous frame images in sequence to obtain an afterimage image of the current frame image.

[0012] In one embodiment, the inter-frame fusion unit is used to perform weighted fusion on the background area of ​​the current frame image and the afterimage image of the previous frame image adjacent to the current frame image to obtain the afterimage image of the current frame image.

[0013] In one embodiment, the current frame image fusion unit is used to perform edge fusion on the edge of the main area of ​​the current frame image and the afterimage image of the current frame image.

[0014] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: a memory for storing instructions; and a processor for calling the instructions stored in the memory to execute the image processing method of the first aspect.

[0015] According to a fourth aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium storing instructions, which, when executed by a processor, execute the image processing method of the first aspect.

[0016] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: by determining the subject, and performing a semantic segmentation algorithm based on the determined subject to obtain a subject area and a background area, the background area is fused between frames to form an afterimage image, and then further fused with the subject area of ​​the current frame image to obtain a fused output image, which can ensure the afterimage effect while ensuring the clarity of the subject, without extending the exposure time, and can capture and form high-quality pictures or videos with afterimage effects in any environment and conditions.

[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0019] Fig. 1 is a schematic flow chart of an image processing method according to an exemplary embodiment.

[0020] Fig. 2 is a schematic flow chart of another image processing method according to an exemplary embodiment.

[0021] FIG3 , FIG4 , and FIG5 are schematic diagrams showing a semantic segmentation algorithm according to an exemplary embodiment.

[0022] Fig. 6, Fig. 7 and Fig. 8 are schematic diagrams showing buffering of preceding frame images according to an exemplary embodiment.

[0023] FIG9 is a schematic diagram of the main area based on binary hard segmentation.

[0024] FIG. 10 is a schematic diagram of the main area after edge fusion.

[0025] Fig. 11 is a schematic block diagram showing an image processing apparatus according to an exemplary embodiment.

[0026] Fig. 12 is a schematic block diagram of a device according to an exemplary embodiment.

[0027] Fig. 13 is a schematic block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0028] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0029] In some related technologies, images with afterimage effects are obtained through time-lapse photography. The shooting principle is to extend the exposure time of the image sensor. When the shooting picture changes, the light reflected from the subject will continue to be exposed on the image sensor to show a continuous moving track. When there is sufficient external light, the sensor sensitivity is adjusted to the lowest, and then the minimum amount of light is set, but the picture will still be overexposed.

[0030] In other technologies, grayscale filters are used for time-lapse photography. Grayscale filters are a kind of translucent medium used to reduce brightness during photography. They are a kind of photographic filter. The principle is that when light enters the medium, the light is attenuated, thereby limiting and filtering most of the light, and only allowing a small amount of light to be directly exposed on the sensor for imaging. However, this method requires more hardware and requires mastering higher shooting skills. In addition, this method reduces the amount of light entering, which reduces the color of the picture.

[0031] To solve the above problems, the embodiment of the present disclosure provides an image processing method 10, which can meet the general camera to form a time-lapse afterimage effect, and ensure that the subject is clear and the overall image quality is high. As shown in FIG1, the image processing method 10 can be applied to an electronic device, and the electronic device can have a camera device for shooting a video, and can also have a storage device capable of storing a video, including steps S11 to S15:

[0032] Step S11, using a camera of an electronic device to shoot a video or selecting a video from an album of the electronic device, wherein the video includes a current frame image and one or more preceding frame images before the current frame image.

[0033] In the disclosed embodiment, the video can be obtained by real-time shooting by a camera device of an electronic device, or by retrieving a video stored in a local album of the electronic device, or by receiving a video transmitted by other devices. In order to ensure the effect, the camera device should be kept stable during real-time shooting to avoid full-image smear caused by the movement of the full screen in the captured video, which affects the final image effect.

[0034] Step S12, determining the subject in the video.

[0035] After acquiring continuous videos, the subject in the video is determined. The subject is a person or object that needs to be clearly displayed in the final generated image or video, and no smear effect needs to be generated for it, so the subject in the video needs to be determined.

[0036] In one embodiment, as shown in FIG. 2 , step S12 may include: step S121, in response to an external instruction, determining the subject of any frame in the video; step S122, based on the subject of any frame in the video, determining the subject in the video through a target tracking algorithm. In this embodiment, the user can determine the subject in the frame based on the screen of any frame through an external instruction similar to a tap-to-focus method or other methods. After confirming the subject of the frame, the subject of each frame in the video is determined through a target tracking algorithm. Among them, the target tracking algorithm, also known as target tracking, is a branch in the field of computer vision. The general task is to find a target in the first frame or one of the frames of a given video, and then continue to determine the position of the target in the remaining video frames. The factors affecting the accuracy of the target tracking algorithm technology are mainly: light changes, target shape or angle changes, and target occlusion. Therefore, in order to ensure that the image processing method 10 of the embodiment of the present disclosure can be more reliable and accurate, the shooting process should try to keep the ambient light stable, the angle of the subject relative to the shooting device as consistent as possible, and try to avoid being blocked during the shooting process.

[0037] Step S13, based on the determined subject, a semantic segmentation algorithm is used to obtain the subject area of ​​the current frame image and the background areas of the current frame image and the previous frame image.

[0038] For a video frame, the subject and the background are subjected to a semantic segmentation algorithm, and a subject area is formed by hard segmentation based on the boundary of the subject, and a background area is formed by the remaining part of the picture except the subject area. Among them, the semantic segmentation algorithm can be based on image classification and object detection, and the boundary of the target in an image is determined, so that the targets in the image and the target and the background are decomposed into separate entities. Take Figures 3, 4, and 5 as examples. The target subject in the video frame is a human body. Figure 3 is an image of a certain frame in the video. By segmenting the human body with the semantic segmentation algorithm of the frame image, the segmentation diagram shown in Figure 4 can be obtained. Then, based on the determined subject, the boundary is binary segmented, that is, hard segmented, to obtain a binary segmented image formed by the subject and the background as shown in Figure 5.

[0039] The order of step S12 and step S13 is not fixed. In some embodiments, step S13 may be performed after the subject of all frames is determined based on step S12; in other embodiments, step S12 may be performed for one frame to determine the subject, and then step S13 may be performed for the frame to perform a semantic segmentation algorithm, and then step S12 and step S13 may be performed for the next frame. The order of step S12 and step S13 may also be adjusted according to actual needs.

[0040] Step S14, obtaining an afterimage image of the current frame image based on the background area of ​​each frame of the current frame image and the previous frame image.

[0041] In the background excluding the main area, there are moving targets, such as people, animals, vehicles and other objects. The background area is fused to form the required trailing effect of the background.

[0042] In one embodiment, step S14 may include: convolving the background area of ​​the current frame image and the background areas of the current frame image with a preset number of previous frame images in sequence to obtain an afterimage image of the current frame image.

[0043] In this embodiment, the number of preceding frame images used can be determined according to a preset number, which can be all preceding frame images before the current frame image; or it can be a fixed number, such as five preceding frame images before the current frame image. For the first few frames of the continuous video that cannot meet the fixed number, all preceding frame images can be processed. The cache can be called accordingly, and the preset number of preceding frame images can be stored in the cache. In the process of processing multiple frames, the current frame image is processed and enters the cache, and the first preceding frame image is cleared from the cache at the same time, so that the preset number of preceding frame images is maintained in the cache, and then the next frame is processed. For example, the preset number is five frames, as shown in Figure 6, when the current frame image is the third frame of the video, the first two frames of the current frame image, that is, the first frame and the second frame of the video, are saved in the cache; as the video advances, as shown in Figure 7, when the current frame image is the fifth frame of the video, the current frame image and the first four frames of the current frame image are saved in the cache; as the video continues to advance, as shown in Figure 8, when the current frame image is the sixth frame of the video, since the preset number is five frames, the current frame image is stored in the cache while the frame farthest from the current frame image, that is, the first frame of the video, is deleted from the cache to ensure that five frames are maintained in the cache, so that when step S14 is executed for any current frame image, all video frames in the current cache can be retrieved for processing.

[0044] Among them, the convolution calculation can be an operation of multiplying the image and the filter matrix element by element and then summing them, which is equivalent to moving a two-dimensional function to all positions of another two-dimensional function. In other words, the background areas of multiple frames are weighted averaged to generate a new residual image, wherein the so-called weighted average is to multiply the pixel value of the corresponding coordinate in each background area by the weight corresponding to the frame, and finally calculate the sum and then average to obtain an average pixel value as the pixel value of the pixel at the corresponding coordinate position of the residual image. If it is an image in RGB format, the values ​​of the three channels of R, G, and B are weighted averaged; if it is a grayscale image, the grayscale value is weighted averaged. In addition, in the convolution calculation, the weight of each frame can be the same or different. In one embodiment of the present disclosure, when processing the residual image of the current frame image, the weight of the background area of ​​the current frame image is the highest, and the weight corresponding to the previous frame image decreases as the distance from the current frame image increases. In other words, among multiple previous frame images, the weight corresponding to the previous frame image adjacent to the current frame image is the highest, and then the weight corresponding to the previous frame image of the previous frame is the second highest, and so on. Due to the possible existence of moving targets in the background area, the smear effect is generated based on this. The position of the moving target in the previous frame image adjacent to the current frame image is closest to the position in the current frame image, while the position of the moving target in the previous frame image far away from the current frame image may be far away. Setting a relatively low weight for the moving target far away can better show the smear effect. For example: the preset number is 3, the weight of the background area of ​​the current frame image is 0.5, the weight of the background area of ​​the previous frame image before the current frame image is 0.34, the weight of the background area of ​​the previous previous frame image is 0.14, and the weight of the background area of ​​the previous previous frame image is 0.02. The weight setting can be fitted by any monotonically increasing curve such as normal distribution, multiple buffer functions, etc.

[0045] In another embodiment, step S14 may also include: weighted fusion of the background area of ​​the current frame image and the afterimage image of the previous frame image adjacent to the current frame image to obtain the afterimage image of the current frame image. In the case of continuous processing of multiple frames of images, the processed previous frame image can be used as a data source for subsequent processing, and the afterimage image of the current frame image can also be generated based on the afterimage image of the previous previous frame image. In this embodiment, the afterimage image of each frame can be stored in the cache as the basis for generating the afterimage image of the corresponding subsequent frame. Specifically, when processing the current frame image, the afterimage image of the previous previous frame image can be weightedly fused with the background area of ​​the current frame image, and the corresponding weights can be preset, such as the weight corresponding to the background area of ​​the current frame image is 0.5 or 0.6, and correspondingly, the weight corresponding to the afterimage image of the previous previous frame image is 0.5 or 0.4. Through the method of this embodiment, when processing the afterimage image of each frame, it carries the image information of all the previous frames. At the same time, because the weights are set, the farther the previous frame image is from the current frame image, the lower the effect of the afterimage image of the current frame image. Therefore, it is also possible to easily generate images with high-quality ghosting effects.

[0046] Step S15, fusing the main body area of ​​the current frame image and the residual image of the current frame image to obtain an output image of the current frame image.

[0047] The main area of ​​the current frame image is a clear main area obtained by performing a semantic segmentation algorithm on the current frame image, while the afterimage image of the current frame image is obtained according to the background area of ​​the current frame image and the previous frame image through the method in any of the above embodiments. Finally, the two are fused to form an output image with a clear main body and a moving target in the background having a trailing effect.

[0048] In one embodiment, step S15 may also include: edge fusion of the edge of the main area of ​​the current frame image with the residual image of the current frame image. Since the semantic segmentation algorithm is a hard segmentation, in order to make the output image more natural, the edge of the main area of ​​the current frame image may be edge fused with the residual image, such as by using alpha blending or other image matting methods to blur the edge, so that the generated image is more natural. For example, alpha edge fusion is used to fuse the main area and the residual image, as shown in Figures 9 and 10, where Figure 9 is based on the main body segmented by binary hard segmentation, and the superposition formula of alpha edge fusion is:

[0049] P = α·F + (1-α)·B

[0050] Among them, P is the output value pixel value; F is the foreground pixel value, that is, the main area pixel value in the embodiment of the present disclosure; B is the background pixel value, that is, the residual image pixel value in the embodiment of the present disclosure; α is a parameter, which is equivalent to transparency. If the value is 1, it is equivalent to directly covering the foreground image. Alpha edge fusion is performed through the above formula to form the edge shown in Figure 10, so that the edge of the fused image is not abrupt, but softer, and the finished image effect is better.

[0051] By using any of the above embodiments, an image with a smear effect or a video clip consisting of multiple consecutive video frames can be generated from a generally shot video, with loose shooting conditions, simple operation, and higher quality of the generated image. Pictures and videos with a prominent subject and a smear effect on the background can be previewed and shot in real time with a mobile phone; at the same time, during video shooting, passers-by will be imaged as smears, which creates a strong contrast between movement and stillness while protecting the privacy of pedestrians.

[0052] Based on the same inventive concept, FIG11 shows an image processing device 100, which is applied to an electronic device, and the image processing device 100 includes: an acquisition unit 110, which is used to use a camera device of the electronic device to shoot a video or select a video from an album of the electronic device, wherein the video includes a current frame image and one or more preceding frame images of the current frame image; a confirmation unit 120, which is used to determine the subject in the video; a segmentation unit 130, which is used to obtain the main area of ​​the current frame image and the background area of ​​the current frame image and the preceding frame image through a semantic segmentation algorithm based on the confirmed subject; an inter-frame fusion unit 140, which is used to obtain the afterimage image of the current frame image based on the background area of ​​the current frame image and the preceding frame image; a current frame image fusion unit 150, which is used to fuse the main area of ​​the current frame image and the afterimage image of the current frame image to obtain the output image of the current frame image.

[0053] In one embodiment, the confirmation unit 120 is used to: determine the subject of any frame in the video in response to an external instruction; determine the subject in the video through a target tracking algorithm based on the subject of any frame in the video.

[0054] In one embodiment, the inter-frame fusion unit 140 is used to convolve the background area of ​​the current frame image and the background areas of the current frame image with a preset number of previous frame images in sequence to obtain an afterimage image of the current frame image.

[0055] In one embodiment, the inter-frame fusion unit 140 is used to perform weighted fusion on the background area of ​​the current frame image and the afterimage image of the previous frame image adjacent to the current frame image to obtain the afterimage image of the current frame image.

[0056] In one embodiment, the current frame image fusion unit 150 is used to perform edge fusion on the edge of the main area of ​​the current frame image and the residual image of the current frame image.

[0057] Regarding the image processing device 100 in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0058] Fig. 12 is a schematic block diagram of an apparatus according to any of the above embodiments, according to an exemplary embodiment. For example, the apparatus 300 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0059] 12 , the device 300 may include one or more of the following components: a processing component 302 , a memory 304 , a power component 306 , a multimedia component 308 , an audio component 310 , an input / output (I / O) interface 312 , a sensor component 314 , and a communication component 316 .

[0060] The processing component 302 generally controls the overall operation of the device 300, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 302 may include one or more processors 320 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 302 may include one or more modules to facilitate the interaction between the processing component 302 and other components. For example, the processing component 302 may include a multimedia module to facilitate the interaction between the multimedia component 308 and the processing component 302.

[0061] The memory 304 is configured to store various types of data to support operations on the device 300. Examples of such data include instructions for any application or method operating on the device 300, contact data, phone book data, messages, pictures, videos, etc. The memory 304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0062] The power component 306 provides power to the various components of the device 300. The power component 306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 300.

[0063] The multimedia component 308 includes a screen that provides an output interface between the device 300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 308 includes a front camera and / or a rear camera. When the device 300 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0064] The audio component 310 is configured to output and / or input audio signals. For example, the audio component 310 includes a microphone (MIC), and when the device 300 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 304 or sent via the communication component 316. In some embodiments, the audio component 310 also includes a speaker for outputting audio signals.

[0065] I / O interface 312 provides an interface between processing component 302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0066] The sensor assembly 314 includes one or more sensors for providing various aspects of the status assessment of the device 300. For example, the sensor assembly 314 can detect the open / closed state of the device 300, the relative positioning of components, such as the display and keypad of the device 300, the sensor assembly 314 can also detect the position change of the device 300 or a component of the device 300, the presence or absence of user contact with the device 300, the orientation or acceleration / deceleration of the device 300, and the temperature change of the device 300. The sensor assembly 314 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 314 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 314 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0067] The communication component 316 is configured to facilitate wired or wireless communication between the device 300 and other devices. The device 300 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 316 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 316 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0068] In an exemplary embodiment, the apparatus 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0069] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 304 including instructions, and the instructions can be executed by the processor 320 of the device 300 to perform the above method. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0070] FIG13 is a block diagram of an electronic device 400 according to an exemplary embodiment. For example, the device 400 may be provided as a server. Referring to FIG13 , the device 400 includes a processing component 422, which further includes one or more processors, and a memory resource represented by a memory 442 for storing instructions executable by the processing component 422, such as an application. The application stored in the memory 442 may include one or more modules, each of which corresponds to a set of instructions. In addition, the processing component 422 is configured to execute instructions to perform the above method.

[0071] The device 400 may also include a power supply component 426 configured to perform power management of the device 300, a wired or wireless network interface 450 configured to connect the device 400 to a network, and an input / output (I / O) interface 458. The device 400 may operate based on an operating system stored in the memory 442, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0072] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0073] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. An image processing method, applied to an electronic device, characterized in that: The method comprises: Using a camera device of the electronic device to shoot a video in real time or selecting a video from a photo album of the electronic device, wherein the video includes a current frame image and one or more preceding frame images before the current frame image; identifying a subject in the video; Based on the determined subject, obtaining the subject area of ​​the current frame image, the background area of ​​the current frame image, and the background area of ​​the previous frame image by using a semantic segmentation algorithm; Based on the background area of ​​the current frame image and the background area of ​​the previous frame image, obtaining an afterimage image of the current frame image; Fusing the main area of ​​the current frame image with the residual image of the current frame image to obtain an output image of the current frame image; The obtaining, based on the background area of ​​the current frame image and the background area of ​​the previous frame image, an afterimage image of the current frame image comprises: Convolving the background area of ​​the current frame image and the background areas of the current frame image with a preset number of previous frame images in sequence to obtain an afterimage image of the current frame image; Wherein, obtaining the residual image of the current frame image based on the background area of ​​the current frame image and the background area of ​​the previous frame image includes: The background area of ​​the current frame image and the afterimage image of the previous frame image adjacent to the current frame image are weightedly fused to obtain the afterimage image of the current frame image, wherein the weight of the background area of ​​the current frame image is the highest, and the weight corresponding to the previous frame image decreases as the distance from the current frame image increases to generate an afterimage image.

2. The image processing method according to claim 1, characterized in that: The determining the subject in the video includes: In response to an external instruction, determining the subject of any frame in the video; Based on the subject in any frame in the video, the subject in the video is determined by a target tracking algorithm.

3. The image processing method according to claim 1, characterized in that: The step of fusing the main area of ​​the current frame image with the residual image of the current frame image to obtain an output image of the current frame image includes: The edge of the main area of ​​the current frame image is edge-fused with the afterimage image of the current frame image.

4. An image processing device, characterized in that: Applied to electronic equipment, the device comprises: an acquisition unit, configured to use a camera of the electronic device to shoot a video in real time or select a video from a photo album of the electronic device, wherein the video includes a current frame image and one or more preceding frame images of the current frame image; A confirmation unit, used to determine a subject in the video; A segmentation unit, configured to obtain, based on the determined subject, a subject region of the current frame image, a background region of the current frame image, and a background region of the previous frame image by a semantic segmentation algorithm; An inter-frame fusion unit is used to obtain an afterimage image of the current frame image based on the background area of ​​the current frame image and the background area of ​​the previous frame image; the inter-frame fusion unit is used to: convolve the background area of ​​the current frame image and the background area of ​​the current frame image with a preset number of previous frame images in sequence to obtain the afterimage image of the current frame image; perform weighted fusion on the background area of ​​the current frame image and the afterimage images of the previous frame images adjacent to the current frame image to obtain the afterimage image of the current frame image, wherein the weight of the background area of ​​the current frame image is the highest, and the weight corresponding to the previous frame image decreases as the distance from the current frame image increases, so as to generate an afterimage image; The current frame image fusion unit is used to fuse the main area of ​​the current frame image and the residual image of the current frame image to obtain an output image of the current frame image.

5. The image processing device according to claim 4, characterized in that: The confirmation unit is used for: In response to an external instruction, determining the subject of any frame in the video; Based on the subject in any frame in the video, the subject in the video is determined by a target tracking algorithm.

6. The image processing device according to claim 4, characterized in that: The current frame image fusion unit is used for: The edge of the main area of ​​the current frame image is edge-fused with the afterimage image of the current frame image.

7. An electronic device, characterized in that: include: A memory for storing instructions; as well as A processor, configured to call the instructions stored in the memory to execute the image processing method according to any one of claims 1 to 3.

8. A computer-readable storage medium, characterized in that: Instructions are stored, and when the instructions are executed by a processor, the image processing method according to any one of claims 1 to 3 is executed.

Images