Background replacement method and device, electronic equipment and readable storage medium

Abstract

A background replacement method and device, electronic equipment and computer readable storage medium, wherein the background replacement method comprises: obtaining a to-be-processed image; performing image segmentation on the to-be-processed image through an AI image segmentation algorithm to obtain a first matting result, the first matting result comprising first mask values of each pixel point in the to-be-processed image; performing color difference-based matting processing on the to-be-processed image through a color difference matting algorithm to obtain a second matting result, the second matting result comprising second mask values of each pixel point in the to-be-processed image; determining target mask values of each pixel point in the to-be-processed image according to the first mask values and the second mask values; and performing background replacement on the to-be-processed image according to the target mask values and a target background image to obtain a replacement image. The above background replacement method, device, electronic equipment and computer readable storage medium can achieve the purpose of improving the accuracy of background replacement.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, and particularly relate to a background replacement method and device, electronic equipment and a computer readable storage medium. BACKGROUND

[0002] With the development of the Internet, background replacement applications are very popular. For example, a video is recorded in a green screen background, a person in the video is extracted, and the extracted person is fused with other backgrounds to obtain a new video. In the prior art, background replacement is mainly achieved by the following method: an input image is classified at the pixel level to obtain a background and one or more categories of foreground defined in advance, and a corresponding model is trained by a deep learning method. The foreground and background are distinguished by the trained model, and the foreground is fused with other images, but the edges of the extracted foreground obtained by this method are often jagged, and the larger the size of the image, the larger the jagged edges. Since the foreground and background extracted by the extraction are not accurate, the background replacement is not accurate and the effect is not good. SUMMARY

[0003] Embodiments of the present application provide a background replacement method, device, electronic equipment and computer readable storage medium, which can achieve the purpose of improving the accuracy of background replacement.

[0004] In one aspect, the embodiments of the present application provide a background replacement method, comprising:

[0005] obtaining a to-be-processed image;

[0006] performing image segmentation on the to-be-processed image by an AI image segmentation algorithm to obtain a first extraction result, the first extraction result comprising first mask values of each pixel point in the to-be-processed image;

[0007] performing color difference-based extraction processing on the to-be-processed image by a color difference extraction algorithm to obtain a second extraction result, the second extraction result comprising second mask values of each pixel point in the to-be-processed image;

[0008] determining target mask values of each pixel point in the to-be-processed image according to the first mask values and the second mask values, and performing background replacement on the to-be-processed image according to the target mask values and a target background image to obtain a replacement image.

[0009] In one aspect, the embodiments of the present application also provide a background replacement device, comprising:

[0010] an image obtaining module configured to obtain a to-be-processed image;

[0011] a first matting module configured to perform image segmentation on the to-be-processed image by using an AI image segmentation algorithm to obtain a first matting result, the first matting result comprising first mask values of each pixel point in the to-be-processed image;

[0012] a second matting module configured to perform color difference-based matting processing on the to-be-processed image by using a color difference matting algorithm to obtain a second matting result, the second matting result comprising second mask values of each pixel point in the to-be-processed image;

[0013] an image correction module configured to determine target mask values of each pixel point in the to-be-processed image according to the first mask values and the second mask values, and perform background replacement on the to-be-processed image according to the target mask values and a target background image to obtain a replacement image.

[0014] In an aspect, the present embodiment also provides an electronic device, comprising a memory and a processor, wherein the memory stores executable program codes, and the processor coupled with the memory invokes the executable program codes stored in the memory to execute the background replacement method provided in the above embodiments.

[0015] In an aspect, the present embodiment also provides a computer readable storage medium, which stores a computer program, wherein the computer program, when executed by a processor, performs the background replacement method provided in the above embodiments.

[0016] As can be seen from the above embodiments of the present application, a to-be-processed image is obtained, the to-be-processed image is segmented by using an AI image segmentation algorithm to obtain a first matting result, the first matting result comprising first mask values of each pixel point in the to-be-processed image, color difference-based matting processing is performed on the to-be-processed image by using a color difference matting algorithm to obtain a second matting result, the second matting result comprising second mask values of each pixel point in the to-be-processed image, target mask values of each pixel point in the to-be-processed image are determined according to the first mask values and the second mask values, and background replacement is performed on the to-be-processed image according to the target mask values and a target background image to obtain a replacement image. In this embodiment, the target mask values of each pixel point in the to-be-processed image are determined by using the first mask values and the second mask values obtained by twice matting, which can accurately divide the foreground and the background in the image, avoid the case that the edge effect in the obtained matting result is poor and the matting is inaccurate, and then perform background replacement on the to-be-processed image according to the target mask values and the target background image, which can obtain an accurate replacement image and achieve the purpose of improving the accuracy of background replacement. BRIEF DESCRIPTION OF DRAWINGS

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating the implementation of a background replacement method according to an embodiment of this application;

[0019] Figure 2 This is a schematic diagram of the structure of a background replacement device provided in an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] See Figure 1 , Figure 1 This is a flowchart illustrating the implementation of a background replacement method according to an embodiment of this application. The method can be applied to electronic devices, which can be portable or fixedly installed in a preset location, such as... Figure 1 As shown, the method specifically includes:

[0023] Step S11: Obtain the image to be processed;

[0024] In this embodiment, the image to be processed is the image whose background is to be replaced. For example, the image to be processed can be an image containing a human figure captured by a camera. Alternatively, the image to be processed can be one frame or at least two frames of a live video acquired after the live video has been captured by the camera.

[0025] Furthermore, in an optional embodiment of the present invention, acquiring the image to be processed includes:

[0026] Acquire sampled images;

[0027] The image containing the region of the target object in the sampled image is determined by the target tracking algorithm and is the image to be processed.

[0028] In this embodiment, the sample image can be an image obtained by collecting a video stream, or a local or server-stored image collected in real time or non-real time.

[0029] For example, the sample image is an image of a person, and a person frame is obtained as a region of interest by using a target tracking algorithm, and the region of interest is determined as the image to be processed.

[0030] The target tracking algorithm can be a convolutional neural network (CNN) model, which can specifically include a convolutional layer, a pooling layer, and a fully connected layer.

[0031] Specifically, in implementation, images containing different objects in different states (such as different postures) in different scenes are collected as a training set and a test set, and the pre-constructed CNN model is supervised trained by using the training set and the test set, so that the CNN model obtained by training can detect a target object in a sample image input into the model. The output of the CNN model can be a region (i.e., coordinate value) of the target object in the sample image and a probability value. If the probability value is greater than a preset probability value (such as 90%), the region is determined as the region of the target object.

[0032] Further, after the region of the target object in the sample image is determined, the region can be expanded by a preset range according to the size of the collected image to obtain an image containing the region of the target object, and the image containing the region of the target object is determined as the image to be processed.

[0033] Further, expanding the region by a preset range according to the size of the collected image to obtain an image containing the region of the target object includes:

[0034] The ratio of the collected image to the region of the target object in the sample image is calculated, and the region of the target object in the sample image is expanded according to the ratio to obtain an image containing the region of the target object.

[0035] In this embodiment, the region is expanded by the ratio of the collected image to the region of the target object in the sample image, so that the obtained image containing the region of the target object is not too large (a too large region reduces the efficiency of subsequent matting and background replacement) or too small (a too small region may not completely contain the real target object, thereby affecting the accuracy of matting and background replacement).

[0036] In this embodiment, the image containing the region of the target object is obtained from the sample image by using the target tracking algorithm, which can quickly locate one or at least two target objects in the sample image, thereby reducing interference information, ensuring the accuracy of matting and background replacement, and improving the efficiency of matting and background replacement.

[0037] In other optional embodiments, the image to be processed can also be a region of interest determined according to a preset of the sampling image or selected by a user.

[0038] For example, a live portrait image captured by a camera is acquired, and a center preset as a region of interest is acquired; or an animal image captured by a camera is acquired, and a region selected by a user and larger than an animal contour is determined as a region of interest.

[0039] Further, in an optional embodiment of the present application, after the image containing the target object in the sampling image is determined by the target tracking algorithm, the method further comprises:

[0040] The image to be processed is subjected to contrast enhancement processing or sharpening processing.

[0041] In this embodiment, the contrast enhancement or sharpening processing of the image to be processed can increase the edge sharpness, thereby enhancing the accuracy of the matting and the accuracy of the background replacement.

[0042] In step S12, the image to be processed is subjected to image segmentation by an AI image segmentation algorithm to obtain a first matting result, and the first matting result includes first mask values of each pixel point in the image to be processed.

[0043] In this embodiment, the AI image segmentation algorithm is a model that can divide the foreground and background of the image to be processed; and the first mask value is a value indicating that each pixel point in the image to be processed belongs to the foreground or the background. That is, the image to be processed is subjected to image segmentation by the AI image segmentation algorithm to obtain a first matting result, and the first matting result includes first mask values indicating that each pixel point in the image to be processed belongs to the foreground or the background or a transition region (i.e., cannot be distinguished as belonging to the foreground or the background).

[0044] Specifically, the AI image segmentation algorithm can obtain a probability value of a certain point in the image to be processed belonging to the foreground image or the background image, and then the pixel points with a probability value greater than a first probability are determined as the pixel points of the foreground or the background, and the pixel points with a probability value less than a second probability are determined as the pixel points of the transition region (i.e., cannot be distinguished as belonging to the foreground or the background), wherein the second probability is less than the first probability, thereby distinguishing the foreground, the background and the transition region of the image to be processed.

[0045] Meanwhile, the mask values of each pixel point belonging to the foreground or the background or the transition region obtained by image segmentation are determined, that is, the first mask value of the pixel point belonging to the background in the image to be processed is determined as 0, the first mask value of the pixel point belonging to the background in the image to be processed is determined as 1, and the first mask value of the pixel point belonging to the transition region in the image to be processed is determined as any value between 0 and 1.

[0046] Further, in an optional embodiment of the present application, before the image segmentation of the to-be-processed image is performed by the AI image segmentation algorithm, the method further comprises:

[0047] obtaining a pre-constructed deep learning-based image semantic segmentation model and a training image set;

[0048] training the deep learning-based image semantic segmentation model by using the training image set to obtain an image segmentation model, and determining the image segmentation model as an AI image segmentation algorithm.

[0049] In this embodiment, the pre-constructed deep learning-based image semantic segmentation model can be composed of an encoder and a decoder, and the encoder structure is constructed based on a residual network (ResNets). When training the deep learning-based image semantic segmentation model, the amount of training data can be hundreds of thousands, and the deep learning-based image semantic segmentation model is supervised trained by using the training data. Specifically, the training data contains multiple groups of data, and each group of data includes an original image (such as an image containing a person, a landscape image, and different images of various scenes) and annotation data of the original image.

[0050] In step S13, the color difference matting algorithm is used to perform color difference-based matting processing on the to-be-processed image to obtain a second matting result, and the second matting result includes second mask values of each pixel point in the to-be-processed image.

[0051] In this embodiment, the color difference matting algorithm can also be obtained by training a pre-constructed deep learning algorithm, and is used to perform color difference-based matting processing on the entire to-be-processed image.

[0052] In this embodiment, the second mask value is a value for identifying whether each pixel point in the to-be-processed image belongs to the foreground or the background. That is, the color difference matting algorithm is used to perform image segmentation on the to-be-processed image to obtain a second matting result, and the second matting result includes second mask values for identifying whether each pixel point in the to-be-processed image belongs to the foreground, the background, or a transition region.

[0053] Further, in an optional embodiment of the present application, the color difference matting algorithm is used to perform color difference-based matting processing on the to-be-processed image to obtain a second matting result, and the second matting result includes second mask values of each pixel point in the to-be-processed image, which comprises:

[0054] calculating a difference value between the RGB value of each pixel point in the to-be-processed image and a preset color value;

[0055] determining the second mask value of each pixel point in the to-be-processed image according to the difference value.

[0056] In the embodiment, the R value, the B value and the G value of each pixel point are weighted and calculated, and a difference between the weighted calculation result and a preset color value is calculated. When the difference is greater than a preset difference value, the second mask value is determined as 0 or 1. When the difference is less than the preset difference value, the second mask value is determined as an arbitrary value between 0 and 1.

[0057] According to the embodiment, the second matting result obtained contains the second mask value of each pixel point in the image to be processed. The second mask value can be used to distinguish the foreground and the background of each pixel point in the image to be processed (especially the transition region in the first matting result), thereby reducing the edge sawtooth in the background synthesis and forming a natural transition.

[0058] Further, in an optional embodiment of the present application, the calculation of the difference between the RBG value of each pixel point in the image to be processed and the preset color value comprises:

[0059] If the background of the image to be processed is green, the distance between the G value and the R value and the distance between the G value and the B value of each pixel point in the image to be processed are calculated.

[0060] If the background of the image to be processed is blue, the distance between the B value and the G value and the distance between the B value and the R value of each pixel point in the image to be processed are calculated.

[0061] The distance is compared with the preset color value, and the difference between the RBG value of each pixel point in the image to be processed and the preset color value is determined according to the comparison result.

[0062] Specifically, if the background of the image to be processed is green, that is, the green screen background of the image to be processed, the first distance between the G component and the R component and the second distance between the G component and the B component of each pixel in the image are calculated. Then, the first distance and the second distance are subtracted by a preset parameter constant (i.e. the preset color value) respectively, and then the two values after subtracting the parameter constant are multiplied by a preset coefficient respectively and then fused (for example, the average value is calculated or weighted calculation is performed) to obtain a fusion value. The fusion value is mapped to the interval [0, 1] through a preset sigmod function, and the mapping result is the second mask value. Therefore, the second mask value of each pixel point in the image is a value in the interval [0, 1].

[0063] Similarly, if the background of the image to be processed is blue, that is, the image to be processed is a blue screen background, the first distance between the B component and the G component of each pixel in the image and the second distance between the B component and the R component are calculated, then the first distance and the second distance are respectively subtracted by a preset parameter constant (that is, a preset color value), and then the two values after subtracting the parameter constant are respectively multiplied by a preset coefficient and then fused (for example, the average value is calculated or weighted calculation is performed) to obtain a fusion value, and then the fusion value is mapped to the interval [0, 1] through a preset sigmod function. The mapping result is the second mask value, and the second mask value of each pixel point in the image is a value in the interval [0, 1].

[0064] Step S14, determining the target mask value of each pixel point in the image to be processed according to the first mask value and the second mask value, and replacing the background of the image to be processed according to the target mask value and a target background image to obtain a replacement image.

[0065] Through the embodiment of the present application, the first matting result and the second matting result are combined to determine the target mask value of the image to be processed, and the target mask value is used to finally identify the foreground and the background of the image to be processed. The matting result is more accurate by comprehensively considering the two matting results.

[0066] Further, in an optional embodiment of the present application, the determination of the target mask value of each pixel point in the image to be processed according to the first mask value and the second mask value comprises:

[0067] sequentially judging whether the first mask value of the pixel point in the processing image is a preset mask value;

[0068] if the first mask value of any pixel point in the processing image is the preset mask value, then the target mask value of the any pixel point in the image to be processed is determined as the first mask value;

[0069] if the first mask value of the any pixel point in the processing image is not the preset mask value, then the target mask value of the any pixel point in the image to be processed is determined as the second mask value;

[0070] Specifically, the preset mask value can be 0 or 1.

[0071] Further, each pixel point in the image to be processed can be judged as follows to determine the target mask value of the image to be processed, and then to accurately distinguish which pixel points in the image to be processed belong to the background image and which pixel points belong to the foreground image:

[0072] if (mask1[idx]==0||mask1[idx]==1) {

[0073] dstMask[idx] = mask1[idx];

[0074] } else {

[0075] dstMask[idx] = mask2[idx];

[0076] }

[0077] wherein, mask1[idx] represents the first mask value of the pixel point idx in the first matting result of the image to be processed, mask2[idx] represents the second mask value of the pixel point idx in the second matting result of the image to be processed, and dstMask represents the target mask value of the pixel point idx in the image to be processed.

[0078] According to the target mask value and the target background image, the background of the image to be processed is replaced to obtain a replacement image, and the replacement image after replacement is obtained by calculating each pixel point as follows:

[0079] dstImage[idx] = srcImage[idx] * mask[idx] + backGroundImage[idx] * (1.0 - mask[idx])

[0080] wherein, dstImage[idx] is the pixel value of the pixel point idx (each coordinate in the image has a pixel point) in the replacement image after background replacement, srcImage[idx] is the pixel value of the pixel point idx in the image to be processed, mask[idx] is the target mask value of the pixel point idx in the image to be processed, and backGroundImage[idx] is the pixel value of the pixel point idx in the target background image.

[0081] For example, the image to be processed is an image containing a person standing in front of a green screen, the image to be processed is segmented by an AI image segmentation algorithm to obtain a first matting result, and the first mask value in the first matting result can identify the pixel points of the green screen, the person and the transition region; the image to be processed is processed by a color difference matting algorithm based on color difference to obtain a second matting result, and the second mask value in the second matting result can further accurately identify the green screen and the person, the target mask value of the image to be processed is determined according to the first matting result and the second matting result, i.e., which pixel points in the image to be processed are the person and which pixel points are the green screen, and then the green screen in the image to be processed is replaced by a landscape image based on the target mask value and the landscape image, thereby obtaining a replacement image of the person standing in front of the landscape.

[0082] In the embodiment of the present application, a to-be-processed image is acquired, an AI image segmentation algorithm is used to perform image segmentation on the to-be-processed image to obtain a first matting result, the first matting result comprising first mask values of each pixel point in the to-be-processed image; a color difference matting algorithm is used to perform color difference-based matting processing on the to-be-processed image to obtain a second matting result, the second matting result comprising second mask values of each pixel point in the to-be-processed image; target mask values of each pixel point in the to-be-processed image are determined according to the first mask values and the second mask values, and background replacement is performed on the to-be-processed image according to the target mask values and a target background image to obtain a replacement image. In this embodiment, the target mask values of each pixel point in the to-be-processed image are determined by the first mask values and the second mask values obtained by twice matting, which can accurately divide the foreground and the background in the image, avoid the case that the edge effect in the obtained matting result is poor and the matting is inaccurate, and then perform background replacement on the to-be-processed image according to the target mask values and the target background image, so that an accurate replacement image can be obtained, and the purpose of improving the accuracy of background replacement is achieved.

[0083] Referring to Figure 2 , a structural schematic diagram of a background replacement device provided by an embodiment of the present application is provided. For ease of illustration, only parts related to the embodiments of the present application are shown. The device can be arranged in an electronic device, which can be portable or fixedly installed at a preset position. The device comprises:

[0084] An image acquisition module 201 is configured to acquire a to-be-processed image.

[0085] A first matting module 202 is configured to perform image segmentation on the to-be-processed image by using an AI image segmentation algorithm to obtain a first matting result, the first matting result comprising first mask values of each pixel point in the to-be-processed image.

[0086] A second matting module 203 is configured to perform color difference-based matting processing on the to-be-processed image by using a color difference matting algorithm to obtain a second matting result, the second matting result comprising second mask values of each pixel point in the to-be-processed image.

[0087] A replacement module 204 is configured to determine target mask values of each pixel point in the to-be-processed image according to the first mask values and the second mask values, and perform background replacement on the to-be-processed image according to the target mask values and a target background image to obtain a replacement image.

[0088] In detail, each module in the background replacement device in the embodiment of the present application adopts the same technical means as the background replacement method described above and can produce the same technical effects when used, and thus will not be described herein again. Figure 1 The background replacement method described above produces the same technical effects when used, and thus will not be described herein again.

[0089] In the embodiment of the present application, a to-be-processed image is acquired, an AI image segmentation algorithm is used to perform image segmentation on the to-be-processed image, and a first matting result is obtained, the first matting result comprising first mask values of each pixel point in the to-be-processed image; a color difference matting algorithm is used to perform color difference-based matting processing on the to-be-processed image, and a second matting result is obtained, the second matting result comprising second mask values of each pixel point in the to-be-processed image; target mask values of each pixel point in the to-be-processed image are determined according to the first mask values and the second mask values, background replacement is performed on the to-be-processed image according to the target mask values and a target background image, and a replacement image is obtained. In this embodiment, the target mask values of each pixel point in the to-be-processed image are determined by the first mask values and the second mask values obtained by twice matting, which can accurately divide the foreground and the background in the image, avoid the case that the edge effect in the obtained matting result is poor and the matting is inaccurate, and then perform background replacement on the to-be-processed image according to the target mask values and the target background image, so that an accurate replacement image can be obtained, and the purpose of improving the accuracy of background replacement is achieved.

[0090] Referring to Figure 3 The hardware structure schematic diagram of the electronic device provided by an embodiment of the present application is provided.

[0091] Exemplarily, the electronic device can be any one of various types of computer system devices that are mobile or portable and perform wireless communication. Specifically, the electronic device can be a mobile phone or a smart phone (for example, an iPhone TM-based phone, an Android TM-based phone), a portable game device (for example, Nintendo DS TM, PlayStation Portable TM, Gameboy Advance TM, iPhone TM), a laptop computer, a PDA, a portable Internet device, a music player, and a data storage device, other handheld devices, and devices such as a watch, a headset, a pendant, an earphone, and the like, and the electronic device can also be other wearable devices (for example, a head-mounted device (HMD) such as electronic glasses, electronic clothes, an electronic bracelet, an electronic necklace, an electronic tattoo, an electronic device, or a smart watch).

[0092] The electronic device may also be any one of a plurality of electronic devices, including but not limited to cellular phones, smartphones, other wireless communication devices, personal digital assistants, audio players, other media players, music recorders, video recorders, cameras, other media recorders, radios, medical devices, vehicle transport instruments, calculators, programmable remote controls, pagers, laptop computers, desktop computers, printers, netbooks, personal digital assistants (PDAs), portable multimedia players (PMPs), Moving Image Experts Group (MPEG-1 or MPEG-2) audio layer 3 (MP3) players, portable medical devices, and digital cameras and combinations thereof.

[0093] In some cases, electronic devices can perform multiple functions (e.g., playing music, displaying video, storing pictures, and receiving and sending telephone calls). If desired, electronic devices can be portable devices such as cellular phones, media players, other handheld devices, wristwatches, pendant devices, handset devices, or other compact portable devices.

[0094] like Figure 3 As shown, the electronic device 10 may include a control circuit, which may include a storage and processing circuit 30. The storage and processing circuit 30 may include a memory, such as a hard disk drive, a non-volatile memory (e.g., flash memory or other electronically programmable erasure-limited memory used to form a solid-state drive), a volatile memory (e.g., static or dynamic random access memory), etc., and this embodiment is not limited thereto. The processing circuit in the storage and processing circuit 30 can be used to control the operation of the electronic device 10. This processing circuit may be implemented based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors, power management units, audio codec chips, application-specific integrated circuits (ASICs), display driver integrated circuits, etc.

[0095] The storage and processing circuitry 30 can be used to run software in the electronic device 10, such as an Internet browsing application, a Voice over Internet Protocol (VOIP) phone call application, an email application, a media play application, an operating system function, etc. These software can be used to perform some control operations, such as, based on camera-based image capturing, based on ambient light sensor-based ambient light measurement, based on proximity sensor-based proximity sensor measurement, based on status indicator such as light emitting diode to implement information display function, based on touch sensor to detect touch event, associated with displaying information on multiple (e.g., layered) displays, associated with performing wireless communication function, associated with collecting and producing audio signals, associated with collecting and processing button press event data, and other functions in the electronic device 10, etc., without limitation.

[0096] Further, the memory stores executable program code, and a processor coupled to the memory, invokes the executable program code stored in the memory to perform the background replacement method described in the above Figure 1 embodiments.

[0097] The executable program code includes each module in the background replacement apparatus described in the above Figure 2 embodiments, such as: an image obtaining module, a first matting module, a second matting module, and a replacing module.

[0098] The electronic device 10 can also include input / output circuitry 42. The input / output circuitry 42 can be used to enable the electronic device 10 to implement input and output of data, i.e., to allow the electronic device 10 to receive data from external devices and also to allow the electronic device 10 to output data from the electronic device 10 to external devices. The input / output circuitry 42 can further include the sensors 32. The sensors 32 can include an ambient light sensor, a light and capacitance-based proximity sensor, a touch sensor (e.g., a light-based touch sensor and / or a capacitive touch sensor, where the touch sensor can be part of a touch display screen or can be used independently as a touch sensor structure), an acceleration sensor, and other sensors, etc.

[0099] The input / output circuitry 42 can also include one or more displays, such as the display 14. The display 14 can include a liquid crystal display, an organic light emitting diode display, an electronic ink display, a plasma display, a combination of one or more of these or other display technologies. The display 14 can include a touch sensor array (i.e., the display 14 can be a touch screen display). The touch sensor can be a capacitive touch sensor formed from an array of transparent touch sensor electrodes (e.g., indium tin oxide (ITO) electrodes), or can be a touch sensor formed using other touch technologies, such as acoustic wave touch, pressure sensitive touch, resistive touch, optical touch, etc.

[0100] The electronic device 10 can also include audio components 36. The audio components 36 can be used to provide audio input and output functionality for the electronic device 10. The audio components 36 in the electronic device 10 can include speakers, microphones, buzzers, tone generators, and other components for producing and detecting sound.

[0101] The communication circuitry 38 can be used to provide the electronic device 10 with the ability to communicate with external devices. The communication circuitry 38 can include analog and digital input / output interface circuitry, and wireless communication circuitry based on radio frequency signals and / or optical signals. The wireless communication circuitry in the communication circuitry 38 can include radio frequency transceiver circuitry, power amplifier circuitry, low noise amplifiers, switches, filters, and antennas. For example, the wireless communication circuitry in the communication circuitry 38 can include circuitry for supporting Near Field Communication (NFC) by transmitting and receiving near-field coupled electromagnetic signals. For example, the communication circuitry 38 can include a near-field communication antenna and a near-field communication transceiver. The communication circuitry 38 can also include cellular telephone transceiver and antenna circuitry, wireless local area network transceiver circuitry and antenna, etc.

[0102] The electronic device 10 can also further include a battery, power management circuitry, and other input / output units 40. The input / output units 40 can include buttons, joysticks, click wheels, scroll wheels, touch pads, key pads, keyboards, cameras, light emitting diodes and other status indicators, etc.

[0103] A user can input commands to control the operation of the electronic device 10 through the input / output circuitry 42, and can use output data from the input / output circuitry 42 to implement receiving status information and other output from the electronic device 10.

[0104] Further, the embodiment of the present application also provides a computer readable storage medium, which can be arranged in the electronic device in the above-mentioned embodiments, and the computer readable storage medium can be the aforementioned Figure 3The memory in the storage and processing circuit 30 in the embodiment. The computer readable storage medium stores a computer program, which is executed by the processor to realize the foregoing Figure 1 The background replacement method described in the embodiment. Further, the computer readable storage medium can also be a U disk, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk or an optical disk, and various storage program codes.

[0105] It should be noted that, for the foregoing method embodiments, in order to facilitate description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0106] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0107] The above is the description of the background replacement method, device and computer readable storage medium provided by the present application. For those skilled in the art, according to the idea of the embodiment of the present application, there will be changes in specific implementation and application range. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A background replacement method, characterized in that, The method includes: Obtain the image to be processed; The image to be processed is segmented by an AI image segmentation algorithm to obtain a first matting result, which includes the first mask value of each pixel in the image to be processed. The image to be processed is processed by color difference matting algorithm to obtain a second matting result, which includes the second mask value of each pixel in the image to be processed. The target mask value of each pixel in the image to be processed is determined based on the first mask value and the second mask value. The background of the image to be processed is replaced based on the target mask value and the target background image to obtain a replacement image. The step of determining the target mask value of each pixel in the image to be processed based on the first mask value and the second mask value includes: Sequentially determine whether the first mask value of each pixel in the processed image is a preset mask value; If the first mask value of any pixel in the processed image is a preset mask value, then the target mask value of any pixel in the image to be processed is determined to be the first mask value. If the first mask value of any pixel in the processed image is not a preset mask value, then the target mask value of any pixel in the image to be processed is determined to be the second mask value. The step of performing color difference-based masking on the image to be processed using a color difference masking algorithm to obtain a second masking result including the second mask value of each pixel in the image to be processed includes: Calculate the difference between the RGB value of each pixel in the image to be processed and the preset color value; Based on the difference, a second mask value is determined for each pixel in the image to be processed, and the second matting result includes the second mask value for each pixel in the image to be processed.

2. The method according to claim 1, characterized in that, The calculation of the difference between the RGB value of each pixel in the image to be processed and the preset color value includes: If the background of the image to be processed is green, then calculate the distance between the G value and R value, and between the G value and B value of each pixel in the image to be processed; If the background of the image to be processed is blue, then calculate the distance between the B value and the G value, and between the B value and the R value of each pixel in the image to be processed; The distance is compared with a preset color value, and the difference between the RGB value of each pixel in the image to be processed and the preset color value is determined based on the comparison result.

3. The method according to claim 1 or 2, characterized in that, The acquisition of the image to be processed includes: Acquire sampled images; The image containing the region of the target object in the sampled image is determined by the target tracking algorithm and is the image to be processed.

4. The method according to claim 3, characterized in that, After determining the image containing the region of the target object in the sampled image using the target tracking algorithm as the image to be processed, the method further includes: The image to be processed is subjected to contrast enhancement or sharpening processing.

5. The method according to claim 1 or 2, characterized in that, Before performing image segmentation on the image to be processed using the AI ​​image segmentation algorithm, the method further includes: Obtain a pre-built deep learning-based image semantic segmentation model and a training image set; The deep learning-based image semantic segmentation model is trained using the training image set to obtain an image segmentation model, which is then identified as an AI image segmentation algorithm.

6. A background replacement device, characterized in that, The device includes: The image acquisition module is used to acquire the image to be processed. The first matting module is used to perform image segmentation on the image to be processed using an AI image segmentation algorithm to obtain a first matting result, wherein the first matting result includes the first mask value of each pixel in the image to be processed. The second matting module is used to perform color difference-based matting on the image to be processed using a color difference matting algorithm to obtain a result. The second matting result includes the second mask value of each pixel in the image to be processed. The replacement module is used to determine the target mask value of each pixel in the image to be processed based on the first mask value and the second mask value, and to perform background replacement on the image to be processed based on the target mask value and the target background image to obtain a replacement image; The step of determining the target mask value of each pixel in the image to be processed based on the first mask value and the second mask value includes: Sequentially determine whether the first mask value of each pixel in the processed image is a preset mask value; If the first mask value of any pixel in the processed image is a preset mask value, then the target mask value of any pixel in the image to be processed is determined to be the first mask value. If the first mask value of any pixel in the processed image is not a preset mask value, then the target mask value of any pixel in the image to be processed is determined to be the second mask value. The step of performing color difference-based masking on the image to be processed using a color difference masking algorithm to obtain a second masking result including the second mask value of each pixel in the image to be processed includes: Calculate the difference between the RGB value of each pixel in the image to be processed and the preset color value; Based on the difference, a second mask value is determined for each pixel in the image to be processed, and the second matting result includes the second mask value for each pixel in the image to be processed.

7. An electronic device, characterized in that, The electronic device includes: Memory and processor; The memory stores executable program code; The processor coupled to the memory invokes the executable program code stored in the memory to execute the background replacement method as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the background replacement method as described in any one of claims 1 to 5.

Images