Image processing method and apparatus, electronic device and storage medium

Abstract

An image processing method and apparatus, an electronic device and a storage medium. The method comprises: displaying on an image processing interface an image to be processed (S110); in response to a material selection operation on the image processing interface, displaying a target material at a corresponding position of said image (S120); in response to a projection trigger operation on the image processing interface, determining coordinate offset information of the target material on the basis of said image (S130); and, on the basis of the coordinate offset information, fusing the target material and said image to obtain a fused image, and displaying on the image processing interface the fused image, the target material in the fused image exhibiting a distorted state (S140) so as to simulate a realistic projection effect.

Description

Image processing method and device, electronic device, and storage medium

[0001] This application claims priority to Chinese Patent Application No. 202410868984.9, filed on June 28, 2024, the disclosure of which is incorporated herein in its entirety as part of the present application. TECHNICAL FIELD

[0002] Embodiments of the present disclosure relate to an image processing method, device, electronic device, and storage medium. BACKGROUND

[0003] With the development of computer technology, the way of processing images is becoming more and more diversified. For example, the image processing method can be to add specific materials to the original image to enrich the image effect.

[0004] Currently, at least one material such as an image, text, and a sticker can be directly superimposed on the original image to obtain a fusion image. However, the direct superimposition method makes the materials in the fusion image and the original image independent of each other, resulting in poor image effects. SUMMARY

[0005] The present disclosure provides an image processing method, device, electronic device, and storage medium, which can make the materials associated with the to-be-processed image produce a distortion effect and improve the image effect.

[0006] In a first aspect, embodiments of the present disclosure provide an image processing method, comprising:

[0007] displaying a to-be-processed image in an image processing interface;

[0008] in response to a material selection operation in the image processing interface, displaying a target material at a corresponding position of the to-be-processed image;

[0009] in response to a projection trigger operation in the image processing interface, determining coordinate offset information of the target material based on the to-be-processed image;

[0010] fusing the target material and the to-be-processed image based on the coordinate offset information to obtain a fusion image, and displaying the fusion image in the image processing interface, wherein the target material in the fusion image is in a distorted state.

[0011] In a second aspect, embodiments of the present disclosure further provide an image processing device, comprising:

[0012] an image display module configured to display a to-be-processed image in an image processing interface;

[0013] a material display module configured to display a target material at a corresponding position of the image to be processed in response to a material selection operation in the image processing interface;

[0014] an offset determination module configured to determine coordinate offset information of the target material based on the image to be processed in response to a projection trigger operation in the image processing interface;

[0015] an image fusion model configured to fuse the target material and the image to be processed based on the coordinate offset information to obtain a fused image, and display the fused image in the image processing interface, wherein the target material in the fused image is in a distorted state.

[0016] In a third aspect, the embodiments of the present disclosure further provide an electronic device, which comprises:

[0017] one or more processors;

[0018] a storage device configured to store one or more programs,

[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method according to any embodiment of the present disclosure.

[0020] In a fourth aspect, the embodiments of the present disclosure further provide a storage medium containing computer executable instructions, which, when executed by a computer processor, are used to perform the image processing method according to any embodiment of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0021] The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent upon reading the following detailed description in conjunction with the accompanying drawings, in which like reference numerals refer to like elements. It is to be understood that the drawings are designed solely for the purpose of illustration and not as a definition of the limits of the disclosure, for which reference should be made only to the appended claims. Throughout the drawings, like reference numerals indicate like elements.

[0022] FIG. 1 is a flowchart of an image processing method according to an embodiment of the present disclosure;

[0023] FIG. 2 is a schematic diagram of an image processing interface according to an embodiment of the present disclosure;

[0024] FIG. 3 is a schematic diagram of another image processing interface according to an embodiment of the present disclosure;

[0025] FIG. 4 is a flowchart of another image processing method according to an embodiment of the present disclosure;

[0026] FIG. 5 is a flowchart of yet another image processing method according to an embodiment of the present disclosure;

[0027] FIG. 6 is a structural schematic diagram of an image processing apparatus according to an embodiment of the present disclosure; and

[0028] FIG. 7 is a structural schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] Embodiments of the present disclosure will be described in more detail with reference to the drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein, but rather the embodiments are provided so that the present disclosure can be more thoroughly and completely understood. It should be understood that the drawings and embodiments of the present disclosure are merely for illustrative purposes and are not intended to limit the scope of protection of the present disclosure.

[0030] It should be understood that each of the steps recited in the method embodiments of the present disclosure can be executed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the execution of the steps shown. The scope of the present disclosure is not limited in this respect.

[0031] The term "comprising" and variations thereof as used in the present disclosure are open-ended, that is, "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Related definitions will be given in the description below.

[0032] It should be noted that the terms "first", "second", and the like mentioned in the present disclosure are merely used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.

[0033] It should be noted that the adjectives "one", "more" mentioned in the present disclosure are illustrative and not limiting, and those skilled in the art should understand that "one or more" should be understood unless the context clearly indicates otherwise.

[0034] The names of the messages or information exchanged between the devices in the embodiments of the present disclosure are merely for illustrative purposes and are not intended to limit the scope of the messages or information.

[0035] It can be understood that, before using the technical solutions disclosed in the embodiments of the present disclosure, the type of personal information involved in the present disclosure, the scope of use, the use scenario, etc. should be informed to the user and the authorization of the user should be obtained in accordance with relevant laws and regulations.

[0036] For example, in response to receiving an active request of a user, a prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed by the user will require obtaining and using personal information of the user. Thus, the user can autonomously select whether to provide the personal information to the software or hardware, such as an electronic device, an application program, a server or a storage medium, performing the operation of the technical solution of the present disclosure according to the prompt information.

[0037] As an optional but non-limiting implementation, in response to receiving an active request of a user, the prompt information can be sent to the user in the form of a pop-up window, in which the prompt information can be presented in the form of text. In addition, the pop-up window can also carry selection controls for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0038] It can be understood that the above notification and obtaining of user authorization process is only illustrative and does not limit the implementation of the present disclosure. Other ways that meet the relevant laws and regulations can also be applied to the implementation of the present disclosure.

[0039] It can be understood that the data involved in the technical solution of the present disclosure (including but not limited to the data itself, the obtaining or use of the data) should comply with the requirements of the relevant laws and regulations and the relevant provisions.

[0040] FIG. 1 is a flowchart of an image processing method provided by an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to a case of adding a target material to a to-be-processed image, for example, fusing the target material into the to-be-processed image to present the effect of material projection on the to-be-processed image. The method can be executed by an image processing device, which can be implemented in the form of software and / or hardware. Optionally, the image processing device is implemented by an electronic device, which can be a mobile terminal, a PC terminal or a server, etc.

[0041] As shown in FIG. 1, the method comprises:

[0042] S110, displaying a to-be-processed image in an image processing interface.

[0043] The image processing interface represents an interactive interface of an image processing tool. The image processing tool can be a client, an applet or a webpage, etc. that implements an image processing function. The image processing includes at least one operation of adding an image, adding text and adding a sticker to the to-be-processed image. The to-be-processed image can be an original image to be fused with a material. The to-be-processed image can be loaded from a local storage to the image processing interface. Alternatively, the to-be-processed image can also be downloaded from the cloud, etc.

[0044] Exemplarily, the image processing interface can be configured to display the to-be-processed image and a candidate control set, the candidate control set including a material selection control and a projection trigger control, where the material selection control is configured to trigger a material selection operation, and the projection trigger control is configured to trigger a projection trigger operation.

[0045] In the embodiments of the present disclosure, when the image processing interface is rendered, the texture information of the to-be-processed image is rendered in a set region of the image processing interface, so that after the rendering is completed, the image processing interface is displayed on the screen of the electronic device, and the set region of the image processing interface is the to-be-processed image.

[0046] S120, in response to the material selection operation in the image processing interface, displaying a target material at a corresponding position of the to-be-processed image.

[0047] The material selection operation is used to represent an operation of selecting a material to be added to the to-be-processed image. The material selection operation can be triggered by a click operation on the material selection control. For example, a material selection window is displayed by clicking the material selection control in the image processing interface. The material selection window includes candidate materials. Based on a selection operation on the candidate materials, a target material is determined. The target material represents the selected candidate material. For example, the target material includes at least one of an image, text, and a sticker. The material can include a picture or a video. The image included in the target material can be a picture taken, a picture downloaded from the network, or a picture generated by a generation model.

[0048] The corresponding position of the to-be-processed image can represent a set position in the to-be-processed image. For example, the corresponding position of the to-be-processed image can be determined based on the center of the to-be-processed image. After the target material is selected, a material layer can be generated based on the target material, and the material layer is superimposed on the corresponding layer of the to-be-processed image, and the position of the material layer is determined based on the center of the to-be-processed image. FIG. 2 is a schematic diagram of an image processing interface provided by an embodiment of the present disclosure. As shown in FIG. 2, a to-be-processed image 220 is displayed in an image processing interface 200. A candidate material is displayed by clicking an imported picture control. After a target material is selected from the candidate material, a material layer 210 corresponding to the target material is displayed in the image processing interface 200. The to-be-processed image in FIG. 2 is a picture generated by a generation model.

[0049] It should be noted that the width and height of the material layer are determined based on the resolution of the target material. The width and height of the material layer can be smaller than the width and height of the layer corresponding to the to-be-processed image. The width and height of the material layer can also be equal to the width and height of the layer corresponding to the to-be-processed image. In some cases, the width and height of the material layer can be greater than the width and height of the layer corresponding to the to-be-processed image. The target material can be adjusted to the expected position by scaling or moving the material layer and the like. The expected position can be an addition position of the target material to the to-be-processed image.

[0050] Exemplarily, in response to the interactive operation on the material selection control, a target layer is superimposed above the layer corresponding to the to-be-processed image, and the target layer includes the target material.

[0051] The interactive operation on the material selection control can represent an operation of triggering the material selection control. For example, the interactive operation includes a click operation, a voice operation, a gesture operation, or a gaze operation, and the like. Based on the click operation on the material selection control, a material layer is generated based on the target material, and the width and height of the material layer are determined based on the resolution of the target material. According to the arrangement order of the layer, the material layer is superimposed above the layer corresponding to the to-be-processed image, so that the target material at least partially occludes the to-be-processed image. Optionally, the target material can also be scaled, shifted, rotated, or mirrored, and the like.

[0052] Optionally, a material box is displayed at the edge of the target material in the material layer. The material box includes a mirroring control and a drag-and-rotate control. The mirroring control is used to realize the mirroring of the target material. The drag-and-rotate control is used to realize the rotation of the target material.

[0053] S130, in response to a projection trigger operation in the image processing interface, determining coordinate offset information of the target material based on the to-be-processed image.

[0054] The projection trigger operation is used to represent an operation of triggering the projection of the target material to the to-be-processed image. The projection trigger operation can be triggered by a click operation on the projection trigger control. For example, a projection trigger control in the image processing interface is clicked, and a fusion image is displayed in the image processing interface. In the fusion image, the target material is in a distorted state. The effect of the fusion image can be considered as projecting the target material to the to-be-processed image, so that the target material is distorted at different depths in the to-be-processed image. For example, the text material is projected to the face, and the nose part is distorted to better fit the nose contour.

[0055] The coordinate offset information represents an offset vector of a coordinate of a pixel in the target material. The coordinate offset information is used to determine the coordinate of a pixel in the target material that is to be fused with a pixel in the image to be processed. The offset vector represents an offset direction and an offset distance, etc. If no coordinate offset is performed, the pixel a in the image to be processed is directly fused with the pixel a' at the corresponding position in the target material, which can result in poor fusion image effect. In order to improve the image effect, the pixel b near the pixel a' in the target material can be fused with the pixel a by performing coordinate offset on the target material. The pixel b can be considered as a pixel in the target material determined in combination with the coordinate offset information. Since the offset vectors of the coordinates of different material pixels in the target material are inconsistent, the target material is equivalent to being subjected to a distortion process.

[0056] For example, for the pixel a in the image to be processed, the pixel coordinate of the pixel a is (0.5, 0.5). If no offset is performed, the pixel a' with the material coordinate (0.5, 0.5) is determined as the material pixel to be fused. In the embodiment of the present disclosure, the material coordinate is offset, and after the offset, the pixel b with the material coordinate (0.5001, 0.5002) can be determined as the material pixel to be fused.

[0057] In the embodiment of the present disclosure, the coordinate offset information can be determined based on attribute information of the image to be processed. The attribute information can include depth information and / or color information, etc. The depth information can represent the depth of a first pixel in a first region in the image to be processed. The first region can represent a face region. For example, the face region can include a human face region, an animal face region, a virtual person face region, or a cartoon character face region, etc. The first pixel can represent a pixel in the face region. For example, the first pixel can represent a pixel in the human face region, a pixel in the animal face region, a pixel in the virtual person face region, or a pixel in the cartoon character face region, etc. The color information can represent the color value of a second pixel in a second region in the image to be processed. The color value can represent the numerical value of a color channel of the second pixel. The second region can represent a non-face region. For example, the non-face region can include a non-face region of a person, a non-face region of an animal, a non-face region of a virtual person, a non-face region of a cartoon character, a plant, or a fruit, etc. The second pixel represents a pixel in the non-face region.

[0058] For example, in response to the interaction operation on the projection trigger control, the coordinate offset information of the target material is determined based on the attribute information of the pixel in the image to be processed.

[0059] For example, in response to the interaction operation on the projection trigger control, the coordinate offset information of the target material is determined based on the attribute information of the pixel in the image to be processed.

[0060] In the embodiments of the present disclosure, the coordinate offset information includes first offset information and second offset information.

[0061] For example, the coordinate offset information of the target material is determined based on the attribute information of the pixels in the to-be-processed image, including: for a first pixel in a first region in the to-be-processed image, determining first offset information corresponding to a first candidate coordinate in the target material according to depth information of the first pixel, wherein the first candidate coordinate is determined based on a pixel coordinate of the first pixel.

[0062] For example, the depth information of the first pixel represents depth data corresponding to the position of the first pixel in a depth map corresponding to the first region. Since the material layer corresponding to the target material is superimposed on the layer corresponding to the to-be-processed image, the first candidate coordinate at the same position in the target material can be queried based on the pixel coordinate of the first pixel. For example, the first candidate coordinate is a coordinate in the target material corresponding to the pixel coordinate of the first pixel. The first offset information represents the offset direction and offset distance of the material coordinate of the target material corresponding to the face region.

[0063] For example, for a first pixel in a face region in the to-be-processed image, a first candidate coordinate of a pixel corresponding to the position of the first pixel in the target material is determined. According to the depth information of the first pixel, the first offset information corresponding to the first candidate coordinate is calculated.

[0064] For example, the coordinate offset information of the target material is determined based on the attribute information of the pixels in the to-be-processed image, including: for a second pixel in a second region in the to-be-processed image, determining second offset information corresponding to a second candidate coordinate in the target material according to color information of the second pixel, wherein the second candidate coordinate is determined based on a pixel coordinate of the second pixel.

[0065] For example, the color information of the second pixel represents color values of the pixels in the second region. Since the material layer corresponding to the target material is superimposed on the layer corresponding to the to-be-processed image, the second candidate coordinate at the same position in the target material can be queried based on the pixel coordinate of the second pixel.

[0066] For example, for a second pixel in a non-face region in the to-be-processed image, a second candidate coordinate of a pixel corresponding to the position of the second pixel in the target material is determined. According to the color value of the second pixel, the second offset information corresponding to the second candidate coordinate is calculated.

[0067] For example, in response to a click operation of the projection trigger control in the image processing interface, for a first pixel in the first region, a material coordinate of a corresponding position in the target material is determined according to the pixel coordinate of the first pixel. Coordinate offset information corresponding to the material coordinate is determined based on the depth information of the first pixel. For a second pixel in the second region, a material coordinate of a corresponding position in the target material is determined according to the pixel coordinate of the second pixel. Coordinate offset information corresponding to the material coordinate is determined based on the color information of the second pixel.

[0068] In S140, the target material and the image to be processed are fused based on the coordinate offset information to obtain a fused image, and the fused image is displayed in the image processing interface, wherein the target material in the fused image presents a distorted state.

[0069] The fused image can represent an image obtained by fusing the target material and the image to be processed in the pixel dimension. For example, for a pixel in the image to be processed, the color value of the pixel and the color value of a target material pixel to be fused are fused to obtain a fused image. The target material pixel represents a pixel in the target material after the material coordinate is distorted.

[0070] For example, for a first pixel in the first region, the material coordinate of a corresponding position in the target material is offset based on the depth information of the first pixel, a target material pixel corresponding to the offset material coordinate is obtained, and the first pixel and the target material pixel are fused. For a second pixel in the second region, the material coordinate of a corresponding position in the target material is offset based on the color information of the second pixel, a target material pixel corresponding to the offset material coordinate is obtained, and the second pixel and the target material pixel are fused.

[0071] FIG. 3 is a schematic diagram of another image processing interface provided by an embodiment of the present disclosure. As shown in FIG. 3, a fused image 310 is displayed in an image processing interface 300. The target material in the fused image 310 presents a distorted deformation effect near the nose and mouth of a person, achieving the effect of material adhering to the face contour. The target material also presents a distorted deformation effect near the clothes and neck of the person. The image processing interface 300 also includes a deformation control control and a transparency control control. In response to an adjustment operation of the deformation control control, the distortion degree of the target material in the fused image 310 is adjusted. In response to an adjustment operation of the transparency control control, the transparency of the target material in the fused image 310 is adjusted.

[0072] The technical scheme of the embodiments of the present disclosure is to display a to-be-processed image in an image processing interface, display a target material at a corresponding position of the to-be-processed image based on a material selection operation in the image processing interface, determine coordinate offset information of the target material according to the to-be-processed image based on a projection trigger operation in the image processing interface, perform distortion processing on the target material through the coordinate offset information to simulate a real projection effect, and obtain a fused image by fusing the target material and the to-be-processed image based on the coordinate offset information and display the fused image in the image processing interface. The embodiments of the present disclosure solve the problem that materials and original images are isolated from each other in a fused image, resulting in poor image effects, and can improve image effects.

[0073] FIG. 4 is a flowchart of another image processing method provided by the embodiments of the present disclosure. The embodiments of the present disclosure specifically limit the obtaining of a fused image by fusing the target material and the to-be-processed image based on the coordinate offset information on the basis of the above-mentioned embodiments.

[0074] As shown in FIG. 4, the method includes:

[0075] S410, displaying a to-be-processed image in an image processing interface.

[0076] S420, in response to a material selection operation in the image processing interface, displaying a target material at a corresponding position of the to-be-processed image.

[0077] S430, in response to a projection trigger operation in the image processing interface, determining coordinate offset information of the target material based on the to-be-processed image.

[0078] S440, for a pixel in the to-be-processed image, determining a target material coordinate according to the coordinate offset information.

[0079] The target material coordinate represents a coordinate of a target material pixel in the target material that is fused with the pixel.

[0080] For example, for a pixel in the to-be-processed image, a target material coordinate is determined according to the coordinate offset information and a candidate coordinate of the pixel in the target material. The candidate coordinate represents a material coordinate in the target material that is the same as the pixel position in the to-be-processed image. For example, the candidate coordinate includes a first candidate coordinate and a second candidate coordinate. The target material coordinate is a coordinate obtained by offsetting the candidate coordinate.

[0081] For a first pixel in a face region in the to-be-processed image, first offset information corresponding to a first candidate coordinate in the target material is determined according to depth information of the first pixel. For a second pixel in a non-face region in the to-be-processed image, second offset information corresponding to a second candidate coordinate in the target material is determined according to color information of the second pixel.

[0082] Determine a target material coordinate of a target material pixel fused with the first pixel in combination with the first candidate coordinate and the first offset information. The target material coordinate of the target material pixel fused with the first pixel is located in the vicinity of the first candidate coordinate. Fusing the target material pixel to the first pixel can present an effect that the target material is projected to the face region of the to-be-processed image in a distorted state.

[0083] Determine a target material coordinate of a target material pixel fused with the second pixel in combination with the second candidate coordinate and the second offset information. The target material coordinate of the target material pixel fused with the second pixel is located in the vicinity of the second candidate coordinate. Superimposing the target material pixel to the second pixel can present an effect that the target material is projected to the non-face region of the to-be-processed image in a distorted state.

[0084] S450, fuse the target material to the to-be-processed image based on the target material coordinate to obtain a fused image.

[0085] Illustratively, for a pixel in the to-be-processed image, color information of a target material pixel is obtained according to a target material coordinate corresponding to the pixel, the color information of the target material pixel is fused with color information of the pixel to obtain an initial fused image, and a light-emitting special effect is added to the initial fused image to obtain a fused image.

[0086] The initial fused image represents a fusion result of the to-be-processed image and the target material. For example, a layer mixing manner can be used to fuse the pixel in the to-be-processed image and the target material pixel after the coordinate offset to obtain a fused layer, and the fused layer includes the fused image.

[0087] In the embodiments of the present disclosure, different fusion modes are set in advance, and the fusion mode represents a layer mixing mode. For example, the fusion mode includes positive superimposition, soft light, superimposition, color filter, and linear lightening, etc. Different fusion modes correspond to different layer mixing formulas. The layer mixing formula is determined according to the fusion mode, and the color value of the pixel in the to-be-processed image and the color value of the target material pixel are fused by using the layer mixing formula to obtain an initial fused image.

[0088] Further, adding a light-emitting special effect to the initial fused image to obtain a fused image includes:

[0089] Perform edge extraction processing on the initial fused image to obtain an edge image. Perform blur processing on the edge image to obtain a blurred edge image. Determine a light-emitting texture image in combination with the initial fused image and the blurred edge image, perform blur processing on the light-emitting texture image to obtain a blurred light-emitting texture image. Determine a light-emitting color based on the light-emitting texture image and the blurred light-emitting texture image, and superimpose the light-emitting color to the initial fused image to obtain a fused image.

[0090] Fig. 5 is a flow diagram of another image processing method according to an embodiment of the present disclosure. The projection implementation process is introduced by taking the example of mixing the text material layer using the projection overlay mode with the equal-sized image to be processed. It should be noted that the text content in the text material layer in Fig. 5 does not constitute a limitation on the undisclosed technical solution, and any content of the text can be mixed with the image to be processed. The text content in the text layer does not affect the understanding of the solution.

[0091] As shown in FIG. 5, a foreground region of the to-be-processed image 510 is extracted by using a background segmentation algorithm, and a foreground mask 520 is generated based on the foreground region. An initial fusion image 540 is obtained by fusing the to-be-processed image 510 and a target material 530. Specifically, for a first pixel in a face region in the to-be-processed image 510, first offset information of a material coordinate of the target material 530 corresponding to the face region is calculated based on depth information of the first pixel. The target material coordinate is determined based on the first offset information, so as to warp the target material 530 based on the depth change of the face contour. A target material pixel corresponding to the target material coordinate is fused to the first pixel, so that the target material 530 can fit the face contour after being fused to the to-be-processed image 510. For a second pixel in a non-face region in the to-be-processed image 510, second offset information of a material coordinate of the target material 530 corresponding to the non-face region is calculated based on color information of the second pixel. The target material coordinate is determined based on the second offset information, so as to warp the target material 530 according to the color value. A target material pixel corresponding to the target material coordinate is fused to the second pixel, so that the target material 530 can present different warping effects according to the color after being fused to the to-be-processed image 510. An edge extraction process is performed on the initial fusion image 540 based on the foreground mask 520, to determine a foreground image and a background image in the initial fusion image 540. The foreground image is darkened, and the background image is brightened, so as to improve the contrast between the foreground image and the background image. The image with the adjusted contrast is taken as an edge image 550. A Gaussian blur process is performed on the edge image 550, to obtain a blurred edge image 560, so as to simulate the effect of light dispersion by Gaussian blur. The texture of the initial fusion image 540 and the texture of the blurred edge image 560 are mixed, so as to strengthen the edge brightness of the initial fusion image 540. The luminous intensity is calculated based on the gray value of the texture of the initial fusion image 540, and the luminous intensity is stored in the alpha channel of the texture, to obtain a luminous texture image 570. The luminous texture image 570 is subjected to Gaussian blur, to obtain a blurred luminous texture image 580. The luminous color is calculated by using the luminous texture image 570 and the blurred luminous texture image 580. First, the pixel change amount is calculated according to the luminous texture image 570 and the blurred luminous texture image 580. The texture in the luminous texture image 570 and the blurred luminous texture image 580 is mixed, to calculate an initial luminous color. The initial luminous color is corrected based on the pixel change amount, so as to correct the pixels that are brightened due to Gaussian blur to darker values, and correct the pixels that are darkened due to Gaussian blur to brighter values. The corrected luminous color is mixed with the texture of the blurred luminous texture image 580, and the luminous intensity stored in the alpha channel in the texture is superimposed, to obtain a target luminous color. The target luminous color is superimposed on the texture of the initial fusion image 540, to obtain a fusion image 590.

[0092] Optionally, if the width and height of the target material are smaller than the width and height of the image to be processed, and the target material is only superimposed on the face region or the non-face region, the fusion step of the target material and the first pixel in the first region in the above step is performed, or the fusion step of the target material and the second pixel in the second region is performed.

[0093] Optionally, if the width and height of the target material are smaller than the width and height of the image to be processed, a foreground mask is generated according to the width and height of the target material and the position of the material layer. Then, based on the foreground mask, edge extraction and Gaussian blur operations are performed to avoid the problem of invalid calculation based on the generation of the mask for the entire image to be processed, and the fusion efficiency is improved.

[0094] S460, displaying the fusion image in the image processing interface.

[0095] The technical scheme of the embodiment of the present disclosure offsets the material coordinates of the corresponding position in the target material for the pixels in the image to be processed to obtain target material coordinates. The color information of the target material pixels is obtained based on the target material coordinates, and the obtained color information is fused with the color information of the pixels in the image to be processed to obtain an initial fusion image, which simulates the projection of the target material to the image to be processed, and enriches the image effect. Then, a light special effect is added to the initial fusion image to simulate the real light, and further, the texture of the film projection is created.

[0096] FIG. 6 is a structural schematic diagram of an image processing device provided by an embodiment of the present disclosure. The method can be executed by the image processing device, which can be realized in the form of software and / or hardware, and can be realized by an electronic device, which can be a mobile terminal, a PC terminal, or a server, etc.

[0097] As shown in FIG. 6, the device includes an image display module 610, a material display module 620, an offset determination module 630, and an image fusion model 640.

[0098] The image display module 610 is configured to display the image to be processed in an image processing interface.

[0099] The material display module 620 is configured to display the target material at the corresponding position of the image to be processed in response to a material selection operation in the image processing interface.

[0100] The offset determination module 630 is configured to determine the coordinate offset information of the target material based on the image to be processed in response to a projection trigger operation in the image processing interface.

[0101] The image fusion model 640 is configured to fuse the target material and the to-be-processed image based on the coordinate offset information to obtain a fused image, and display the fused image in the image processing interface, where the target material in the fused image is in a distorted state.

[0102] Optionally, the image display module 610 is specifically configured to:

[0103] display the to-be-processed image and a candidate control set in the image processing interface, where the candidate control set includes a material selection control and a projection trigger control, the material selection control is used to trigger a material selection operation, and the projection trigger control is used to trigger a projection trigger operation.

[0104] Optionally, the material display module 620 is specifically configured to:

[0105] in response to an interaction operation on the material selection control, superimpose and display a target layer on a layer corresponding to the to-be-processed image, where the target layer includes a target material.

[0106] Optionally, the offset determination module 630 is specifically configured to:

[0107] in response to an interaction operation on the projection trigger control, determine coordinate offset information of the target material based on attribute information of a pixel in the to-be-processed image.

[0108] Further, the determination of the coordinate offset information of the target material based on the attribute information of the pixel in the to-be-processed image includes:

[0109] for a first pixel in a first region in the to-be-processed image, determine first offset information corresponding to a first candidate coordinate in the target material according to depth information of the first pixel, where the first candidate coordinate is determined based on a pixel coordinate of the first pixel.

[0110] Further, the determination of the coordinate offset information of the target material based on the attribute information of the pixel in the to-be-processed image includes:

[0111] for a second pixel in a second region in the to-be-processed image, determine second offset information corresponding to a second candidate coordinate in the target material according to color information of the second pixel, where the second candidate coordinate is determined based on a pixel coordinate of the second pixel.

[0112] Optionally, the image fusion model 640 is specifically configured to:

[0113] for a pixel in the to-be-processed image, determine a target material coordinate according to the coordinate offset information, where the target material coordinate represents a coordinate of a target material pixel in the target material that is fused with the pixel.

[0114] fuse the target material to the image to be processed based on the target material coordinates to obtain a fused image.

[0115] Further, the step of fusing the target material to the image to be processed based on the target material coordinates to obtain a fused image comprises:

[0116] For a pixel in the image to be processed, color information of a target material pixel corresponding to the pixel is obtained according to the target material coordinates corresponding to the pixel, and the color information of the target material pixel and the color information of the pixel are fused to obtain an initial fused image.

[0117] A light-emitting special effect is added to the initial fused image to obtain a fused image.

[0118] Further, the step of adding a light-emitting special effect to the initial fused image to obtain a fused image comprises:

[0119] An edge extraction process is performed on the initial fused image to obtain an edge image.

[0120] A blur process is performed on the edge image to obtain a blurred edge image.

[0121] A light-emitting texture image is determined in combination of the initial fused image and the blurred edge image, a blur process is performed on the light-emitting texture image to obtain a blurred light-emitting texture image.

[0122] A light-emitting color is determined based on the light-emitting texture image and the blurred light-emitting texture image, and the light-emitting color is superimposed on the initial fused image to obtain a fused image.

[0123] The image processing apparatus provided by the embodiments of the present disclosure can execute the image processing method provided by any of the embodiments of the present disclosure, and has the corresponding function modules and beneficial effects of the execution method.

[0124] It is worth noting that each unit and module included in the above apparatus is only divided according to the function logic, but is not limited to the above division, as long as the corresponding function can be implemented; in addition, the specific name of each functional unit is only for convenient mutual distinction, and is not used to limit the protection scope of the embodiments of the present disclosure.

[0125] FIG. 7 is a structural diagram of an electronic device according to an embodiment of the disclosure. Below, referring to FIG. 7, a structural diagram of an electronic device (e.g., a terminal device or a server in FIG. 7) 700 suitable for implementing an embodiment of the disclosure is illustrated. The terminal device in an embodiment of the disclosure can include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet Personal Computer), a PMP (Portable Multimedia Player), a car terminal (e.g., a car navigation terminal), and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. The electronic device illustrated in FIG. 7 is merely an example, and should not impose any limitation on the functions and use range of an embodiment of the disclosure.

[0126] As illustrated in FIG. 7, the electronic device 700 can include a processing device (e.g., a central processor, a graphic processor, or the like) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0127] Generally, the following devices can be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, or the like; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, or the like; a storage device 708 including, for example, a magnetic tape, a hard disk, or the like; and a communication device 709. The communication device 709 can allow the electronic device 700 to communicate with other devices wirelessly or via a wire to exchange data. Although FIG. 7 illustrates the electronic device 700 having various devices, it should be understood that all of the illustrated devices are not required to be implemented or possessed. More or fewer devices can be alternatively implemented or possessed.

[0128] According to an embodiment of the disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, an embodiment of the disclosure includes a computer program product including a computer program carried on a non-transitory computer readable medium, the computer program containing program code for executing the methods illustrated in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the above-described functions defined in the methods of an embodiment of the disclosure are performed.

[0129] Names of messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes, and are not used to limit the scope of the messages or information.

[0130] The electronic device provided by the embodiments of the present disclosure and the image processing method provided by the above embodiments belong to the same inventive concept, and the technical details not described in detail in the present embodiment can be referred to the above embodiments, and the present embodiment has the same beneficial effects as the above embodiments.

[0131] The embodiments of the present disclosure provide a computer storage medium, which stores a computer program, and the program is executed by a processor to implement the image processing method provided by the above embodiments.

[0132] It should be noted that the computer readable medium of the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, an RF (radio frequency) or the like, or any suitable combination of the above.

[0133] In some embodiments, the client, server, or both can communicate using any known or future developed network protocols, such as the HyperText Transfer Protocol (HTTP), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current or future developed networks.

[0134] The computer-readable medium described above can be included in the electronic device described above; alternatively, the computer-readable medium can exist as a standalone entity.

[0135] The computer-readable medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to:

[0136] Display the image to be processed in an image processing interface;

[0137] In response to a material selection operation in the image processing interface, display a target material at a corresponding position of the image to be processed;

[0138] In response to a projection triggering operation in the image processing interface, determine coordinate offset information of the target material based on the image to be processed;

[0139] Fuse the target material and the image to be processed based on the coordinate offset information to obtain a fused image, and display the fused image in the image processing interface, wherein the target material in the fused image appears in a distorted state.

[0140] Computer program code for carrying out operations of the present disclosure can be written in any one or more programming languages, including object oriented programming languages such as Java, Smalltalk, C++, as well as conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network ("LAN") or a wide area network ("WAN"), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0141] The computer program product of the first aspect can include one or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform the operations of the method of the first aspect. The computer program product of the first aspect can include a computer-readable medium storing instructions that, when executed, cause one or more processors to perform the operations of the method of the first aspect.

[0142] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware, or by a combination of software and hardware. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0143] The functions described in the present disclosure can be performed by one or more hardware logic components. For example, non-limiting examples of hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SOCs), complex programmable logic devices (CPLDs), etc.

[0144] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0145] The above description merely illustrates the embodiments of the present disclosure and a principle of applied technologies. It should be understood by those skilled in the art that the disclosed scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the technical features described above, and should also cover other technical solutions formed by the combinations of the technical features described above or their equivalent features without departing from the concept disclosed above. For example, the technical solutions formed by the mutual replacement of the above-described features and the technical features with similar functions disclosed in the present disclosure (but not limited to) can be used.

[0146] Further, although operations are depicted in a particular order, this should not be understood as requiring the operations to be performed in the particular order shown or in a sequential order, and that certain operations can be performed in parallel or concurrently with other operations disclosed herein. Similarly, while operations have been depicted as following a specific sequence or order, this is not intended to mean that other specific sequences or orders can not be employed. In one aspect, multi-tasking and parallel processing can be advantageous. Additionally, the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. The various illustrative logical blocks and modules of the logical blocks can be implemented or performed by electronic hardware, a combination of hardware and software, or software. The software can be stored in memory and executed by a suitable instruction execution system, such as a processor. As will be apparent, a software implementation can include an operating system, one or more applications, other software, etc. Furthermore, the software can be initially stored in a removable storage medium, which can be later loaded onto the instruction execution system.

[0147] Although the present disclosure has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. An image processing method, comprising: Display the image to be processed within the image processing interface; In response to the material selection operation within the image processing interface, the target material is displayed at the corresponding position of the image to be processed; In response to a projection trigger operation within the image processing interface, the coordinate offset information of the target material is determined based on the image to be processed; Based on the coordinate offset information, the target material and the image to be processed are fused to obtain a fused image, which is then displayed in the image processing interface. In the fused image, the target material is distorted.

2. The method according to claim 1, wherein, The step of displaying the image to be processed within the image processing interface includes: The image processing interface displays the image to be processed and a set of candidate controls. The set of candidate controls includes a material selection control and a projection trigger control. The material selection control is used to trigger a material selection operation, and the projection trigger control is used to trigger a projection trigger operation.

3. The method according to claim 2, wherein, The step of displaying the target material at the corresponding position of the image to be processed in response to the material selection operation within the image processing interface includes: In response to an interactive operation on the material selection control, a target layer is overlaid and displayed above the layer corresponding to the image to be processed, the target layer including the target material.

4. The method according to claim 2 or 3, wherein, The step of responding to a projection trigger operation within the image processing interface and determining the coordinate offset information of the target material based on the image to be processed includes: In response to an interactive operation on the projection trigger control, the coordinate offset information of the target material is determined based on the attribute information of the pixels in the image to be processed.

5. The method according to claim 4, wherein, The step of determining the coordinate offset information of the target material based on the attribute information of pixels in the image to be processed includes: For a first pixel in a first region of the image to be processed, first offset information corresponding to a first candidate coordinate in the target material is determined based on the depth information of the first pixel, wherein the first candidate coordinate is determined based on the pixel coordinate of the first pixel.

6. The method according to claim 4 or 5, wherein, The step of determining the coordinate offset information of the target material based on the attribute information of pixels in the image to be processed includes: For a second pixel in a second region of the image to be processed, second offset information corresponding to a second candidate coordinate in the target material is determined based on the color information of the second pixel, wherein the second candidate coordinate is determined based on the pixel coordinate of the second pixel.

7. The method according to any one of claims 1-6, wherein, The process of fusing the target material and the image to be processed based on the coordinate offset information to obtain a fused image includes: For a pixel in the image to be processed, the coordinates of the target material are determined according to the coordinate offset information, wherein the coordinates of the target material represent the coordinates of the target material pixel that is fused with the pixel within the target material; The target material is fused into the image to be processed based on the coordinates of the target material to obtain a fused image.

8. The method according to claim 7, wherein, The step of fusing the target material into the image to be processed based on the target material coordinates to obtain a fused image includes: For each pixel in the image to be processed, the color information of the target material pixel is obtained according to the target material coordinates corresponding to the pixel, and the color information of the target material pixel is fused with the color information of the pixel to obtain an initial fused image; Adding a glowing effect to the initial fused image yields the fused image.

9. The method according to claim 8, wherein, Adding a glowing effect to the initial fused image to obtain the fused image includes: The initial fused image is subjected to edge extraction processing to obtain an edge image; The edge image is blurred to obtain a blurred edge image; The luminescent texture image is determined by combining the initial fused image and the blurred edge image, and the luminescent texture image is blurred to obtain a blurred luminescent texture image; The emission color is determined based on the emission texture image and the blurred emission texture image, and the emission color is superimposed on the initial fusion image to obtain the fusion image.

10. An image processing apparatus, comprising: The image display module is configured to display the image to be processed within the image processing interface; The material display module is configured to display the target material at the corresponding position of the image to be processed in response to the material selection operation in the image processing interface; The offset determination module is configured to determine the coordinate offset information of the target material based on the image to be processed in response to a projection trigger operation within the image processing interface. An image fusion model is configured to fuse the target material and the image to be processed based on the coordinate offset information to obtain a fused image, and to display the fused image in the image processing interface, wherein the target material in the fused image is distorted.

11. An electronic device, comprising: One or more processors; A storage device is configured to store one or more programs, wherein, When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in any one of claims 1-9.

12. A storage medium containing computer-executable instructions, wherein, The computer-executable instructions, when executed by a computer processor, are used to perform the image processing method as described in any one of claims 1-9.

Images