Image processing method and device and computer readable storage medium

Abstract

The embodiment of the invention discloses an image processing method and device and a computer readable storage medium. An object image and a corresponding target material image are acquired; performing object recognition processing on the object image to obtain a target object, a mask image of the target object and a target image area where a target part of the target object is located; performing plane identification in the target image area to obtain a plane area of a scene plane contacted with the target part in the object image; drawing the target material image into the plane area to obtain a first target material image; superposing a target object in the object image to the first target material image based on the mask image to obtain an object superposed image; and performing image fusion based on the object superposition image and the object image to obtain a target image. Therefore, the special effect of superposing the target object and the target material image in the scene plane is realized by generating the target image, the image generation effect is enriched, and the image processing effect is improved.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to an image processing method, device and computer-readable storage medium. Background Art

[0002] With the rapid development of Internet technology, the requirements for image processing are increasing. For example, by mixing at least two images, an image with a specific image effect is obtained, thereby improving the image display effect. In existing image processing methods, the mixed image is often obtained by simply superimposing the images.

[0003] In the process of studying and practicing the prior art, it is found that in the existing image processing method of mixing images by simple superposition, the generated image effect is relatively monotonous, resulting in poor image processing effect. Summary of the invention

[0004] The embodiments of the present application provide an image processing method, device and computer-readable storage medium, which can achieve a special effect of superimposing a target object with a target material image in a scene plane, enrich the image generation effect, and thus improve the image processing effect.

[0005] The present application provides an image processing method, including:

[0006] Obtaining an object image and a corresponding target material image;

[0007] Performing object recognition processing on the object image to obtain a target object and a mask image of the target object, and a target image region where a target part of the target object is located;

[0008] Performing plane recognition in the target image region to obtain a plane region of the scene plane contacted by the target part in the object image;

[0009] Drawing the target material image into the plane area to obtain a first target material image;

[0010] superimposing the target object in the object image onto the first target material image based on the mask image to obtain an object superimposed image;

[0011] Image fusion is performed based on the object superposition image and the object image to obtain a target image.

[0012] Accordingly, an embodiment of the present application provides an image processing device, including:

[0013] An image acquisition unit, used to acquire an object image and a corresponding target material image;

[0014] An object recognition unit, used for performing object recognition processing on the object image to obtain a target object and a mask image of the target object, and a target image region where a target part of the target object is located;

[0015] A plane recognition unit, used for performing plane recognition in the target image area to obtain a plane area of ​​the scene plane that the target part in the object image contacts;

[0016] An image drawing unit, used for drawing the target material image into the plane area to obtain a first target material image;

[0017] an image superposition unit, configured to superimpose the target object in the object image onto the first target material image based on the mask image to obtain an object superposition image;

[0018] An image fusion unit is used to perform image fusion based on the object superposition image and the object image to obtain a target image.

[0019] In one embodiment, the plane identification unit includes:

[0020] A feature point recognition subunit, used to recognize a plurality of feature points of the target part in the target image area;

[0021] A coordinate determination subunit, used to determine the plane coordinate information of the scene plane that the target part contacts in the object image according to the feature points;

[0022] The plane area construction subunit is used to construct a plane area corresponding to the scene plane based on the plane coordinate information.

[0023] In one embodiment, the coordinate determination subunit is used to:

[0024] constructing a plurality of initial planes in the object image based on the feature points;

[0025] Calculating the distance between the feature point and the initial plane;

[0026] Determining, in the initial plane according to the distance value, a target plane that matches a scene plane that the target part in the object image contacts;

[0027] Plane coordinate information corresponding to the scene plane is determined based on the target plane.

[0028] In one embodiment, the object recognition unit is used to:

[0029] identifying a target object, a mask image of the target object, and object posture information of the target object in the object image in the object image;

[0030] Determining a target part corresponding to the target object based on the object posture information;

[0031] A target image region where a target part of the target object is located is identified in the object image.

[0032] In one embodiment, the image superposition unit is used to:

[0033] generating a target object image of the target object based on the mask image and the object image;

[0034] determining, in the first target material image, a target contact position between the target object and the target material in the target material image;

[0035] The target object image is superimposed on the first target material image based on the target contact position to obtain an object superimposed image.

[0036] In one embodiment, the image fusion unit includes:

[0037] a target object filtering subunit, configured to filter out the target object from the object image based on the mask image and the object image to obtain a first background image;

[0038] an image region removing subunit, configured to remove the image region corresponding to the plane region from the first background image based on the first target material image, so as to obtain a second background image;

[0039] The image mixing subunit is used to mix the object superimposed image and the second background image to obtain a target image.

[0040] In one embodiment, the image area removal subunit is used to:

[0041] Obtaining a reverse mask image corresponding to the mask image;

[0042] extracting an alpha channel image of the first target material image;

[0043] Multiplying the inverse mask image, the alpha channel image, and the object image to obtain a plane region image corresponding to the plane region;

[0044] The first background image is subjected to a difference process with the plane area image to obtain a second background image.

[0045] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for a processor to load to execute the steps in any image processing method provided in the embodiment of the present application.

[0046] In addition, an embodiment of the present application also provides a computer device, including a processor and a memory, wherein the memory stores an application program, and the processor is used to run the application program in the memory to implement the image processing method provided in the embodiment of the present application.

[0047] The embodiment of the present application also provides a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the image processing method provided in the embodiment of the present application.

[0048] The embodiment of the present application obtains an object image and a corresponding target material image; performs object recognition processing on the object image to obtain a target object and a mask image of the target object, as well as a target image area where a target part of the target object is located; performs plane recognition in the target image area to obtain a plane area of ​​the scene plane that the target part in the object image contacts; draws the target material image into the plane area to obtain a first target material image; superimposes the target object in the object image onto the first target material image based on the mask image to obtain an object superimposed image; and performs image fusion based on the object superimposed image and the object image to obtain a target image. In this way, by identifying the scene plane that the target part in the target object in the object image contacts, and drawing the target material image into the plane area corresponding to the scene plane to obtain the first target material image, the target object in the object image is superimposed onto the first target material image, and the image is fused with the object image to obtain the target image, thereby realizing a special effect of superimposing the target object and the target material image in the scene plane based on the target image, enriching the image generation effect, and thus improving the image processing effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 It is a schematic diagram of an implementation scenario of an image processing method provided in an embodiment of the present application;

[0051] Figure 2 It is a flowchart of an image processing method provided in an embodiment of the present application;

[0052] Figure 3a is an image schematic diagram of an image processing method provided by an embodiment of the present application;

[0053] Figure 3b is a schematic diagram of a mask image of an image processing method provided in an embodiment of the present application;

[0054] Figure 3c is a schematic diagram of a first target material image of an image processing method provided in an embodiment of the present application;

[0055] Figure 3d This is a schematic diagram of an object superimposed image of an image processing method provided in an embodiment of the present application;

[0056] Figure 4 is a schematic diagram of a target image of an image processing method provided in an embodiment of the present application;

[0057] Figure 5 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;

[0058] Figure 6 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0060] The embodiments of the present application provide an image processing method, device and computer-readable storage medium. The image processing device can be integrated in a computer device, which can be a server or a terminal.

[0061] Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms. Terminals may include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, etc. Terminals and servers can be directly or indirectly connected via wired or wireless communications, and this application does not limit this.

[0062] See also Figure 1 , taking the image processing device integrated into a computer device as an example, Figure 1 A schematic diagram of an implementation scenario of the image processing method provided in an embodiment of the present application, wherein the computer device can be a server or a terminal, and the computer device can obtain an object image and a corresponding target material image; perform object recognition processing on the object image to obtain a target object and a mask image of the target object, and a target image area where a target part of the target object is located; perform plane recognition in the target image area to obtain a plane area of ​​a scene plane contacted by the target part in the object image; draw the target material image into the plane area to obtain a first target material image; superimpose the target object in the object image on the first target material image based on the mask image to obtain an object superimposed image; perform image fusion based on the object superimposed image and the object image to obtain a target image.

[0063] It should be noted that Figure 1 The implementation environment scenario diagram of the image processing method shown is only an example. The implementation environment scenario of the image processing method described in the embodiment of the present application is to more clearly illustrate the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided by the embodiment of the present application. It is known to those skilled in the art that with the evolution of data processing and the emergence of new business scenarios, the technical solution provided by the present application is also applicable to similar technical problems.

[0064] The solutions provided by the embodiments of the present application are specifically described by the following embodiments. It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0065] This embodiment will be described from the perspective of an image processing apparatus. The image processing apparatus may be integrated into a computer device, which may be a server. This application does not limit this.

[0066] See also Figure 2 , Figure 2: is a flow chart of an image processing method provided in an embodiment of the present application. The image processing method comprises:

[0067] In step 101, an object image and a corresponding target material image are obtained.

[0068] The object image may be an image including an object, such as a person or an animal, and the target material image may be a material image used to generate special effects with the object image, such as a song list image, a starry sky image, a cloud image, and the like. Figure 3a , Figure 3a It is an image schematic diagram of an image processing method provided in an embodiment of the present application. The user can input an object image containing a person, for example, an image of a person on a beach, and can select a desired target material image from multiple material images, for example, a playlist image.

[0069] Among them, there can be many ways to obtain the object image and the corresponding target material image. For example, the user can input the object image and trigger the operation of generating a special effect image, so that when receiving the special effect generation operation for the object image, the currently playing song information can be determined, and a playlist image can be generated according to the song information, and the playlist image can be used as the target material image.

[0070] The song information may be information related to the song currently being played, for example, the name of the song currently being played, the play progress, the total play time, the song author, etc. Optionally, the song currently being played may be a song currently being played in the current terminal, or a song currently being played in an application for special effect generation, etc., which may be specifically set according to the actual application scenario.

[0071] In step 102, object recognition processing is performed on the object image to obtain a target object and a mask image of the target object, as well as a target image region where a target part of the target object is located.

[0072] The target object may be an object identified in the object image, or may be an object in the main body of the object image. The mask image may be information indicating the region where the target object is located in the object image. The mask image may be a binary image composed of 0 and 1. In the mask image of the target object, the value of the pixel belonging to the target object is 1, and the value of the pixel not belonging to the target object is 0, etc. For example, please refer to Figure 3b , Figure 3bIt is a mask image schematic diagram of an image processing method provided by an embodiment of the present application. In the mask image of the target object, the value of the pixel belonging to the target object is 1, and the value of the pixel not belonging to the target object is 0. The target part can be the part of the target object in the object image that contacts the scene plane, for example, it can be the feet, buttocks, back and other parts. The scene plane can be the plane where the target object is located in the object image, for example, it can be a plane such as the ground plane and the sea plane. For example, when the target object in the object image stands on the beach, the scene plane can be the ground plane of the beach. When the target object in the object image sits on the grass, the scene plane can be the ground plane corresponding to the grass. When the target object in the object image sits on the tree, the scene plane can be the plane corresponding to the branch where the target object sits, etc., which can be determined according to actual conditions. The target image area can be an area in the object image that contains the target part of the target object. Optionally, it can be a region of interest (ROI) for the target part in the object image.

[0073] There are many ways to perform object recognition processing on the object image to obtain the target object and the mask image of the target object, as well as the target image area where the target part of the target object is located. For example, the target object, the mask image of the target object, and the object posture information of the target object in the object image can be identified in the object image, the target part corresponding to the target object is determined based on the object posture information, and the target image area where the target part of the target object is located is identified in the object image.

[0074] The object posture information may be information describing the posture of the target object in the object image, and the posture may include standing, sitting, lying, and other types of postures.

[0075] There are many ways to determine the target part corresponding to the target object based on the object posture information. For example, the posture type of the target object in the object image can be determined according to the object posture information, and the target part of the target object corresponding to the posture type can be determined in a preset mapping relationship according to the posture type.

[0076] Among them, the preset mapping relationship can be information describing the correspondence between the posture type and the target part. For example, the target part corresponding to the standing posture can be the foot, the target part corresponding to the sitting posture can be the buttocks, and the target part corresponding to the lying posture can be the back, etc. In this way, when the posture of the target object in the object image is standing, the target part of the target object can be the foot, when the posture of the target object in the object image is sitting, the target part of the target object can be the buttocks, and when the posture of the target object in the object image is lying, the target part of the target object can be the back, etc., which can be set according to the actual application scenario, and the embodiment of the present application is not limited here.

[0077] In step 103, plane recognition is performed in the target image region to obtain a plane region of the scene plane that the target part in the object image contacts.

[0078] The plane region may be a region where a plane corresponding to the scene plane is located, the plane corresponding to the scene plane may be a plane parallel to the scene plane, and the normal of the plane parallel to the scene plane may be parallel to the normal of the scene plane. Optionally, a plane mesh of the plane corresponding to the scene plane may be constructed in the form of a mesh, thereby obtaining the plane region corresponding to the scene plane.

[0079] There are many ways to perform plane recognition in the target image area and obtain the planar area of ​​the scene plane that the target part contacts in the object image. For example, multiple feature points of the target part can be identified in the target image area, and the target part can be determined based on the feature points. The plane coordinate information of the scene plane that the target part contacts in the object image can be used to construct the planar area corresponding to the scene plane based on the plane coordinate information.

[0080] The feature point may be a point in the target image region that reflects the characteristics of the target part, for example, a point where the image grayscale value changes dramatically or a point with a large curvature on the edge of the image, etc. The plane coordinate information may be coordinate information for determining the plane corresponding to the scene plane, for example, the plane coordinate information may be coordinate information of at least three points in the plane corresponding to the scene plane.

[0081] There are many ways to identify multiple feature points of the target part in the target image area. For example, the Scale Invariant Feature Transform (SIFT) algorithm can be used to identify multiple feature points of the target part in the target image area. Or, feature point extraction algorithms such as the Oriented Fast and Rotated Brief (ORB) algorithm and the Speeded Up Robust Features (SURF) can be used to identify multiple feature points of the target part in the target image area.

[0082] After multiple feature points of the target part are identified in the target image area, the plane coordinate information of the scene plane that the target part contacts in the object image can be determined based on the feature points. There are multiple ways to determine the plane coordinate information of the scene plane that the target part contacts in the object image based on the feature points. For example, multiple initial planes can be constructed in the object image based on the feature points, and the distance values ​​between the feature points and the initial planes can be calculated. According to the distance values, a target plane that matches the scene plane that the target part contacts in the object image can be determined in the initial plane, and the plane coordinate information corresponding to the scene plane can be determined based on the target plane.

[0083] The initial plane may be a plurality of planes determined according to the feature points, for example, three or more feature points may be randomly selected from the plurality of feature points to construct the plurality of initial planes. The distance value may be information describing the distance between the feature point and the initial plane, and the target plane may be a plane in the initial plane that matches the scene plane in contact with the target part in the object image, for example, it may be a plane in the initial plane that is closest to the scene plane.

[0084] There are many ways to determine the target plane that matches the scene plane that the target part in the object image contacts in the initial plane based on the distance value. For example, since under ideal conditions, the distance from the feature point in the target part to the scene plane is the smallest, the distance value corresponding to each initial plane can be accumulated to obtain the total distance value corresponding to each initial plane, so that the initial plane with the smallest total distance value in the initial plane can be determined as the target plane that matches the scene plane that the target part in the object image contacts.

[0085] After determining the target plane matching the scene plane contacted by the target part in the object image in the initial plane according to the distance value, the plane coordinate information corresponding to the scene plane can be determined based on the target plane. There are many ways to determine the plane coordinate information corresponding to the scene plane based on the target plane. For example, a plane model representation of the target plane can be defined. For example, it can be represented as Ax+By+Cz+D=0, where A, B, C and D are parameters of the target plane, parameters A, B and C define the normal vector of the target plane, and parameter D defines the distance between the target plane and the origin. In this way, the point coordinates of at least four points can be extracted in the target plane. Substituting the point coordinates into the plane model representation, the specific values ​​of the parameters A, B, C and D can be solved, so that the target plane can be determined. Optionally, the characteristic points belonging to the target plane and the least square method can be used to fit the parameters of the plane model representation of the target plane, so that the target plane matches the extracted characteristic points as much as possible. The fitted target plane can also be verified according to the characteristic points of the target part to ensure that the target plane meets the geometric constraints of the plane (such as flatness and angle) and the consistency in the object image, so that the plane corresponding to the scene plane can be determined more accurately. In this way, by fitting and verifying the target plane multiple times, the final target plane matching the scene plane can be obtained, so that the plane coordinate information of four points in the target plane can be obtained to construct the plane area corresponding to the scene plane in the two-dimensional image based on the plane coordinate information.

[0086] In step 104, the target material image is drawn into the plane area to obtain a first target material image.

[0087] The first target material image may be a target material image drawn into the plane area. For example, please refer to Figure 3c , Figure 3c This is a schematic diagram of a first target material image of an image processing method provided in an embodiment of the present application. By drawing the target material image of a playlist image into a plane area, a target material image in the same plane as the plane area can be obtained, that is, the first target material image.

[0088] Optionally, in the first target material image, the area where the target material image is located is non-transparent, and the transparency value of the pixels in the plane area may be 1; other areas may be transparent, and the transparency value of the pixels in other areas may be 0.

[0089] In step 105, the target object in the object image is superimposed on the first target material image based on the mask image to obtain an object superimposed image.

[0090] The object superimposed image may be an image obtained by superimposing the target object on the first target material image. For example, please refer to Figure 3d , Figure 3d This is a schematic diagram of an object overlay image of an image processing method provided in an embodiment of the present application. The target part of the target object is the foot. In the object overlay image, the target object is superimposed on the playlist image, thereby obtaining an effect that the target object's feet are on the playlist.

[0091] Among them, there are many ways to obtain the object superimposed image by superimposing the target object in the object image on the first target material image based on the mask image. For example, a target object image of the target object can be generated based on the mask image and the object image, and the target contact position between the target object and the target material in the target material image can be determined in the first target material image. The target object image is superimposed on the first target material image based on the target contact position to obtain the object superimposed image.

[0092] Among them, the target object image can be an image that only contains the target object in the object image, the target contact position can be the position where the target part of the target object contacts the first target material image when the target object is superimposed on the first target material image, and the target material can be an object in the target material image. For example, when the target material image is a playlist image, the target material can be a playlist in the playlist image; when the target material image is a cloud image, the target material can be a cloud in the cloud image, etc.

[0093] Among them, based on the mask image and the object image, there can be many ways to generate a target object image of the target object. For example, the color value of each pixel in the mask image can be multiplied with the color value of the corresponding pixel in the object image, and the target object image can be obtained according to the multiplication result of the color values.

[0094] After the target object image of the target object is generated based on the mask image and the object image, the target contact position between the target object and the target material in the target material image can be determined in the first target material image. There are many ways to determine the target contact position between the target object and the target material in the target material image in the first target material image. For example, the center position of the target material image can be determined in the first target material image, and the center position is determined as the target contact position between the target object and the target material in the target material image. In addition, the corresponding position in the first target material image can be determined as the target contact position based on the position in the scene plane of the object image that contacts the target part of the target object. The target contact position can also be determined in the first target material image through user operation, etc.

[0095] After determining the target contact position between the target object and the target material in the target material image in the first target material image, the target object image can be superimposed on the first target material image based on the target contact position to obtain an object superimposed image. There are many ways to superimpose the target object image on the first target material image based on the target contact position. For example, the target part of the target object image can be superimposed on the target contact position, thereby realizing superimposing the target object image on the first target material image.

[0096] For example, the process of superimposing the target object in the object image onto the first target material image based on the mask image to obtain the object superimposed image can be expressed as:

[0097] res=mix(srcColor,stickerColor,1.0-segmentColor)

[0098] Among them, res represents the object overlay image, mix(a,b,x) represents the mixing function, wherein mix(a,b,x)=a*(1.0-x)+b*x, srcColor can represent the pixel value of the object image, stickerColor can represent the pixel value of the first target material image, and segmentColor represents the pixel value of the mask image.

[0099] In step 106, image fusion is performed based on the object superposition image and the object image to obtain a target image.

[0100] The target image may be an image that is a fusion of an object overlay image and an object image, in which the target object is overlaid on the target material image in the scene plane, achieving an effect that the target object is on the plane where the target material image is located. For example, please refer to Figure 4 , Figure 4 This is a schematic diagram of a target image of an image processing method provided in an embodiment of the present application. Assuming that in the object image the target object is standing on the beach, the target part of the target object is the feet, and the target material image is a playlist image, through the image processing method provided in an embodiment of the present application, a target image with the effect that the target object's feet are on the playlist and the playlist is on the scene plane of the beach can be generated, thereby enriching the image mixing effect and improving the image display effect.

[0101] There are many ways to obtain the target image by performing image fusion based on the object overlay image and the object image. For example, based on the mask image and the object image, the target object can be filtered out from the object image to obtain a first background image. Based on the first target material image, the image area corresponding to the plane area can be removed from the first background image to obtain a second background image. The object overlay image and the second background image can be mixed to obtain the target image.

[0102] Among them, the first background image can be an object image that does not include the target object, the image area can be the area corresponding to the plane area in the first background image, that is, the area occupied by the plane area in the object image, and the second background image can be the first background image from which the image area corresponding to the plane area is removed.

[0103] Among them, based on the mask image and the object image, there are many ways to filter out the target object from the object image and obtain the first background image. For example, the pixel values ​​of the pixels in the mask image can be inverted. For example, the pixel a with a pixel value of 1 is inverted to obtain a pixel a with a pixel value of 0, and the pixel b with a pixel value of 0 is inverted to obtain a pixel b with a pixel value of 1, etc., so that a background mask image that does not contain the target object can be obtained. The background mask image and the object image are mixed to obtain the first background image that does not contain the target object.

[0104] After filtering out the target object from the object image based on the mask image and the object image to obtain the first background image, the image area corresponding to the plane area can be removed from the first background image based on the first target material image to obtain the second background image. There are multiple ways to remove the image area corresponding to the plane area from the first background image based on the first target material image. For example, an inverse mask image corresponding to the mask image can be obtained, and the alpha channel image of the first target material image can be extracted. The inverse mask image, the alpha channel image and the object image are multiplied to obtain a plane area image corresponding to the plane area, and the first background image is subtracted from the plane area image to obtain the second background image.

[0105] The inverse mask image may be an image obtained by inverting the pixel values ​​of the pixels in the mask image. For example, assuming that there is a pixel c with a pixel value of 1 in the mask image, the pixel value of pixel c in the inverse mask image is 0, and if there is a pixel d with a pixel value of 0 in the mask image, the pixel value of pixel d in the inverse mask image is 1, etc. The alpha channel image may be an image describing the transparency value of each pixel in the alpha channel of the color space of the first target material image. For example, in the first target material image, the transparency value of each pixel in the plane area where the target material image is located may be 1, and the transparency value of the pixels in other areas may be 0. Thus, in the alpha channel image, the pixel value in the area where the target material image is located is 1, and the pixel value in other areas is 0. The plane area image may be an image of the area where the first background image and the first target material image overlap.

[0106] Optionally, there are multiple ways to obtain the second background image by removing the image area corresponding to the plane area from the first background image based on the first target material image. For example, a material mask image of the first target material image can be generated based on the first target material image. In the material mask image, the area where the plane area is located is a non-transparent area, and the non-plane area is a transparent area. Then, the material mask image, the reverse mask image, and the object image can be multiplied to obtain a plane area image corresponding to the plane area, and the first background image and the plane area image can be subtracted to obtain the second background image.

[0107] In one embodiment, the process of performing image fusion based on the object superposition image and the object image to obtain the target image can be expressed as:

[0108] Fsrc=srcColor–srcColor*segmentColor+res–stickerColor*srcColor*(1.0–segmentColor)

[0109] Among them, Fsrc can represent the pixel value of the target image, srcColor–srcColor*segmentColor can represent the pixel value of the first background image, (1.0–segmentColor) can represent the pixel value of the inverse mask image, stickerColor*srcColor*(1.0–segmentColor) can represent the pixel value of the plane area image, and srcColor–srcColor*segmentColor–stickerColor*srcColor*(1.0–segmentColor) can represent the pixel value of the second background image.

[0110] As can be seen from the above, the embodiment of the present application obtains an object image and a corresponding target material image; performs object recognition processing on the object image to obtain a target object and a mask image of the target object, as well as a target image area where the target part of the target object is located; performs plane recognition in the target image area to obtain a plane area of ​​the scene plane that the target part in the object image contacts; draws the target material image into the plane area to obtain a first target material image; superimposes the target object in the object image on the first target material image based on the mask image to obtain an object superimposed image; performs image fusion based on the object superimposed image and the object image to obtain a target image. In this way, by identifying the scene plane that the target part in the target object contacts in the object image, and drawing the target material image into the plane area corresponding to the scene plane, the first target material image is obtained, thereby superimposing the target object in the object image on the first target material image, and performing image fusion with the object image to obtain the target image, thereby realizing the special effect of superimposing the target object and the target material image in the scene plane based on the target image, enriching the image generation effect, and thus improving the image processing effect.

[0111] In order to better implement the above method, an embodiment of the present invention further provides an image processing apparatus, which can be integrated in a computer device, and the computer device can be a server or a terminal.

[0112] For example, Figure 5 As shown, it is a schematic diagram of the structure of the image processing device provided in the embodiment of the present application. The image processing device may include an image acquisition unit 201, an object recognition unit 202, a plane recognition unit 203, an image drawing unit 204, an image superposition unit 205 and an image fusion unit 206, as follows:

[0113] The image acquisition unit 201 is used to acquire the object image and the corresponding target material image;

[0114] The object recognition unit 202 is used to perform object recognition processing on the object image to obtain the target object and the mask image of the target object, and the target image area where the target part of the target object is located;

[0115] A plane recognition unit 203, used to perform plane recognition in the target image area to obtain a plane area of ​​the scene plane that the target part in the object image contacts;

[0116] An image drawing unit 204 is used to draw a target material image into a plane area to obtain a first target material image;

[0117] An image superposition unit 205, configured to superimpose a target object in the object image onto the first target material image based on the mask image to obtain an object superposition image;

[0118] The image fusion unit 206 is used to perform image fusion based on the object superposition image and the object image to obtain a target image.

[0119] In one embodiment, the plane identification unit 203 includes:

[0120] A feature point recognition subunit is used to recognize multiple feature points of a target part in a target image area;

[0121] A coordinate determination subunit, used to determine the plane coordinate information of the scene plane that the target part contacts in the object image according to the feature points;

[0122] The plane area construction subunit is used to construct a plane area corresponding to the scene plane based on the plane coordinate information.

[0123] In one embodiment, the coordinate determination subunit is used to:

[0124] constructing a plurality of initial planes in the object image based on the feature points;

[0125] Calculate the distance between the feature point and the initial plane;

[0126] Determining a target plane in the initial plane that matches the scene plane that the target part in the object image contacts according to the distance value;

[0127] The plane coordinate information corresponding to the scene plane is determined based on the target plane.

[0128] In one embodiment, the object identification unit 202 is used to:

[0129] Recognize a target object, a mask image of the target object, and object posture information of the target object in the object image in the object image;

[0130] Determine a target part corresponding to the target object based on the object posture information;

[0131] A target image region where a target part of a target object is located is identified in the object image.

[0132] In one embodiment, the image superposition unit 205 is used to:

[0133] generating a target object image of the target object based on the mask image and the object image;

[0134] determining, in the first target material image, a target contact position between the target object and the target material in the target material image;

[0135] The target object image is superimposed on the first target material image based on the target contact position to obtain an object superimposed image.

[0136] In one embodiment, the image fusion unit 206 includes:

[0137] A target object filtering subunit, configured to filter the target object from the object image based on the mask image and the object image to obtain a first background image;

[0138] An image region removing subunit, configured to remove the image region corresponding to the plane region from the first background image based on the first target material image, so as to obtain a second background image;

[0139] The image mixing subunit is used to mix the object superimposed image and the second background image to obtain a target image.

[0140] In one embodiment, the image area removal subunit is used to:

[0141] Obtaining an inverse mask image corresponding to the mask image;

[0142] extracting an alpha channel image of a first target material image;

[0143] Multiplying the inverse mask image, the alpha channel image, and the object image to obtain a plane region image corresponding to the plane region;

[0144] The first background image is subjected to difference processing with the plane region image to obtain a second background image.

[0145] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can refer to the previous method embodiments, which will not be repeated here.

[0146] As can be seen from the above, the embodiment of the present application acquires an object image and a corresponding target material image through an image acquisition unit 201; the object recognition unit 202 performs object recognition processing on the object image to obtain a target object and a mask image of the target object, and a target image area where a target part of the target object is located; the plane recognition unit 203 performs plane recognition in the target image area to obtain a plane area of ​​the scene plane contacted by the target part in the object image; the image drawing unit 204 draws the target material image into the plane area to obtain a first target material image; the image overlay unit 205 overlays the target object in the object image onto the first target material image based on the mask image to obtain an object overlay image; the image fusion unit 206 performs image fusion based on the object overlay image and the object image to obtain a target image. In this way, by identifying the scene plane in contact with the target part of the target object in the object image, and drawing the target material image into the plane area corresponding to the scene plane, a first target material image is obtained, thereby superimposing the target object in the object image on the first target material image, and performing image fusion with the object image to obtain the target image, thereby achieving a special effect of superimposing the target object with the target material image in the scene plane based on the target image, enriching the image generation effect, and thus improving the image processing effect.

[0147] The present application also provides a computer device, such as Figure 6 As shown, it shows a schematic diagram of the structure of a computer device involved in an embodiment of the present application, and the computer device may be a server or a terminal. Specifically:

[0148] The computer device may include one or more processing core processors 301, one or more computer readable storage media memories 302, a power supply 303, an input unit 304 and other components. Those skilled in the art will appreciate that Figure 6 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0149] The processor 301 is the control center of the computer device. It uses various interfaces and lines to connect various parts of the entire computer device. It executes various functions of the computer device and processes data by running or executing software programs and / or modules stored in the memory 302, and calling data stored in the memory 302. Optionally, the processor 301 may include one or more processing cores; preferably, the processor 301 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 301.

[0150] The memory 302 can be used to store software programs and modules. The processor 301 executes various functional applications and data processing by running the software programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0151] The computer device also includes a power supply 303 for supplying power to various components. Preferably, the power supply 303 can be logically connected to the processor 301 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 303 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0152] The computer device may further include an input unit 304, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0153] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 301 in the computer device will load the executable files corresponding to the processes of one or more application programs into the memory 302 according to the following instructions, and the processor 301 will run the application programs stored in the memory 302, thereby realizing various functions, as follows:

[0154] Acquire an object image and a corresponding target material image; perform object recognition processing on the object image to obtain a target object and a mask image of the target object, and a target image area where a target part of the target object is located; perform plane recognition in the target image area to obtain a plane area of ​​a scene plane contacted by the target part in the object image; draw the target material image into the plane area to obtain a first target material image; superimpose the target object in the object image on the first target material image based on the mask image to obtain an object superimposed image; perform image fusion based on the object superimposed image and the object image to obtain a target image.

[0155] The specific implementation of each of the above operations can be found in the previous embodiments, which will not be described in detail here. It should be noted that the computer device provided in the embodiment of the present application and the image processing method in the above embodiment belong to the same concept, and its specific implementation process is detailed in the above method embodiment, which will not be described in detail here.

[0156] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0157] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any one of the image processing methods provided in the embodiments of the present application. For example, the instructions can execute the following steps:

[0158] Acquire an object image and a corresponding target material image; perform object recognition processing on the object image to obtain a target object and a mask image of the target object, and a target image area where a target part of the target object is located; perform plane recognition in the target image area to obtain a plane area of ​​a scene plane contacted by the target part in the object image; draw the target material image into the plane area to obtain a first target material image; superimpose the target object in the object image on the first target material image based on the mask image to obtain an object superimposed image; perform image fusion based on the object superimposed image and the object image to obtain a target image.

[0159] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0160] Since the instructions stored in the computer-readable storage medium can execute the steps in any image processing method provided in the embodiments of the present application, the beneficial effects that can be achieved by any image processing method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0161] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the various optional implementations provided in the above embodiments.

[0162] The above is a detailed introduction to an image processing method, device and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image processing method, characterized in that: include: Obtaining an object image and a corresponding target material image; Performing object recognition processing on the object image to obtain a target object and a mask image of the target object, and a target image region where a target part of the target object is located; Performing plane recognition in the target image region to obtain a plane region of the scene plane contacted by the target part in the object image; Drawing the target material image into the plane area to obtain a first target material image; superimposing the target object in the object image onto the first target material image based on the mask image to obtain an object superimposed image; Image fusion is performed based on the object superposition image and the object image to obtain a target image.

2. The image processing method according to claim 1, characterized in that: The performing plane recognition in the target image region to obtain a plane region of the scene plane contacted by the target part in the object image includes: Identifying a plurality of feature points of the target part in the target image area; Determine, based on the feature points, the plane coordinate information of the scene plane that the target part contacts in the object image; A plane area corresponding to the scene plane is constructed based on the plane coordinate information.

3. The image processing method according to claim 2, characterized in that: The determining of the plane coordinate information of the scene plane contacted by the target part in the object image according to the feature points includes: constructing a plurality of initial planes in the object image based on the feature points; Calculating the distance between the feature point and the initial plane; Determining, in the initial plane according to the distance value, a target plane that matches a scene plane that the target part in the object image contacts; Plane coordinate information corresponding to the scene plane is determined based on the target plane.

4. The image processing method according to claim 1, wherein: The object recognition processing is performed on the object image to obtain the target object and the mask image of the target object, and the target image area where the target part of the target object is located, including: identifying a target object, a mask image of the target object, and object posture information of the target object in the object image in the object image; Determining a target part corresponding to the target object based on the object posture information; A target image region where a target part of the target object is located is identified in the object image.

5. The image processing method according to claim 1, wherein: The step of superimposing the target object in the object image onto the first target material image based on the mask image to obtain an object superimposed image includes: generating a target object image of the target object based on the mask image and the object image; determining, in the first target material image, a target contact position between the target object and the target material in the target material image; The target object image is superimposed on the first target material image based on the target contact position to obtain an object superimposed image.

6. The image processing method according to any one of claims 1 to 5, characterized in that: The step of performing image fusion based on the object superposition image and the object image to obtain a target image includes: Based on the mask image and the object image, filtering out the target object from the object image to obtain a first background image; Based on the first target material image, removing the image area corresponding to the plane area from the first background image to obtain a second background image; The object overlay image and the second background image are mixed to obtain a target image.

7. The image processing method according to claim 6, characterized in that: The step of removing the image area corresponding to the plane area from the first background image based on the first target material image to obtain a second background image includes: Obtaining a reverse mask image corresponding to the mask image; extracting an alpha channel image of the first target material image; Multiplying the inverse mask image, the alpha channel image, and the object image to obtain a plane region image corresponding to the plane region; The first background image is subjected to a difference process with the plane area image to obtain a second background image.

8. An image processing device, characterized in that: include: An image acquisition unit, used to acquire an object image and a corresponding target material image; An object recognition unit, used for performing object recognition processing on the object image to obtain a target object and a mask image of the target object, and a target image region where a target part of the target object is located; A plane recognition unit, used for performing plane recognition in the target image area to obtain a plane area of ​​the scene plane that the target part in the object image contacts; An image drawing unit, used for drawing the target material image into the plane area to obtain a first target material image; an image superposition unit, configured to superimpose the target object in the object image onto the first target material image based on the mask image to obtain an object superposition image; An image fusion unit is used to perform image fusion based on the object superposition image and the object image to obtain a target image.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the image processing method according to any one of claims 1 to 7.

10. A computer device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the image processing method according to any one of claims 1 to 7 when executing the computer program.

Images