Image processing method, apparatus, device, and medium

Abstract

The present disclosure relates to an image processing method, apparatus, device, and medium. The present disclosure recognizes an image main subject object in an image to be processed, and further generates a target polygon region that can position the remaining part outside a part of the image main subject object. After that, according to the target polygon region, the image to be processed can be fused with additional image materials to obtain a fused image. Thereby, the image content of the target polygon region of the fused image and the image to be processed is made to coincide with the image content within the first image region occupied by the image main subject object. The image content within the second image region outside the first image region of the fused image presents additional image materials, and further presents the effect of a three-dimensional display of the image main subject object with respect to the target polygon region, satisfying the display requirements of the user for the three-dimensional display of the image and improving the user experience.

Description

Technical Field

[0001] [Cross - reference to Related Applications] This application claims the priority of Chinese Patent Application No. 202211193567.6 filed on September 28, 2022, and the entire content disclosed in the above - mentioned Chinese Patent Application is incorporated herein by reference as part of this application.

[0002] This disclosure relates to an image processing method, apparatus, device, and medium.

Background Art

[0003] As the requirements for the display (presentation) of users' pictures or videos are becoming increasingly high, many users hope that some or all of the content of the images presented in the pictures or videos can be presented three - dimensionally. However, all the pictures or videos obtained by editing the content of images or videos using conventional image - editing software or video - editing software are two - dimensional pictures or videos, which cannot meet the display requirements for the three - dimensional display of users' pictures or videos, resulting in a low user experience.

Summary of the Invention

Means for Solving the Problems

[0004] To solve the above technical problems, this disclosure provides an image processing method, apparatus, device, and medium.

[0005] In a first aspect, this disclosure includes the steps of obtaining an image to be processed, performing main subject recognition on the image to be processed to obtain an image main - subject object, and generating a target polygon area corresponding to the image main - subject object, where a part of the image main - subject object is located within the target polygon area and the remaining part of the image main - subject object is located outside the target polygon area. According to a target polygon region, a step of obtaining a fused image by fusing an image to be processed with additional image materials, wherein the fused image coincides with the image content within a first image region of the image to be processed, the image content within a second image region of the fused image presents the additional image materials, the first image region is a merged region of the target polygon region and the image region occupied by the image main subject object, and the second image region is an image region other than the first image region, and providing an image processing method including the step.

[0006] In a second aspect, the present disclosure An image acquisition module configured to acquire an image to be processed, A main subject recognition module configured to perform main subject recognition on the image to be processed to obtain an image main subject object, A region generation module configured to generate a target polygon region corresponding to the image main subject object, a part of the image main subject object being located within the target polygon region and the remaining part of the image main subject object being located outside the target polygon region, An image fusion module configured to obtain a fused image by fusing the image to be processed with additional image materials according to the target polygon region, the fused image coinciding with the image content within a first image region of the image to be processed, the image content within a second image region of the fused image presenting the additional image materials, the first image region being a merged region of the target polygon region and the image region occupied by the image main subject object, and the second image region being an image region other than the first image region, and providing an image processing apparatus including the module.

[0007] In a third aspect, the present disclosure A processor, A memory for storing executable instructions, and including The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the image processing method of the first aspect, and providing an image processing device.

[0008] In a fourth aspect, the present disclosure provides a computer-readable medium storing a computer program, which, when executed by a processor, causes the processor to implement the image processing method of the first aspect.

[0009] With reference to the following specific embodiments in conjunction with the drawings, the above and other features, advantages and aspects of each example of the present disclosure will become more apparent. Throughout the drawings, the same or similar reference numerals indicate the same or similar elements. It should be understood that the drawings are exemplary and the elements and components are not necessarily drawn to scale.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present disclosure will be described in more detail with reference to the drawings. Although some specific embodiments of the present disclosure are shown in the drawings, the present disclosure can be realized in various forms and should not be construed as being limited to the embodiments described herein. Rather, it should be understood that these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the protection scope of the present disclosure.

[0012] It should be understood that each step described in the method embodiment of the present disclosure may be executed in a different order and / or executed in parallel. Also, the method embodiment may include additional steps and / or omit the execution of the steps shown. In this regard, the scope of the present disclosure is not limited.

[0013] The term "comprising" and its variants used herein are to be construed in an open-ended manner, i.e., "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" represents "at least one embodiment", the term "another embodiment" represents "at least one another embodiment", and the term "some embodiments" represents "at least some embodiments". Related definitions of other terms will be described in the following explanations.

[0014] Note that the concepts such as "first" and "second" mentioned in the present disclosure are only for the purpose of distinguishing different devices, modules or units, and are not intended to limit the order or interdependence of the functions executed by these devices, modules or units.

[0015] Note that the modifications of "one" and "a plurality" mentioned in the present disclosure are exemplary and not restrictive. Those skilled in the art should understand that, unless the context clearly indicates otherwise, it should be understood as "one or more".

[0016] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for the purpose of illustration and are not intended to limit the scope of these messages or information.

[0017] Most users edit pictures or videos using image editing software or video editing software. However, the pictures or videos edited by image editing software or video editing software are flat pictures or videos and cannot meet the display requirements for the three-dimensional display of the user's pictures or videos.

[0018] Taking attaching a photo frame to a picture as an example, when a user uses image editing software to attach a photo frame to a picture, the attached photo frame is a flat photo frame, and the picture with the photo frame attached has no three-dimensional sense, resulting in a low user experience.

[0019] There are several methods for generating a three-dimensional picture that can attach a three-dimensional photo frame to a picture. However, in the conventional methods for generating three-dimensional images, it is necessary to use a depth estimation algorithm and a three-dimensional (3D) image affine transformation algorithm. However, the above algorithms have a relatively high computational complexity, and the effect depends on the accuracy of the depth estimation algorithm. If the depth estimation algorithm is inaccurate, it may cause distortion of the picture.

[0020] Furthermore, in some movies and TV works, a stereoscopic photo frame can be added to the video by editing, but the added photo frame only creates a pseudo-3D effect by utilizing the occlusion relationship between the video image and the photo frame.

[0021] In view of the above problems, embodiments of the present disclosure provide an image processing method, apparatus, device, and medium. First, the image processing method will be described below with reference to specific embodiments.

[0022] FIG. 1 is a flowchart of an image processing method according to an embodiment of the present disclosure.

[0023] In embodiments of the present disclosure, the image processing method may be executed by an image processing device. The image processing device may be an electronic device or a server. The electronic device may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet PCs), PMPs (Portable Multimedia Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable devices, etc., and fixed terminals such as digital TVs, desktop computers, smart home devices, etc. The server may be an independent server, a cluster of multiple servers, or may include a locally constructed server and a server mounted in the cloud.

[0024] As shown in FIG. 1, the image processing method mainly includes the following steps.

[0025] S110: Obtain an image to be processed.

[0026] In embodiments of the present disclosure, the image processing device can obtain an image to be processed and perform stereoscopic image processing on the image to be processed.

[0027] Here, the image to be processed may be an image that requires three-dimensional image processing. The image to be processed may be one image frame in one picture or video.

[0028] When the image processing device is an electronic device, the image to be processed may be one image frame of one picture or one video uploaded or taken by the user, may be one image frame of one picture or one video downloaded from the server, or may be one image frame of one picture or one video transmitted to it by another device.

[0029] When the image processing device is a server, the image to be processed may be one image frame of one picture or one video carried in an image processing request transmitted to it by another device.

[0030] S120: Perform main subject recognition on the image to be processed to obtain an image main subject object.

[0031] In a normal case, when taking a picture or video, the photographer generally designs elements such as the image main subject object, image secondary subject object, image foreground, image background, and image margin in the screen. And when the user edits an image, in most cases, the user wants to edit the image main subject object in the image.

[0032] Taking the example of adding a photo frame to the image to be processed, in a normal case, the user wants to add a photo frame to the image main subject object. Therefore, in the embodiments of the present disclosure, after the image processing device acquires the image to be processed, it is first necessary to recognize the image main subject object in the image to be processed.

[0033] The image main subject object may be an object mainly represented in the image to be processed. The image main subject object may be a person, may be an object, may be one person or one object, or may be a plurality of people or a plurality of objects.

[0034] In some embodiments, the main subject recognition can be performed on the image to be processed by an image segmentation algorithm, and the main subject object of the image can be obtained.

[0035] Specifically, the image segmentation algorithm divides the image to be processed into a plurality of specific regions with unique characteristics, and finds the region where the main subject object of the image is located from the above-mentioned plurality of regions.

[0036] In a normal case, since the region where the main subject object of the image is located is the foreground element and the other regions are background elements, all the contents included in the foreground element are used as the main subject object of the image. The image segmentation algorithm includes a threshold-based segmentation algorithm, a region-based segmentation method, etc., and the embodiments of the present disclosure do not limit a specific image segmentation algorithm.

[0037] For example, as shown in FIG. 2, the picture shown in FIG. 2 is the image to be processed, and the regions where image elements such as a person, a mountain, and the sun are located can be recognized by using an image segmentation algorithm. The region where the person is located is the foreground element. Therefore, the person in the image to be processed is used as the main subject object of the image.

[0038] In some other embodiments, the image to be processed is input into a pre-trained main subject recognition model for recognizing the main subject of the image, and the main subject object of the image to be processed is directly recognized by the main subject recognition model, and the main subject object output by the main subject recognition model is obtained.

[0039] S130: Generate a target polygon region corresponding to the main subject object of the image, a part of the main subject object of the image is located within the target polygon region, and the remaining part of the main subject object of the image is located outside the target polygon region.

[0040] In an embodiment of the present disclosure, after obtaining an image main subject object, an image processing device can generate a target polygon area corresponding to the image main subject object.

[0041] The target polygon area can be used to fuse with the image to be processed to achieve the effect of a photo frame. Also, the position and size of the target polygon area can be determined according to the image main subject object, with a part of the image main subject object located within the target polygon area and the remaining part located outside the target polygon area, thereby realizing the three-dimensional display effect of the image main subject object in the photo frame.

[0042] Furthermore, the target polygon area may be a polygon area of any shape such as a quadrilateral area or a circular area.

[0043] Specifically, the image processing device generates one material image with the same size as the image to be processed, and generates a target polygon area in the material image according to the relative position and relative size of the image main subject object in the image to be processed. The relative position of the target polygon area in the material image is appropriate for the relative position of the image area object in the image to be processed, and the relative size of the target polygon area in the material image adapts to the relative size of the image main subject object in the image to be processed, with a part of the image main subject object located within the target polygon area and the remaining part located outside the target polygon area.

[0044] For example, if the target polygon area is a quadrilateral area, the target polygon area generated according to the image main subject object is shown in FIG. 3.

[0045] S140: According to the target polygon region, fuse the image to be processed with additional image materials to obtain a fused image. The fused image is consistent with the image content within the first image region of the image to be processed. The image content within the second image region of the fused image presents the additional image materials. The first image region is the merged region of the target polygon region and the image region occupied by the image main subject object, and the second image region is the image region other than the first image region.

[0046] In the embodiments of the present disclosure, after generating the target polygon region, as shown in FIG. 4, according to the target polygon region, the image to be processed can be fused with additional image materials to obtain a fused image.

[0047] The fused image is consistent with the image content within the first image region of the image to be processed. The first image region is the merged region of the target polygon region and the image region occupied by the image main subject object. Thereby, the target polygon region for realizing the photo frame effect in the fused image and the background image content and the image main subject object located within the target polygon region in the image to be processed can be displayed. A part of the image main subject object is located within the target polygon region together with the background image content, and the remaining part of the image main subject object is located outside the target polygon region, realizing the effect of a three-dimensional photo frame.

[0048] Furthermore, the additional image materials may be preset background materials or background materials selected by the user. The image content within the second image region of the fused image presents the additional image materials. The second image region is the image region other than the first image region. Thereby, the image content other than the first image region of the background material can be used as the photo frame background of the three-dimensional photo frame, further improving the aesthetic property of the three-dimensional photo frame effect.

[0049] When the image processing device is an electronic device, the obtained fused image can be directly displayed on the electronic device.

[0050] When the image processing device is a server, the server can send the obtained fused image to other devices, and the other devices can receive and display the image sent from the server.

[0051] In the embodiments of the present disclosure, after recognizing the main image subject object in the image to be processed and further generating a target polygon region that can position the remaining part outside a part of the main image subject object, according to the target polygon region, the image to be processed can be fused with additional image materials to obtain a fused image, so that the image content of the target polygon region of the fused image and the image to be processed is made to coincide with the image content within the first image region occupied by the main image subject object, and the image content within the second image region outside the first image region of the fused image presents additional image materials, and further presents the three-dimensional display effect of the main image subject object with respect to the target polygon region, meeting the display requirements of the user for the three-dimensional display of the image and improving the user experience.

[0052] In another embodiment of the present disclosure, S130 may specifically include the steps of determining the minimum circumscribed rectangle region corresponding to the main image subject object, and based on the minimum circumscribed rectangle region, performing region adjustment on a preset polygon region to obtain a target polygon region, where the region adjustment includes size adjustment and position adjustment.

[0053] In some embodiments of the present disclosure, the image processing device first performs a masking process on the main image subject object to obtain a mask image of the main image subject object, and then calculates a minimum circumscribed rectangle frame according to the mask image of the main image subject object to obtain a minimum circumscribed rectangle region.

[0054] For example, the image processing device sets each pixel value of the image main subject object in the image to be processed to 255, sets the mask value corresponding to each pixel value to 1, and thus, the image main subject object in the image to be processed is displayed in white in the image to be processed. Each pixel value of the non-image main subject object in the image to be processed is set to 255, and the mask value corresponding to each pixel value is set to 0. Thus, the non-image main subject object in the image to be processed is displayed in black in the image to be processed, and a mask image of the image main subject object can be obtained by this step.

[0055] Optionally, performing mask processing on the image main subject object may be to set the mask value to another value, and the embodiments of the present disclosure do not limit this.

[0056] For example, the mask image obtained by processing the image main subject object shown in FIG. 2 is shown in FIG. 5.

[0057] Next, the step of calculating the minimum bounding rectangle frame according to the mask value of the image main subject object specifically includes that the image processing device calculates the sum of the pixel values of each column and the sum of the pixel values of each row of the mask image, and obtains an array A composed of the sum of the pixel values of each column and an array B composed of the sum of the pixel values of each row, respectively.

[0058] After obtaining the arrays A and B, the image processing device traverses from the first element of the array A backward to find the position of the first non-zero element, and further obtains the abscissa P l of the left boundary point of the image main subject object. Traverse from the last element of the array A forward to find the position of the first non-zero element, and further obtain the abscissa P r of the right boundary point of the image main subject object. Traverse from the first element of the array B backward to find the position of the first non-zero element, and further obtain the ordinate P t of the upper boundary point of the image main subject object. Traverse from the last element of the array B forward to find the position of the first non-zero element, and further obtain the ordinate P of the lower boundary point of the image main subject objectb is obtained. Taking the upper left corner of the image to be processed as the coordinate origin, the abscissa P of the left boundary point l and the abscissa P of the right boundary point r are used to create parallel lines to the Y-axis, and the left boundary line and the right boundary line are obtained. The ordinate P of the upper boundary point t and the ordinate P of the lower boundary point b are used to create parallel lines to the X-axis, and the upper boundary line and the lower boundary line are obtained. The rectangular frame composed of the left boundary line, the right boundary line, the upper boundary line, and the lower boundary line is used as the minimum circumscribed rectangular frame, and the area formed by the minimum circumscribed rectangular frame is the minimum circumscribed rectangular area.

[0059] For example, the minimum circumscribed rectangular area calculated according to the mask image of the main subject object of the image shown in FIG. 5 is shown in FIG. 6.

[0060] In some other embodiments of the present disclosure, after obtaining the minimum circumscribed rectangular area, the image processing device can further perform area adjustment on a preset polygonal area based on the minimum circumscribed rectangular area to obtain a target polygonal area.

[0061] The preset polygonal area may be a polygonal area having a 3D effect. For example, the preset polygonal area may be a quadrilateral area having a 3D effect or a circular area having a 3D effect.

[0062] The image processing device adjusts the preset polygonal area in the preset material image according to the obtained minimum circumscribed rectangular area, and further obtains a polygonal area that matches the minimum circumscribed rectangular area. Taking the polygonal area that matches the minimum circumscribed rectangular area as the target polygonal area, a material image having the target polygonal area can be obtained.

[0063] Specifically, after obtaining the preset material image, the image processing device first trims the preset material image according to the image size of the image to be processed, so that the preset material image has the same image size as the image to be processed.

[0064] Next, the image processing device matches the adjusted target polygon region to the minimum bounding rectangle region by adjusting a preset polygon region in a preset material image according to the minimum bounding rectangle region.

[0065] The region adjustment includes size adjustment and position adjustment. That is, performing material adjustment on a preset polygon region includes adjusting the size and position of the preset polygon region.

[0066] For example, the preset polygon region may be located at the center of the preset material image by default, and the image processing device adjusts the size and position of the preset polygon region in the preset material image according to the size and position of the minimum bounding rectangle region in the image to be processed.

[0067] For example, the preset polygon region is a quadrilateral region such as an isosceles trapezoid. After adjusting the size and position of the preset polygon region, a quadrilateral region with a 3D effect that matches the image main subject object, which is the minimum bounding rectangle region, is obtained. Also, for example, the preset polygon region is a circular region such as an ellipse. After adjusting the size and position of the preset polygon region, a circular region with a 3D effect that matches the image main subject object, which is the minimum bounding rectangle region, is obtained.

[0068] In still some other embodiments of the present disclosure, when the preset polygon region is a quadrilateral region, FIG. 7 shows a flowchart of a processing process of performing region adjustment on the preset polygon region according to the minimum bounding rectangle region according to an embodiment of the present disclosure to obtain a target polygon region.

[0069] As shown in FIG. 7, the processing process may include the following several steps.

[0070] S710: Obtain the corner coordinates of each rectangular region corner point of the minimum bounding rectangle region.

[0071] The corner points of the rectangular region are the vertex points of each region of the minimum bounding rectangle region.

[0072] In an embodiment of the present disclosure, with the upper left corner of the image to be processed as the coordinate origin, after the image processing device obtains the minimum bounding rectangle region, it can determine the coordinates of each rectangular region corner point of the minimum bounding rectangle region according to each boundary point of the minimum bounding rectangle region.

[0073] Taking FIG. 6 as an example, according to the abscissa P1 of the left boundary point, the abscissa P r of the right boundary point, the ordinate P t of the upper boundary point, and the ordinate P b of the lower boundary point in FIG. 6, the corner coordinates of the four rectangular region corner points of the minimum bounding rectangle region in FIG. 6 can be obtained. Specifically, the coordinates of the upper left vertex are (P l , P t ), the coordinates of the upper right vertex are (P r , P t ), the coordinates of the lower left vertex are (P l , P b ), and the coordinates of the lower right vertex are (P r , P b ).

[0074] S720: Calculate the corner coordinates of each polygon region corner point according to the corner coordinates of each rectangular region corner point.

[0075] In an embodiment of the present disclosure, after the image processing device obtains the corner coordinates of each rectangular region corner point of the minimum bounding rectangle region, it calculates the corner coordinates of each polygon region corner point according to the corner coordinates of each rectangular region corner point.

[0076] Specifically, for the image processing device, the functional relationship between the corner coordinates of each rectangular region corner point of the minimum bounding rectangle region and the corner coordinates of each polygon region corner point of the polygon region is preset. The image processing device substitutes the corner coordinates of each rectangular region corner point of the minimum bounding rectangle region into the above functional relationship to obtain the corner coordinates of each polygon region corner point of the polygon region.

[0077] S730: Perform region adjustment on the preset polygon region according to the corner coordinates of each polygon region corner point to obtain the target polygon region.

[0078] In an embodiment of the present disclosure, the image processing device calculates a vector including a moving direction and a moving distance according to the corner coordinates of each corner of the polygon region and the corner coordinates of each preset corner of the preset polygon region, where each preset corner of the preset polygon region needs to move. Next, according to the vector, each preset corner of the preset polygon region is moved to realize the size adjustment and position adjustment of the preset polygon region, and further a target polygon region is obtained.

[0079] As can be seen from the above, the embodiment of the present disclosure obtains the corner coordinates of each corner of the polygon region according to the functional relationship between the corner coordinates of each rectangle corner of the minimum circumscribed rectangle region and the corner coordinates of each polygon corner of the polygon region, calculates the vector that each preset corner of the preset polygon region needs to move, and performs region adjustment on the preset polygon region based on the vector to obtain a target polygon region, so that the target polygon region can be obtained only by simple calculation, improving the calculation efficiency.

[0080] In still some other embodiments of the present disclosure, when the preset polygon region is a circular region, FIG. 8 shows a flowchart of another processing process for performing region adjustment on the preset polygon region according to the minimum circumscribed rectangle region according to the embodiment of the present disclosure to obtain a target polygon region.

[0081] As shown in FIG. 8, the processing process may include the following steps.

[0082] S810: Obtain the corner coordinates of each rectangle corner of the minimum circumscribed rectangle region and the center coordinates of the rectangle region center.

[0083] The rectangle corner is each vertex of the minimum circumscribed rectangle region, and the rectangle region center is the center point of the minimum circumscribed rectangle region.

[0084] In an embodiment of the present disclosure, with the upper left corner of the image to be processed as the coordinate origin, after the image processing device obtains the minimum bounding rectangle region, it can determine the coordinates of each rectangular region corner point and the central coordinates of the rectangular region center of the minimum bounding rectangle region according to each boundary point of the minimum bounding rectangle region.

[0085] Continuing to take FIG. 6 as an example, according to the abscissa P1 of the left boundary point, the abscissa P r of the right boundary point, the ordinate P t of the upper boundary point, and the ordinate P b of the lower boundary point in FIG. 6, the corner coordinates of the four rectangular region corner points and the central coordinates of the rectangular region center of the minimum bounding rectangle region in FIG. 6 can be obtained. Specifically, the coordinates of the upper left vertex are (P l , P t ), the coordinates of the upper right vertex are (P r , P t ), the coordinates of the lower left vertex are (P l , P b ), the coordinates of the lower right vertex are (P r , P b ), and the central coordinates are

Number

[0086] S820: Calculate the maximum distance from the rectangular region center to each rectangular region boundary of the minimum bounding rectangle region according to the corner coordinates of each rectangular region corner point and the central coordinates of the rectangular region center.

[0087] In an embodiment of the present disclosure, the image processing device connects the adjacent rectangular region corner points of the minimum bounding rectangle region to obtain each region boundary of the minimum bounding rectangle region.

[0088] Furthermore, the image processing device calculates the distance from the rectangular region center to each rectangular region boundary of the minimum bounding rectangle region according to the central coordinates of the rectangular region center and each region boundary of the minimum bounding rectangle region, and selects the maximum distance from the calculated distances.

[0089] S830: Set the maximum distance as the boundary distance from the center of the polygon region to each polygon region boundary.

[0090] In an embodiment of the present disclosure, after determining the maximum distance from the center of the rectangular region to each rectangular region boundary of the minimum bounding rectangle region, the image processing device can use the maximum distance as the boundary distance from the center of the polygonal region to each polygonal region boundary.

[0091] S840: According to the center coordinates of the center of the rectangular region and the boundary distance, perform region adjustment on a preset polygonal region to obtain a target polygonal region.

[0092] In an embodiment of the present disclosure, the image processing device calculates a vector including a moving direction and a moving distance that the center point of the preset polygonal region needs to move according to the center coordinates of the center of the rectangular region. Next, according to the vector, move the preset polygonal region to realize the position adjustment of the preset polygonal region. Subsequently, according to the maximum distance, adjust the polygonal boundary of the preset polygonal region to realize the size adjustment of the preset polygonal region, and further obtain a target polygonal region.

[0093] Specifically, for the image processing device, the functional relationship between the center coordinates of the center of the rectangular region of the minimum bounding rectangle region and the center coordinates of the center point of the polygonal region is preset. The image processing device substitutes the center coordinates of the center of the rectangular region of the minimum bounding rectangle region into the above functional relationship to obtain the center coordinates of the center point of the polygonal region.

[0094] Furthermore, for the image processing device, the functional relationship between the boundary distance and the adjustment distance of the polygonal boundary of the polygonal region is preset. The image processing device substitutes the boundary distance into the above functional relationship to obtain the adjustment distance of the polygonal boundary of the polygonal region, and further expands or shrinks the polygonal boundary of the polygonal region according to the adjustment distance to expand and contract the preset polygonal region and realize the size adjustment of the preset polygonal region.

[0095] As can be seen from the above, in the embodiments of the present disclosure, according to the functional relationship between the central coordinates of the center of the rectangular region of the minimum circumscribed rectangle region and the central coordinates of the center of the polygon of the polygon region, the central coordinates of the center of the polygon of the polygon region are obtained, and the vector that the center of the polygon of the preset polygon region needs to move is calculated. At the same time, according to the functional relationship between the boundary distance and the adjustment distance of the polygon boundary of the polygon region, the adjustment distance of the polygon boundary of the polygon region is obtained. Based on the vector and the adjustment distance, region adjustment is performed on the preset polygon region to obtain a target polygon region, so that the target polygon region can be obtained only by simple calculation, improving the calculation efficiency.

[0096] In still another embodiment of the present disclosure, when the generated target polygon region is a quadrilateral region such as an isosceles trapezoid region, FIG. 9 shows a flowchart of a processing process for generating a target polygon region according to the minimum circumscribed rectangle region and the preset corner point mapping relationship according to the embodiments of the present disclosure.

[0097] As shown in FIG. 9, the processing process may include the following several steps.

[0098] S910: Determine the minimum circumscribed rectangle region corresponding to the main subject object of the image.

[0099] In some embodiments of the present disclosure, the image processing device first performs a masking process on the main subject object of the image to obtain a masked image of the main subject object of the image. Then, according to the masked image of the main subject object of the image, a minimum circumscribed rectangle frame is calculated, and the minimum circumscribed rectangle region can be obtained. Here, repeated descriptions are omitted.

[0100] S920: Obtain the corner coordinates of each rectangular region corner of the minimum circumscribed rectangle region.

[0101] In some embodiments of the present disclosure, with the upper left corner of the image to be processed as the coordinate origin, after the image processing device obtains the minimum bounding rectangle region, it can determine the coordinates of each rectangular region corner point of the minimum bounding rectangle region according to each boundary point of the minimum bounding rectangle region, and the repeated description is omitted here.

[0102] S930: Calculate the corner coordinates of the polygon region corner points corresponding to the corner coordinates of each rectangular region corner point according to a preset corner mapping relationship.

[0103] In some embodiments of the present disclosure, the image processing device has a preset corner mapping relationship, and the preset corner mapping relationship includes the correspondence between each rectangular region corner point and each polygon region corner point of the minimum bounding rectangle region.

[0104] Specifically, the preset corner mapping relationship may be a functional relationship between the corner coordinates of each rectangular region corner point of the preset minimum bounding rectangle region and the corner coordinates of the corresponding polygon region corner point.

[0105] The image processing device substitutes the corner coordinates of each rectangular region corner point of the minimum bounding rectangle region into the above functional relationship to obtain the corner coordinates of the polygon region corner points corresponding to the corner coordinates of each rectangular region corner point.

[0106] S940: Generate a target polygon region according to the corner coordinates of the polygon region corner points.

[0107] In an embodiment of the present disclosure, the image processing device can generate a material image having a target polygon region according to the corner coordinates of the polygon region corner points.

[0108] The image processing device first generates a canvas of the same size as the image to be processed, generates corner points at corresponding positions in the canvas according to the corner coordinates of the polygon region corner points, performs internal filling on the polygon region based on the generated corner points, and obtains a target polygon region.

[0109] In an embodiment of the present disclosure, after obtaining the corner coordinates of the corner points of the polygonal region, the image processing device connects the corner points of the polygonal region, fills the interior of the connection region formed by connecting adjacent corner points, and obtains a target polygonal region.

[0110] The filling may be to fill the pixels in the connection region with the same pixel value.

[0111] Optionally, the pixel value for filling the connection region may be 255, that is, the target polygonal region is displayed in white.

[0112] As can be seen from the above, the embodiments of the present disclosure can directly generate a target polygonal region according to the corner coordinates of each rectangular region corner point of the minimum circumscribed rectangle region and a preset corner mapping relationship, thereby obtaining the target polygonal region with only simple calculations and improving the calculation efficiency.

[0113] In an embodiment of the present disclosure, optionally, before calculating the corner coordinates of the corner points of the polygonal region corresponding to the corner coordinates of each rectangular region corner point according to a preset corner mapping relationship, a step of determining the region display direction of the target polygonal region with respect to the image main subject object and a step of obtaining a preset corner mapping relationship corresponding to the region display direction are further included.

[0114] The region display direction may be the position of the target polygonal region with respect to the image main subject object. The target polygonal region can be arranged in any direction above, below, to the left, and to the right of the image main subject object.

[0115] In some embodiments, the calculation formula of the preset corner mapping relationship may be as follows.

[0116] The abscissa of the corner point at the upper left corner of the polygon

Equation

Number

Number

Number

Number

Number

Number

Number

[0117] The calculation formula of the preset corner point mapping relationship realizes arranging the target polygon area below the main object of the image, and forms a three-dimensional effect in which the main object of the image protrudes upward with respect to the photo frame.

[0118] In some other embodiments, the calculation formula of the preset corner point mapping relationship may be as follows.

[0119] The abscissa of the upper left corner point of the polygon

Number

Number

Number

Number

Number

Number

Number

Number

[0120] The calculation formula of the preset corner point mapping relationship realizes arranging the target polygon area to the left of the main subject object in the image, and forms a three-dimensional effect that the main subject object in the image protrudes to the right with respect to the photo frame.

[0121] In some other embodiments, the calculation formula of the preset corner point mapping relationship may be as follows.

[0122] The horizontal coordinate of the upper left corner point of the polygon

Number

Number

Number

Number

Number

Number

Number

Number

[0123] The calculation formula of the preset corner point mapping relationship realizes arranging the target polygon area to the right of the main object of the image, and forms a three-dimensional effect that the main object of the image protrudes to the left with respect to the photo frame.

[0124] Thereby, in the embodiments of the present disclosure, different three-dimensional effects can be realized based on different region display directions, improving the diversity of image processing effects.

[0125] In another embodiment of the present disclosure, S140 may specifically include the steps of creating additional pixel materials of the same size as the image to be processed, extracting the first image content located outside the target polygon area from the additional pixel materials to obtain the first background image, filling the second image content located inside the target polygon area in the image to be processed into the first background image to obtain the second background image, and filling the main object of the image into the second background image to obtain the fused image.

[0126] In an embodiment of the present disclosure, the image processing device first creates additional image materials of the same size as the image to be processed. Specifically, the creation method may generate additional image materials of the same size as the image to be processed according to a preset color, a color specified by the user, and an image generation method such as a preset single color or a gradient color. It is also possible to obtain a preset background image, perform image clipping on the periphery of the preset background image with the preset background image as the center, and obtain additional image materials of the same size as the image to be processed.

[0127] The image processing device further extracts first image content located outside the target polygon region from the additional image materials, fills it in the image region located outside the target polygon region of the material image having the target polygon region, and obtains a first background image, that is, maps the first image content located outside the target polygon region in the additional image materials to the material image having the target polygon region to obtain a first background image.

[0128] In some examples, the step of extracting the first image content located outside the target polygon region from the additional image materials and obtaining the first background image specifically includes the step of obtaining the region boundary position of the target polygon region, the step of extracting the first image content located outside the region boundary position from the additional image materials, and the step of filling the first image content into the corresponding position in the material image having the target polygon region to obtain the first background image.

[0129] The step of obtaining the region boundary position of the target polygon region may specifically include the step of performing a mask process on the target polygon region to obtain a mask image of the target polygon region.

[0130] For example, the image processing device sets each pixel value within the target polygon region in the material image having the target polygon region to 255, sets the mask value corresponding to each pixel value to 1, and thus, the target polygon region is displayed in white. Each pixel value within the non-target polygon region corresponding to the material image of the target polygon region is set to 0, and the mask value corresponding to each pixel value is set to 0. Thus, the non-target polygon region corresponding to the material image having the target polygon region is displayed in black, and a mask image of the target polygon region can be obtained by this step. Thereby, a mask image of the first background image can be obtained.

[0131] For example, performing mask processing on the target polygon region shown in FIG. 3 to obtain a mask image of the target polygon region is shown in FIG. 10.

[0132] In some embodiments, in order to more stereoscopically display the image main subject content, a morphological dilation process can be performed on the mask image of the target polygon region. The morphological dilation operation may be to expand the boundary of the target polygon region.

[0133] Specifically, a specific implementation method for the image processing device to attach the first image content to the mask image of the target polygon region is

Number

Number

Number

Number

[0134] For example, the mask image of the first background image obtained by performing morphological dilation processing on the mask image of the target polygonal region shown in FIG. 10 is shown in FIG. 11.

[0135] Furthermore, the image processing device then fills the first background image with the second image content located inside the target polygonal region in the image to be processed, to obtain a second background image, that is, maps the second image content located inside the target polygonal region in the image to be processed to the first background image, to obtain a second background image.

[0136] In some examples, the step of filling the first background image with the second image content located inside the target polygonal region in the image to be processed to obtain a second background image may specifically include: the step of obtaining the region boundary position of the target polygonal region that has not been subjected to morphological dilation processing; the step of extracting the second image content located inside the region boundary position from the image to be processed; and the step of filling the second image content into the corresponding position in the first background image to obtain a second background image.

[0137] For example, the image processing device fills the second image content into the corresponding position in the mask image of the first background image to obtain a second background image.

[0138] Specifically, the specific implementation method for the image processing device to attach the second image content to the mask image of the first background image is

Number

Number

Number

Number

[0139] For example, the second background image obtained by adding second image content to the first background image shown in FIG. 11 is shown in FIG. 12.

[0140] Furthermore, the image processing device then fills the second background image with the image main subject object to obtain a fused image, that is, maps the image main subject object in the image to be processed to the second background image to obtain a fused image.

[0141] In some examples, the step of filling the second background image with the image main subject object to obtain a fused image may specifically include the step of obtaining the object boundary position of the image main subject object, the step of extracting the main subject object content located inside the object boundary position from the image to be processed, and the step of filling the main subject object content into the corresponding position in the second background image to obtain a fused image.

[0142] For example, the image processing device fills the main subject object content into the corresponding position in the second background image to obtain a fused image.

[0143] Specifically, the specific implementation method for the image processing device to add the main subject object content to the second background image is

Number

Number

Number

Number

[0144] For example, the fused image obtained by attaching the main subject object content to the second background image shown in FIG. 12 is shown in FIG. 4.

[0145] Thereby, in the embodiments of the present disclosure, the mask image of the target polygon region can quickly and accurately fuse the image to be processed and the additional image materials, and a fused image with high image quality and fusion quality can be obtained.

[0146] FIG. 13 is a structural schematic diagram of an image processing apparatus according to an embodiment of the present disclosure. The image processing apparatus according to the embodiment of the present disclosure can execute the processing process according to the embodiment of the image processing method. As shown in FIG. 13, the image processing apparatus 1300 includes an image acquisition module 1310, a main subject recognition module 1320, a region generation module 1330, and an image fusion module 1340.

[0147] The image acquisition module 1310 can be configured to acquire the image to be processed.

[0148] The main subject recognition module 1320 can be configured to perform main subject recognition on the image to be processed to obtain the main subject object of the image.

[0149] The region generation module 1330 can be configured to generate a target polygon region corresponding to the main subject object of the image, a part of the main subject object of the image is located within the target polygon region, and the remaining part of the main subject object of the image is located outside the target polygon region.

[0150] The image fusion module 1340 can be configured to fuse the image to be processed with additional image materials according to the target polygon region to obtain a fused image. The fused image is consistent with the image content within the first image region of the image to be processed, and the image content within the second image region of the fused image presents the additional image materials. The first image region is the merged region of the target polygon region and the image region occupied by the image main subject object, and the second image region is the image region other than the first image region.

[0151] In the embodiments of the present disclosure, after recognizing the image main subject object in the image to be processed and further generating a target polygon region that can position the remaining part inside and the other part outside of the image main subject object, the image to be processed can be fused with additional image materials according to the target polygon region to obtain a fused image. Thereby, the image content of the target polygon region of the fused image and the image to be processed is made consistent with the image content within the first image region occupied by the image main subject object, and the image content within the second image region other than the first image region of the fused image presents the additional image materials. Furthermore, the three-dimensional display effect of the image main subject object with respect to the target polygon region is presented, meeting the display requirements of the user for the three-dimensional display of the image and improving the user experience.

[0152] In some embodiments of the present disclosure, the region generation module 1330 may include a region determination unit and a region adjustment unit.

[0153] The region determination unit can be configured to determine the minimum bounding rectangle region corresponding to the image main subject object.

[0154] The region adjustment unit can be configured to perform region adjustment on the preset polygon region based on the minimum bounding rectangle region to obtain the target polygon region. The region adjustment includes size adjustment and position adjustment.

[0155] In some embodiments of the present disclosure, the region adjustment unit is further configured to obtain the corner coordinates of each corner point of the minimum circumscribed rectangle region, calculate the corner coordinates of each corner point of the polygon region according to the corner coordinates of each corner point of the rectangle region, and perform region adjustment on a preset polygon region according to the corner coordinates of each corner point of the polygon region to obtain a target polygon region.

[0156] In some embodiments of the present disclosure, the region adjustment unit is further configured to obtain the corner coordinates of each corner point of the minimum circumscribed rectangle region and the center coordinates of the center of the rectangle region, calculate the maximum distance from the center of the rectangle region to each boundary of the minimum circumscribed rectangle region according to the corner coordinates of each corner point of the rectangle region and the center coordinates of the center of the rectangle region, use the maximum distance as the boundary distance from the center of the polygon region to each boundary of the polygon region, and perform region adjustment on a preset polygon region according to the center coordinates of the center of the rectangle region and the boundary distance to obtain a target polygon region.

[0157] In some embodiments of the present disclosure, the region generation module 1330 may further include a region determination unit, a coordinate acquisition unit, a coordinate calculation unit, and a region generation unit.

[0158] The region determination unit may be configured to determine a minimum circumscribed rectangle region corresponding to the main subject object in the image.

[0159] The coordinate acquisition unit may be configured to acquire the corner coordinates of each corner point of the minimum circumscribed rectangle region.

[0160] The coordinate calculation unit may be configured to calculate the corner coordinates of the corner points of the polygon region corresponding to the corner coordinates of each corner point of the rectangle region according to a preset corner mapping relationship. The region generation unit may be configured to generate a target polygon region according to the corner coordinates of the corner points of the polygon region.

[0161] In some embodiments of the present disclosure, the region generation module 1330 may further include a direction determination unit and a relationship acquisition unit.

[0162] The direction determination unit can be configured to determine the region display direction of the target polygon region with respect to the image main subject object.

[0163] The relationship acquisition unit can be configured to acquire a preset corner point mapping relationship corresponding to the region display direction.

[0164] In some embodiments of the present disclosure, the image fusion module 1340 may include a material creation unit, a background extraction unit, a first filling unit, and a second filling unit.

[0165] The material creation unit can be configured to create additional image materials of the same size as the image to be processed.

[0166] The background extraction unit can be configured to extract the first image content located outside the target polygon region from the additional image materials to obtain a first background image.

[0167] The first filling unit can be configured to fill the second image content located inside the target polygon region in the image to be processed into the first background image to obtain a second background image.

[0168] The second filling unit can be configured to fill the image main subject object into the second background image to obtain a fused image.

[0169] It should be noted that the image processing apparatus 1300 shown in FIG. 13 can execute each step of the method embodiments shown in FIGS. 1 to 12 and realize each process and effect of the method embodiments shown in FIGS. 1 to 12, and duplicate descriptions are omitted here.

[0170] Embodiments of the present disclosure further provide an image processing device, which may include a processor and a memory, and the memory can be used to store executable instructions. The processor can read the executable instructions from the memory and execute the executable instructions to implement the image processing method in the above embodiments.

[0171] FIG. 14 shows a structural schematic diagram of an image processing device according to an embodiment of the present disclosure. Hereinafter, as specifically referred to in FIG. 14, a structural schematic diagram suitable for implementing the image processing device 1400 in the embodiment of the present disclosure is shown.

[0172] The image processing device 1400 in the embodiment of the present disclosure may be an electronic device or a server. Here, the electronic device may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs, PADs, PMPs, in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable devices, etc., and fixed terminals such as digital TVs, desktop computers, smart home devices, etc. The server may be an independent server, or a cluster of multiple servers, and may include a locally constructed server and a server mounted on the cloud.

[0173] It should be noted that the image processing device 1400 shown in FIG. 14 is only an example and does not impose any limitations on the functions and usage ranges of the embodiments of the present disclosure.

[0174] As shown in FIG. 14, the image processing apparatus 1400 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 1401, and may execute various appropriate operations and processes according to a program stored in a read-only memory (ROM) 1402 or a program loaded from a storage device 1408 into a random access memory (RAM) 1403. The RAM 1403 further stores various programs and data necessary for the operation of the information processing apparatus 1400. The processing device 1601, the ROM 1402, and the RAM 1403 are connected to each other via a bus 1604. An input / output (I / O) interface 1405 is also connected to the bus 1404.

[0175] Normally, the devices connected to the I / O interface 1405 include an input device 1406 such as a touch panel, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc., an output device 1607 such as a liquid crystal display (LCD), a speaker, a vibrator, etc., a storage device 1408 such as a tape, a hard disk, etc., and a communication device 1409. The communication device 1409 enables the image processing apparatus 1400 to communicate wirelessly or wiredly with other devices for data exchange. Although FIG. 14 shows an image processing apparatus 1400 having various devices, it should be understood that it is not necessary to implement or include all of the shown devices. Instead, more or fewer devices may be implemented or included.

[0176] Embodiments of the present disclosure further provide a computer-readable medium, on which a computer program is stored, and when the computer program is executed by a processor, the image processing method in the above embodiments is realized by the processor.

[0177] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product including a computer program mounted on a non-transitory computer-readable medium, the computer program including program code for executing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication device 1409, or may be installed from the storage device 1408, or may be installed from the ROM 1402. When the computer program is executed by the processing device 1401, the above functions limited to the image processing method of the embodiment of the present disclosure are executed.

[0178] Note that the computer-readable medium of the present disclosure may be a computer-readable signal medium, a computer-readable medium, or any combination of the two. The computer-readable medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of the computer-readable medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, the computer-readable medium may be any tangible medium that includes or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave carrying computer-readable program code. Such a propagated data signal can take various forms including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable medium, and the computer-readable signal medium can transmit, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code included in the computer-readable medium can be transmitted using any suitable medium including, but not limited to, wires, optical cables, RF (radio frequency), or any suitable combination thereof.

[0179] In some embodiments, the client and the server can communicate by any network protocol known currently or developed in the future, such as HTTP for example, and can be interconnected with digital data communication of any form or medium (e.g., communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), Internet networks (e.g., the Internet) and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), and any network known currently or developed in the future.

[0180] The computer-readable medium may be included in the image processing device, or may exist alone without being assembled into the image processing device.

[0181] The computer-readable medium carries one or more programs, and when the one or more programs are executed by the image processing device, the image processing device is caused to perform steps of obtaining an image to be processed, performing main subject recognition on the image to be processed to obtain an image main subject object, generating a target polygonal region corresponding to the image main subject object, wherein a part of the image main subject object is located within the target polygonal region and the remaining part of the image main subject object is located outside the target polygonal region, and fusing the image to be processed with additional image materials according to the target polygonal region to obtain a fused image, wherein the fused image coincides with the image content within a first image region of the image to be processed, the image content within a second image region of the fused image presents the additional image materials, the first image region is a combined region of the target polygonal region and the image region occupied by the image main subject object, and the second image region is an image region other than the first image region.

[0182] In embodiments of the present disclosure, the computer program code for performing the operations of the present disclosure can be programmed in one or more programming languages, or combinations thereof. The programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, etc., and also include conventional procedural programming languages such as the "C" language and similar programming languages. The program code may be executed entirely on the user computer, partially on the user computer, executed as an independent software package, executed partially on the user computer and partially on a remote computer, or executed entirely on a remote computer or server. When a remote computer is involved, the remote computer may be connected to the user computer via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., connected via the Internet using an Internet service provider).

[0183] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation that can be implemented by the systems, methods, and computer program products of various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram can represent a module, program segment, or part of code that includes one or more executable instructions for implementing a given logical function. Note that in some alternative implementations, the functions indicated in the blocks may be performed in an order different from that shown in the drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and in some cases, depending on the relevant functions, may be executed in the reverse order. Note that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing a given function or operation, or may be implemented by a combination of dedicated hardware and computer instructions.

[0184] The units described in the embodiments of the present disclosure may be implemented in the form of software or in the form of hardware. In some cases, the name of the unit does not limit the unit itself.

[0185] The functions described above in this specification may be at least partially executed by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0186] In the context of the present disclosure, a computer-readable medium may be a tangible medium that includes or stores a program that can be used by or in combination with an instruction execution system, apparatus, or device. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable medium may be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination thereof. More specific examples of the computer-readable medium may include, but are not limited to, electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0187] The above description is only an explanation of the preferred embodiments of the present disclosure and the technical principles used. It should be understood by those skilled in the art that the scope of the disclosure according to the present disclosure is not limited to the technical solutions formed by specific combinations of the above technical features, and other technical solutions formed by any combination of the above technical features or their equivalent features are also included within the scope not departing from the gist of the above disclosure. For example, a technical solution formed by replacing the above features with technical features having similar functions (but not limited to) disclosed in the present disclosure.

[0188] Also, although the operations are presented in a particular order, this should not be understood as requiring that the operations be performed in the particular order or sequence shown. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above description, these should not be construed as limiting the scope of the present disclosure. Some features described in the context of individual embodiments may be implemented in combination in a single embodiment. Rather, the various features described in the context of a single embodiment may be implemented separately, or in any suitable sub-combination, in multiple embodiments.

[0189] The present invention has been described in terms of language specific to structural features and / or methodological acts, but it should be understood that the subject matter defined in the claims is not necessarily limited to the specific features or acts described above. Rather, the above specific features and acts are merely exemplary forms for implementing the claims.

Claims

1. An image processing method, comprising: obtaining an image to be processed; performing main subject recognition on the image to be processed to obtain an image main subject object; generating a target polygon region corresponding to the image main subject object, wherein a part of the image main subject object is located within the target polygon region and the remaining part of the image main subject object is located outside the target polygon region; fusing the image to be processed with additional image materials according to the target polygon region to obtain a fused image, wherein the fused image coincides with the image content within a first image region of the image to be processed, the image content within a second image region of the fused image presents the additional image materials, the first image region is a combined region of the target polygon region and the image region occupied by the image main subject object, and the second image region is an image region other than the first image region.

2. The step of generating a target polygon region corresponding to the image main subject object as described above comprises: determining a minimum bounding rectangle region corresponding to the image main subject object; performing region adjustment on a preset polygon region based on the minimum bounding rectangle region to obtain the target polygon region, wherein the region adjustment includes size adjustment and position adjustment. The method according to claim 1.

3. The step of performing region adjustment on a preset polygon region based on the minimum bounding rectangle region to obtain the target polygon region as described above comprises: obtaining the corner coordinates of each rectangle region corner point of the minimum bounding rectangle region; calculating the corner coordinates of each polygon region corner point according to the corner coordinates of each rectangle region corner point; performing region adjustment on the preset polygon region according to the corner coordinates of each polygon region corner point to obtain the target polygon region. The method according to claim 2.

4. The step of performing region adjustment on a preset polygon region based on the minimum bounding rectangle region to obtain the target polygon region as described above comprises: obtaining the corner coordinates of each rectangle region corner point and the center coordinates of the rectangle region center of the minimum bounding rectangle region; Calculating the maximum distance from the center of the rectangular region to each boundary of the minimum circumscribed rectangular region according to the corner coordinates of the corner points of each rectangular region and the center coordinates of the center of the rectangular region; Taking the maximum distance as the boundary distance from the center of the polygonal region to each boundary of the polygonal region; Performing region adjustment on the preset polygonal region according to the center coordinates of the center of the rectangular region and the boundary distance to obtain the target polygonal region, the method according to claim 2.

5. The step of generating a target polygonal region corresponding to the main object of the image is as follows: Determining a minimum circumscribed rectangular region corresponding to the main object of the image; Obtaining the corner coordinates of the corner points of each rectangular region of the minimum circumscribed rectangular region; Calculating the corner coordinates of the corner points of the polygonal region corresponding to the corner coordinates of the corner points of each rectangular region according to a preset corner mapping relationship; Generating the target polygonal region according to the corner coordinates of the corner points of the polygonal region, the method according to claim 1.

6. Before the step of calculating the corner coordinates of the corner points of the polygonal region corresponding to the corner coordinates of the corner points of each rectangular region according to a preset corner mapping relationship, Determining the region display direction of the target polygonal region for the main object of the image; Further obtaining the preset corner mapping relationship corresponding to the region display direction, the method according to claim 5.

7. The step of fusing the image to be processed with additional image materials according to the target polygonal region to obtain a fused image is as follows: Creating the additional image materials of the same size as the image to be processed; Extracting the first image content located outside the target polygonal region from the additional image materials to obtain a first background image; Filling the second image content located inside the target polygonal region in the image to be processed into the first background image to obtain a second background image; Filling the main object of the image into the second background image to obtain the fused image, the method according to any one of claims 1 to 6.

8. An image processing apparatus, An image acquisition module configured to acquire an image to be processed; A main subject recognition module configured to perform main subject recognition on the image to be processed and obtain an image main subject object; An area generation module configured to generate a target polygonal area corresponding to the image main subject object, wherein a part of the image main subject object is located within the target polygonal area and the remaining part of the image main subject object is located outside the target polygonal area; An image fusion module configured to fuse the image to be processed with additional image materials according to the target polygonal area to obtain a fused image, wherein the fused image coincides with the image content within a first image area of the image to be processed, the image content within a second image area of the fused image presents the additional image materials, the first image area is a combined area of the target polygonal area and the image area occupied by the image main subject object, and the second image area is an image area other than the first image area. An image processing apparatus comprising the image fusion module.

9. An image processing device, A processor; A memory for storing executable instructions, and The processor reads the executable instructions from the memory and uses the executable instructions to execute to implement the image processing method according to any one of claims 1 to 7. An image processing device.

10. A computer-readable medium having a computer program stored therein, wherein when the computer program is executed by a processor, the computer-readable medium causes the processor to implement the image processing method according to any one of claims 1 to 7.

Images