Image processing method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose an image processing method and device, electronic equipment and storage medium; including: obtaining the subject image in the input image; obtaining the minimum circumscribed rectangle of the subject image in the input image; obtaining the first vertex and the second vertex of the minimum circumscribed rectangle, and determining the center point of the minimum circumscribed rectangle based on the first vertex and the second vertex; for each pixel point of the material image, based on the first vertex, the second vertex and the center point, the initial position of the pixel point is scaled to obtain the updated position of the pixel point; based on the updated position of each pixel point of the material image, the material image is added in the input image. The updated position of each pixel point of the material image is determined by scaling operation based on the first vertex, the second vertex and the center point, so that the material image will be displayed around the subject image after scaling operation, thereby producing a good contrast effect on the subject image in the input image, which is beneficial to increase user stickiness.

Description

Technical Field

[0001] This application relates to the field of computers, and more specifically to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] To make the process of editing images more fun, image-based applications often offer users the ability to add materials to their pictures. These materials include heart icons, sparkling star patterns, or frames composed of multiple colors.

[0003] However, in the existing technology, the aforementioned materials often only appear in a fixed position on the image, or simply move as the user drags them, resulting in a rather monotonous effect of the combined image, which is not conducive to increasing user stickiness in the aforementioned image-based applications. Summary of the Invention

[0004] This application provides an image processing method, apparatus, electronic device, and storage medium, which can improve the problem of monotonous combined image effects in the prior art, which is not conducive to increasing user stickiness.

[0005] This application provides an image processing method, the method comprising:

[0006] Obtain the main image from the input image;

[0007] Obtain the smallest bounding rectangle of the main image in the input image;

[0008] Obtain the first vertex and the second vertex of the minimum bounding rectangle, and determine the center point of the minimum bounding rectangle based on the first vertex and the second vertex, wherein the first vertex and the second vertex are a pair of vertices located on the diagonal of the minimum bounding rectangle;

[0009] For each pixel in the source image, the initial position of the pixel is scaled based on the first vertex, the second vertex, and the center point to obtain the updated position of the pixel; wherein, the initial position is the position of the pixel when the source image is tiled on the input image;

[0010] The source image is added to the input image based on the updated position of each pixel in the source image.

[0011] This application embodiment also provides an image processing apparatus, the apparatus comprising:

[0012] The main image acquisition unit is used to acquire the main image in the input image;

[0013] A bounding rectangle acquisition unit is used to acquire the smallest bounding rectangle of the main image in the input image;

[0014] The center point determination unit is used to obtain the first vertex and the second vertex of the minimum bounding rectangle, and determine the center point of the minimum bounding rectangle based on the first vertex and the second vertex, wherein the first vertex and the second vertex are a pair of vertices located on the diagonal of the minimum bounding rectangle;

[0015] The position update unit is used to scale the initial position of each pixel in the source image based on the first vertex, the second vertex, and the center point to obtain the updated position of the pixel; wherein, the initial position is the position of the pixel when the source image is tiled on the input image;

[0016] The material adding unit is used to add the material image to the input image based on the updated position of each pixel of the material image.

[0017] In some embodiments, the circumscribed rectangle acquisition unit includes:

[0018] The contour acquisition subunit is used to acquire the contour of the main image;

[0019] The initial selection determines the sub-unit, which is used to determine the initial minimum bounding rectangle of the main image based on the contour;

[0020] The final selection determination subunit is used to perform corresponding movement processing on the border of the initially selected minimum bounding rectangle in response to the movement operation of the border of the initially selected minimum bounding rectangle, to obtain the final selected minimum bounding rectangle, wherein the final selected minimum bounding rectangle is the minimum bounding rectangle of the main image in the input image.

[0021] In some embodiments, the center point determination unit includes:

[0022] The initial vertex acquisition sub-unit is used to acquire the initial first vertex and the initial second vertex of the minimum bounding rectangle;

[0023] The first normalization subunit is used to normalize the initially selected first vertex based on the resolution of the input graph to obtain the first vertex;

[0024] The second normalization subunit is used to normalize the initially selected second vertex based on the resolution of the input graph to obtain the second vertex.

[0025] In some embodiments, the initial selection of vertices to obtain sub-units includes:

[0026] The parameter acquisition sub-unit is used to acquire the initial first vertex of the minimum bounding rectangle, the width value of the minimum bounding rectangle, and the height value, wherein the initial first vertex is any vertex of the minimum bounding rectangle;

[0027] The coordinate value sub-unit is used to calculate the coordinate values ​​of the vertex located diagonally opposite the initially selected first vertex based on the coordinate values ​​of the initially selected first vertex, the width value, and the height value, so as to obtain the initially selected second vertex.

[0028] In some embodiments, the location update unit includes:

[0029] The scaling factor acquisition sub-unit is used to obtain the scaling factor of the source material;

[0030] The update coordinate calculation subunit is used to calculate the updated position coordinates of the pixel based on the material scaling factor, the coordinates of the first vertex, the coordinates of the second vertex, the coordinates of the center point, and the coordinates of the initial position of the pixel.

[0031] In some embodiments, updating the coordinate calculation subunit includes:

[0032] The first difference sub-unit is used to calculate the first difference between the coordinates of the initial position of the pixel and the coordinates of the first vertex.

[0033] The second difference sub-unit is used to calculate the second difference between the coordinate values ​​of the second vertex and the coordinate values ​​of the first vertex;

[0034] The ratio sub-unit is used to calculate the ratio of the first difference to the second difference;

[0035] The third difference sub-unit is used to calculate the third difference between the ratio and the coordinate value of the center point;

[0036] The product sub-unit is used to calculate the product of the third difference and the material scaling factor;

[0037] The summation sub-unit is used to calculate the sum of the product and the coordinates of the center point, where the summation value is the coordinates of the updated position of the pixel.

[0038] In some embodiments, the source image includes foreground and background elements; correspondingly, the source image adding unit includes:

[0039] The special effects main image subunit is used to add a material image to the main image based on the first occlusion relationship between the foreground material and the main image, and the second occlusion relationship between the background material and the main image, to obtain a main image carrying the material image, denoted as the special effects main image;

[0040] The overlay processing subunit is used to overlay the main image of the special effect with the input image to obtain the processed result image.

[0041] In some embodiments, the special effects main image subunit includes:

[0042] The first combination sub-unit is used to combine the background material and the main image based on the second occlusion relationship to obtain a first combination result, wherein the second occlusion relationship is: for pixels in the main image and the background material that are in the same position, the pixels in the main image are displayed;

[0043] The second combination sub-unit is used to combine the foreground material and the first combination result based on the first occlusion relationship to obtain the main image of the special effects, wherein the first occlusion relationship is: for pixels in the foreground material and the first combination result that are in the same position, the pixels in the foreground material are displayed.

[0044] In some embodiments, the overlay processing subunit is specifically used to cover the main image in the input image with the special effects main image to obtain the processing result image.

[0045] In the image processing method provided in this application embodiment, the main image in the input image can be obtained first, and then the minimum bounding rectangle of the main image in the input image can be obtained. Subsequently, a pair of vertices located on the diagonal of the minimum bounding rectangle are obtained: the first vertex and the second vertex, and the center point of the minimum bounding rectangle is determined based on the first vertex and the second vertex. For each pixel of the source image, the initial position of the pixel can be scaled based on the aforementioned first vertex, second vertex, and center point to obtain the updated position of the pixel. The initial position of the pixel is: the position corresponding to the pixel when the source image is tiled on the input image. After obtaining the updated position of each pixel, the source image can be added to the input image based on the updated positions of all pixels of the source image.

[0046] In this application, since the update position of each pixel in the source image is determined by scaling operations based on the first vertex, the second vertex, and the center point, and the first vertex, the second vertex, and the center point are all generated based on the main image in the input image, the source image will be displayed around the main image after scaling operations, thus producing a better contrasting effect on the main image in the input image, improving the problem of monotonous combined image effects in the prior art, and helping to increase user stickiness. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1a This is a schematic diagram of a scene illustrating the image processing method provided in an embodiment of this application;

[0049] Figure 1b This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0050] Figure 1c (1) shows a schematic diagram of an input diagram provided in a specific embodiment;

[0051] Figure 1c (2) shows from Figure 1c (1) A schematic diagram of extracting the main image;

[0052] Figure 1c (3) shows the acquisition Figure 1c A schematic diagram of the minimum bounding rectangle of the main image in (2) of the diagram;

[0053] Figure 1d (1) shows a schematic diagram of each pixel of the background material at its initial position;

[0054] Figure 1d (2) shows a schematic diagram of the updated position of each pixel in the background material;

[0055] Figure 1d (3) shows a schematic diagram of the first combination result of the background material and the main image;

[0056] Figure 1e (1) shows a schematic diagram of each pixel in the foreground material at its initial position;

[0057] Figure 1e (2) shows a schematic diagram of the updated position of each pixel in the foreground material;

[0058] Figure 1e (3) shows a schematic diagram of combining the foreground material with the first combination result to form the main image of the special effects;

[0059] Figure 1f A schematic diagram of the processing results is shown;

[0060] Figure 2 This is a schematic flowchart of an image processing method provided in another embodiment of this application;

[0061] Figure 3 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application;

[0062] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0063] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0064] This application provides an image processing method, apparatus, electronic device, and storage medium.

[0065] Specifically, the image processing device can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, or personal computer (PC), etc. In some embodiments, the terminal can be deployed in a vehicle. The server can be a single server or a server cluster consisting of multiple servers.

[0066] In some embodiments, the image processing apparatus may also be integrated into multiple electronic devices, such as multiple servers, with the image processing method of this application being implemented by the multiple servers.

[0067] In some embodiments, the server may also be implemented as a terminal.

[0068] For example, refer to Figure 1a The aforementioned electronic device can perform the following method: acquiring the main image in the input image; acquiring the minimum bounding rectangle of the main image in the input image; acquiring the first vertex and the second vertex of the minimum bounding rectangle, and determining the center point of the minimum bounding rectangle based on the first vertex and the second vertex, wherein the first vertex and the second vertex are a pair of vertices located on the diagonal of the minimum bounding rectangle; for each pixel of the source image, scaling the initial position of the pixel based on the first vertex, the second vertex, and the center point to obtain the updated position of the pixel; wherein the initial position is the position of the pixel when the source image is tiled on the input image; and adding the source image to the input image based on the updated position of each pixel of the source image.

[0069] In the above embodiments, since the first vertex, the second vertex, and the center point are all generated based on the main image in the input image, the source image, after being scaled based on the first vertex, the second vertex, and the center point, will be displayed around the main image, thus producing a better contrasting effect on the main image in the input image.

[0070] The following sections provide detailed descriptions of each example. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0071] In this embodiment, an image processing method is provided, such as... Figure 1b As shown, this image processing method is applied to a terminal, and the specific process of this method may include the following steps 110 to 150:

[0072] 110. Obtain the main image from the input image.

[0073] The input image is the picture to be processed. The main image is the image of the object that is highlighted in the input image; the main image can be an image of a person or a scene. It should be understood that the specific object type of the main image should not be construed as a limitation of this application.

[0074] The main image can be extracted from the input image using an artificial intelligence (AI) model, or it can be manually extracted by staff using drawing software. It should be understood that the specific steps taken to obtain the main image from the input image should not be construed as a limitation of this application. For details, please see [link to relevant documentation]. Figure 1c , Figure 1c (2) in the middle is from Figure 1c (1) The main image extracted from the image.

[0075] The AI ​​model described above can be a Convolutional Neural Network (CNN) model, or other types of models, such as a Recurrent Neural Network (RNN) model or a Long Short-Term Memory (LSTM) model. It should be understood that the specific type of AI model should not be construed as a limitation of this application.

[0076] 120. Obtain the smallest bounding rectangle of the main image in the input image.

[0077] The minimum bounding rectangle is the smallest rectangle that contains the main image and whose border is parallel to the coordinate axes. See details in [link to relevant documentation]. Figure 1c (3) The coordinate axes include the x-axis and y-axis. For details, please refer to [link to relevant documentation].Figure 1c In (1), let the horizontal direction of the input graph be the x-axis, and the direction from left to right in the horizontal direction be the positive direction of the x-axis; let the vertical direction of the input graph be the y-axis, and the direction from top to bottom in the vertical direction be the y-axis.

[0078] Optionally, in one specific implementation, the minimum bounding rectangle of the main image can be calculated through the following steps 121 to 123:

[0079] 121. Obtain the outline of the main image.

[0080] 122. Based on the outline, determine the initial minimum bounding rectangle of the main image.

[0081] In the above implementation, the initial minimum bounding rectangle of the main image can be determined based on the outline of the main image. Specifically, the following can be obtained: the unique intersection point of the main image's outline with the top border of the bounding rectangle in the "upward" direction, which is the top border; the unique intersection point of the main image's outline with the bottom border of the bounding rectangle in the "downward" direction, which is the bottom border; the unique intersection point of the main image's outline with the left border of the bounding rectangle in the "leftward" direction, which is the left border; and the unique intersection point of the main image's outline with the right border of the bounding rectangle in the "rightward" direction, which is the right border. Then, the initial minimum bounding rectangle is formed using the obtained top border, bottom border, left border, and right border.

[0082] Let's take the top border as an example to illustrate how to obtain the border: the top border of the initial smallest bounding rectangle can be determined by moving the top border of the outer rectangle up and down until the top border and the outline of the main image have only one intersection point in the "upward" direction.

[0083] Optionally, in another specific embodiment, the initial minimum bounding rectangle of the main image can also be obtained in other ways. For example, the coordinate values ​​of each pixel in the main image can be obtained, and then the pixels with the minimum x-coordinate, the maximum x-coordinate, the minimum y-coordinate, and the maximum y-coordinate can be obtained respectively. For the pixels with the minimum x-coordinate and the pixels with the maximum x-coordinate, straight lines parallel to the y-axis are drawn to obtain the left and right borders; for the pixels with the minimum and the pixels with the maximum y-coordinate, straight lines parallel to the x-axis are drawn to obtain the top and bottom borders. Then, the initial minimum bounding rectangle is formed using the top, bottom, left, and right borders obtained above. It should be understood that the specific method of determining the minimum bounding rectangle of the main image should not be construed as a limitation of this application.

[0084] 123. In response to the movement operation of the border of the initially selected minimum bounding rectangle, the border of the initially selected minimum bounding rectangle is moved accordingly to obtain the final selected minimum bounding rectangle, wherein the final selected minimum bounding rectangle is the minimum bounding rectangle of the main image in the input image.

[0085] Optionally, in one implementation, the border movement processing operation can be as follows: keeping the center point of the minimum bounding rectangle unchanged, when the user pulls outward or inward on one side of the border, the other side of the border can be pulled outward or inward accordingly. For example, suppose the user moves the left border to the left by m pixels, then the right border moves to the right by m pixels to keep the center point of the minimum bounding rectangle unchanged. If the user moves the top border downward by n pixels, then the bottom border moves upward by n pixels to keep the center point of the minimum bounding rectangle unchanged.

[0086] Alternatively, in another implementation, the movement processing operation for the borders can be as follows: the user can adjust the movement direction of the four borders according to their own wishes. It should be understood that the specific movement operation for the borders of the initially selected smallest bounding rectangle should not be construed as a limitation of this application.

[0087] In one specific implementation, the manual adjustment step 123 can be omitted, and the initially selected minimum bounding rectangle can be directly used as the final minimum bounding rectangle.

[0088] In the above implementation, since the main image is usually irregular, the initial minimum bounding rectangle determined by the electronic device may not completely encompass the main image. Therefore, after obtaining the initial minimum bounding rectangle, the border can be fine-tuned in response to the user's movement and adjustment operation on the border of the initial minimum bounding rectangle; thereby obtaining the final minimum bounding rectangle, achieving more accurate containment of the main image.

[0089] 130. Obtain the first vertex and the second vertex of the minimum bounding rectangle, and determine the center point of the minimum bounding rectangle based on the first vertex and the second vertex.

[0090] The first vertex and the second vertex are a pair of vertices located on the diagonal of the minimum bounding rectangle. The minimum bounding rectangle has four vertices, which can be divided into two pairs based on their location on the diagonal: the top-left vertex and the bottom-right vertex, and the bottom-left vertex and the top-right vertex.

[0091] The first vertex and the second vertex can be either of the two pairs of vertices mentioned above. For ease of description, let's take the example where the first vertex is the top left vertex and the second vertex is the bottom right vertex mentioned above.

[0092] After determining the first and second vertices, the average of the x-coordinates of the first and second vertices can be calculated to determine the x-coordinate of the center point; and the average of the y-coordinates of the first and second vertices can be calculated to determine the y-coordinate of the center point.

[0093] Optionally, in one specific embodiment, obtaining the first vertex and the second vertex of the minimum bounding rectangle may specifically include the following steps 131 to 133:

[0094] 131. Obtain the first and second vertices of the minimum bounding rectangle.

[0095] The first vertex can be any one of the four vertices of the smallest bounding rectangle, and the second vertex is the vertex located diagonally opposite the first vertex. Optionally, in one specific embodiment, step 131 may specifically include the following steps 1311 to 1312:

[0096] 1311. Obtain the initial first vertex of the minimum bounding rectangle, the width value of the minimum bounding rectangle, and the height value of the minimum bounding rectangle.

[0097] The first vertex selected initially is any vertex of the minimum bounding rectangle.

[0098] 1312. Based on the coordinates of the first initially selected vertex, the width value, and the height value, calculate the coordinates of the vertex located diagonally opposite the first initially selected vertex to obtain the second initially selected vertex.

[0099] Given the coordinates of the first initially selected vertex, the width of the smallest bounding rectangle, and its height, the coordinates of the second initially selected vertex can be obtained by adding or subtracting the width value from the x-coordinate of the first initially selected vertex, and by adding or subtracting the height value from the y-coordinate of the first initially selected vertex. The addition or subtraction operation depends on the position of the first initially selected vertex and the positive direction of the coordinate axes.

[0100] For ease of description, continuing with the example above: Let's assume the first vertex is the top-left corner vertex of the smallest bounding rectangle, and the positive direction of the coordinate axes is as follows... Figure 1c Let's continue the explanation using the direction shown as an example. Let's assume that the coordinates of the first vertex are (x0, y0), the width is z, and the height is w. Then the coordinates of the second vertex are (x0+z, y0+w).

[0101] In the above implementation, the relative positional relationship between the two vertices can be obtained by calculating the coordinate values ​​of the first and second vertices selected initially, thus providing an intuitive and accurate description of the relative positional relationship between the two vertices.

[0102] 132. Based on the resolution of the input graph, the initially selected first vertex is normalized to obtain the first vertex.

[0103] For ease of description, let's assume the resolution of the input image is p×q, meaning the main image has p columns of pixels, with q pixels in each column.

[0104] Then, the initially selected first vertex can be normalized to obtain its coordinates:

[0105]

[0106] 133. Based on the resolution of the input graph, the initially selected second vertex is normalized to obtain the second vertex.

[0107] Continuing with the example above, we can normalize the initially selected second vertex to obtain its coordinates:

[0108] In the above implementation, normalization can unify the computational dimensions of vertex coordinates, which facilitates subsequent processing.

[0109] 140. For each pixel in the source image, based on the first vertex, the second vertex, and the center point, the initial position of the pixel is scaled to obtain the updated position of the pixel.

[0110] The initial position is the position of the aforementioned pixel when the source image is tiled onto the input image.

[0111] Optionally, in one specific embodiment, step 140 may specifically include the following steps 141 to 142:

[0112] 141. Obtain the scaling factor of the source material.

[0113] The scaling factor of the source material can be set by the user according to their own needs. Let's denote the scaling factor as "scale". In one specific implementation, scale = adjustable_size / 100, where adjustable_size can be any integer between 0 and 100, which can be adjusted by the user as needed. It should be understood that the scaling factor can also be obtained in other ways, such as the user randomly setting a value between [0,1]; the specific method of obtaining the scaling factor should not be construed as a limitation of this application.

[0114] 142. Based on the scaling factor of the material, the coordinates of the first vertex, the coordinates of the second vertex, the coordinates of the center point, and the coordinates of the initial position of the pixel, calculate the coordinates of the updated position of the pixel.

[0115] Optionally, in one specific embodiment, step 142 may specifically include the following steps 1421 to 1426:

[0116] 1421. Calculate the first difference between the initial position coordinates of the pixel and the coordinates of the first vertex.

[0117] Let's assume the initial coordinates of the pixel are uv and the coordinates of the first vertex are newPos.xy. Then the first difference is: uv - newPos.xy.

[0118] 1422. Calculate the second difference between the coordinates of the second vertex and the coordinates of the first vertex.

[0119] Let's assume the coordinates of the second vertex are newPos.zw. Continuing with the example above, the second difference is: newPos.zw – newPos.xy.

[0120] 1423. Calculate the ratio of the first difference to the second difference.

[0121] Continuing with the example above, the ratio is as follows:

[0122] 1424. Calculate the third difference between the ratio and the coordinates of the center point.

[0123] Let's assume the coordinates of the center point are newCenter. Continuing with the example above, the third difference mentioned above is:

[0124] 1425. Calculate the product of the third difference and the material scaling factor.

[0125] Continuing with the example above, the product is:

[0126]

[0127] 1426. Calculate the sum of the product and the coordinates of the center point, where the sum is the coordinates of the updated position of the pixel.

[0128] Continuing with the example above, the sum of the above values ​​is: The sum of the above values ​​is the coordinate value of the updated position new.uv of the pixel.

[0129] In the above embodiments, the initial position coordinates uv of any pixel in the source image can be scaled based on the source image scaling factor, the coordinates of the first vertex, the coordinates of the second vertex, and the coordinates of the center point to obtain the updated position coordinates new.uv of that pixel. Since the above scaling process is based on the coordinates of the first vertex, the second vertex, and the center point, and all three points are related to the main image, the above process can make any pixel in the source image appear near the main image.

[0130] 150. Based on the updated position of each pixel in the source image, add the source image to the input image.

[0131] Performing the scaling process from steps 1421 to 1426 on all pixels of the source image yields the updated position of each pixel. Adding the source image to the input image based on the updated position of each pixel allows the source image to be displayed around the main image, thus better complementing the main image of the input image and increasing the interest of the image processing process.

[0132] Optionally, in one specific embodiment, the source image includes foreground material and background material; accordingly, step 150 may include the following steps 151 to 152:

[0133] 151. Based on the first occlusion relationship between the foreground material and the main image, and the second occlusion relationship between the background material and the main image, a material image is added to the main image to obtain a main image carrying the material image, denoted as the special effects main image.

[0134] The first occlusion relationship is: if the foreground material overlaps with the main image, then the foreground material occludes the main image; the second occlusion relationship is: if the main image overlaps with the background material, then the main image occludes the background material.

[0135] Optionally, in one specific embodiment, step 151 may specifically include the following steps 1511 to 1512:

[0136] 1511. Based on the second occlusion relationship, the background material and the main image are combined to obtain a first combination result, wherein the second occlusion relationship is: for pixels in the main image and the background material that are in the same position, the pixels of the main image are displayed.

[0137] Let bgColor represent the texture or color of the background image, srcColor represent the texture or color of the main image, and the blending factor be segmentColor.r. The blending factor segmentColor.r takes a value of 1 when the background image and the main image overlap, and a value of 0 when they do not overlap.

[0138] The display rules for textures or colors are as follows:

[0139] Value1=[srcColor*(segmentColor.r)]+[bgColor*(1-segmentColor.r)]

[0140] Based on the above display rules, we obtain the first combined result, which we can denote as renderTexture1. For details, please refer to [link to relevant documentation]. Figure 1d Let's take a background element made of LED strips as an example. Figure 1d (1) is a schematic diagram of each pixel of the background material at its initial position; Figure 1d (2) is a schematic diagram of the updated position of each pixel in the background material; Figure 1d (3) is a schematic diagram of the first combination result of the background material and the main image.

[0141] 1512. Based on the first occlusion relationship, the foreground material and the first combination result are combined to obtain a special effects main image, wherein the first occlusion relationship is: for pixels in the foreground material and the first combination result that are in the same position, the pixels in the foreground material are displayed.

[0142] Let fgColor represent the texture or color of the foreground material, and let lassPass1 represent the texture or color of the first combined result renderTexture1. The blending factor is fgColor.a. The blending factor fgColor.a takes a value of 1 when the foreground material overlaps with the first combined result, and a value of 0 when the foreground material does not overlap with the first combined result.

[0143] The display rules for textures or colors are as follows:

[0144] value 2=[fgColor*(fgColor.a)]+[lassPass1*(1-fgColor.a)]

[0145] Please see details Figure 1e Continuing with the example above, Figure 1e (1) is a schematic diagram of each pixel in the foreground material at its initial position; Figure 1e(2) is a schematic diagram of the updated position of each pixel in the foreground material; Figure 1e (3) is a schematic diagram of the combination of the foreground material and the first combination result into the main image of the special effect.

[0146] In the above implementation, a sense of depth can be added to the source image based on the first occlusion relationship and the second occlusion relationship, thereby more vividly showing the surrounding effect of the source image on the main image and increasing its interest.

[0147] 152. Overlay the main image of the special effect with the input image to obtain the processed result image.

[0148] Optionally, in one specific embodiment, step 152 may specifically include: covering the main image in the input image with the special effects main image to obtain the processed result image. For details, please refer to [link to relevant documentation]. Figure 1f , Figure 1f A schematic diagram of the processing results is shown.

[0149] In the above implementation, in the scene of the input image, having the main image with special effects cover the original main image without special effects can increase the depth of field of the overall image, making the image more layered and vivid.

[0150] In the image processing method provided in this application embodiment, the main image in the input image can be obtained first, and then the minimum bounding rectangle of the main image in the input image can be obtained. Subsequently, a pair of vertices on the diagonal of the minimum bounding rectangle are obtained: the first vertex and the second vertex, and the center point of the minimum bounding rectangle is determined based on the first vertex and the second vertex. For each pixel of the source image, the initial position of the pixel can be scaled based on the first vertex, the second vertex, and the center point to obtain the updated position of the pixel. The initial position of the pixel is the position corresponding to the pixel when the source image is tiled on the input image. After obtaining the updated position of each pixel, the source image can be added to the input image based on the updated positions of all pixels of the source image. Since the updated position of each pixel of the source image is determined by scaling operations based on the first vertex, the second vertex, and the center point, and the first vertex, the second vertex, and the center point are all generated based on the main image in the input image, the source image, after scaling, will be displayed around the main image, thus producing a better contrasting effect on the main image in the input image.

[0151] This application can improve user stickiness in image processing applications.

[0152] The method described in the above embodiments will be further described in detail below.

[0153] In this embodiment, the method of this application embodiment will be described in detail, taking a human portrait as the main image.

[0154] like Figure 2 As shown, the specific process of an image processing method is as follows:

[0155] 201. Obtain the portrait from the input image.

[0156] 202. Obtain the initial minimum bounding rectangle of the human image in the input image.

[0157] 203. Obtain the top-left and bottom-right vertices of the initially selected smallest bounding rectangle.

[0158] 204. Based on the resolution of the input graph, normalize the top-left and bottom-right vertices respectively to obtain the normalized top-left and bottom-right vertices.

[0159] 205. Based on the normalized top-left vertex and the normalized bottom-right vertex, calculate the center point of the initially selected minimum bounding rectangle.

[0160] 206. In response to the movement operation of the border of the initially selected minimum bounding rectangle, the border of the initially selected minimum bounding rectangle is moved while keeping the center point fixed to obtain the final selected minimum bounding rectangle, and the updated final selected top left corner vertex and final selected bottom right corner vertex are obtained.

[0161] 207. Obtain the scaling factor of the source material.

[0162] 208. For each pixel in the source image, calculate the coordinates of the updated position of the pixel based on the source scaling factor, the coordinates of the final selected top-left vertex, the coordinates of the final selected bottom-right vertex, the coordinates of the center point, and the coordinates of the initial position of the pixel.

[0163] 209. Based on the updated position of each pixel in the source image, add the source image to the input image.

[0164] Optionally, the source images include foreground and background images.

[0165] Step 209 above includes:

[0166] 2091. Based on the second occlusion relationship, the background material and the main image are combined to obtain a first combination result, wherein the second occlusion relationship is: for pixels in the main image and the background material that are in the same position, the pixels of the main image are displayed.

[0167] 2092. Based on the first occlusion relationship, the foreground material and the first combination result are combined to obtain the main image of the special effects, wherein the first occlusion relationship is: for pixels in the foreground material and the first combination result that are in the same position, the pixels of the foreground material are displayed.

[0168] 2093. Overlay the main image in the input image with the special effects image to obtain the processed result image.

[0169] The specific execution process of steps 201 to 209 has been explained in detail above, and will not be repeated here.

[0170] In the image processing method provided in this application embodiment, the main image in the input image can be obtained first, and then the minimum bounding rectangle of the main image in the input image can be obtained. Subsequently, a pair of vertices on the diagonal of the minimum bounding rectangle are obtained: the first vertex and the second vertex, and the center point of the minimum bounding rectangle is determined based on the first vertex and the second vertex. For each pixel of the source image, the initial position of the pixel can be scaled based on the first vertex, the second vertex, and the center point to obtain the updated position of the pixel. The initial position of the pixel is the position corresponding to the pixel when the source image is tiled on the input image. After obtaining the updated position of each pixel, the source image can be added to the input image based on the updated positions of all pixels of the source image. Since the updated position of each pixel of the source image is determined by scaling operations based on the first vertex, the second vertex, and the center point, and the first vertex, the second vertex, and the center point are all generated based on the main image in the input image, the source image, after scaling, will be displayed around the main image, thus producing a better contrasting effect on the main image in the input image.

[0171] This application can improve user stickiness in image processing applications.

[0172] To better implement the above methods, this application also provides an image processing apparatus, which can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, or personal computer (PC); the server can be a single server or a server cluster composed of multiple servers. For example, in this embodiment, the method of this application embodiment will be described in detail using the example of the image processing apparatus being specifically integrated into a terminal.

[0173] For example, such as Figure 3 As shown, the image processing apparatus may include:

[0174] The main image acquisition unit 301 is used to acquire the main image in the input image;

[0175] The bounding rectangle acquisition unit 302 is used to acquire the smallest bounding rectangle of the main image in the input image;

[0176] The center point determination unit 303 is used to obtain the first vertex and the second vertex of the minimum bounding rectangle, and determine the center point of the minimum bounding rectangle based on the first vertex and the second vertex, wherein the first vertex and the second vertex are a pair of vertices located on the diagonal of the minimum bounding rectangle;

[0177] The position update unit 304 is used to scale the initial position of each pixel in the source image based on the first vertex, the second vertex, and the center point to obtain the updated position of the pixel; wherein, the initial position is the position of the pixel when the source image is tiled on the input image;

[0178] The material adding unit 305 is used to add the material image to the input image based on the updated position of each pixel of the material image.

[0179] In some embodiments, the circumscribed rectangle acquisition unit 302 includes:

[0180] The contour acquisition subunit is used to acquire the contour of the main image;

[0181] The initial selection determines the sub-unit, which is used to determine the initial minimum bounding rectangle of the main image based on the contour;

[0182] The final selection determination subunit is used to perform corresponding movement processing on the border of the initially selected minimum bounding rectangle in response to the movement operation of the border of the initially selected minimum bounding rectangle, to obtain the final selected minimum bounding rectangle, wherein the final selected minimum bounding rectangle is the minimum bounding rectangle of the main image in the input image.

[0183] In some embodiments, the center point determination unit 303 includes:

[0184] The initial vertex acquisition sub-unit is used to acquire the initial first vertex and the initial second vertex of the minimum bounding rectangle;

[0185] The first normalization subunit is used to normalize the initially selected first vertex based on the resolution of the input graph to obtain the first vertex;

[0186] The second normalization subunit is used to normalize the initially selected second vertex based on the resolution of the input graph to obtain the second vertex.

[0187] In some embodiments, the initial selection of vertices to obtain sub-units includes:

[0188] The parameter acquisition sub-unit is used to acquire the initial first vertex of the minimum bounding rectangle, the width value of the minimum bounding rectangle, and the height value, wherein the initial first vertex is any vertex of the minimum bounding rectangle;

[0189] The coordinate value sub-unit is used to calculate the coordinate values ​​of the vertex located diagonally opposite the initially selected first vertex based on the coordinate values ​​of the initially selected first vertex, the width value, and the height value, so as to obtain the initially selected second vertex.

[0190] In some embodiments, the location update unit 304 includes:

[0191] The scaling factor acquisition sub-unit is used to obtain the scaling factor of the source material;

[0192] The update coordinate calculation subunit is used to calculate the updated position coordinates of the pixel based on the material scaling factor, the coordinates of the first vertex, the coordinates of the second vertex, the coordinates of the center point, and the coordinates of the initial position of the pixel.

[0193] In some embodiments, updating the coordinate calculation subunit includes:

[0194] The first difference sub-unit is used to calculate the first difference between the coordinates of the initial position of the pixel and the coordinates of the first vertex.

[0195] The second difference sub-unit is used to calculate the second difference between the coordinate values ​​of the second vertex and the coordinate values ​​of the first vertex;

[0196] The ratio sub-unit is used to calculate the ratio of the first difference to the second difference;

[0197] The third difference sub-unit is used to calculate the third difference between the ratio and the coordinate value of the center point;

[0198] The product sub-unit is used to calculate the product of the third difference and the material scaling factor;

[0199] The summation sub-unit is used to calculate the sum of the product and the coordinates of the center point, where the summation value is the coordinates of the updated position of the pixel.

[0200] In some embodiments, the source image includes foreground and background elements; correspondingly, the source image adding unit 305 includes:

[0201] The special effects main image subunit is used to add a material image to the main image based on the first occlusion relationship between the foreground material and the main image, and the second occlusion relationship between the background material and the main image, to obtain a main image carrying the material image, denoted as the special effects main image;

[0202] The overlay processing subunit is used to overlay the main image of the special effect with the input image to obtain the processed result image.

[0203] In some embodiments, the special effects main image subunit includes:

[0204] The first combination sub-unit is used to combine the background material and the main image based on the second occlusion relationship to obtain a first combination result, wherein the second occlusion relationship is: for pixels in the main image and the background material that are in the same position, the pixels in the main image are displayed;

[0205] The second combination sub-unit is used to combine the foreground material and the first combination result based on the first occlusion relationship to obtain the main image of the special effects, wherein the first occlusion relationship is: for pixels in the foreground material and the first combination result that are in the same position, the pixels in the foreground material are displayed.

[0206] In some embodiments, the overlay processing subunit is specifically used to cover the main image in the input image with the special effects main image to obtain the processing result image.

[0207] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.

[0208] In the image processing method provided in this application embodiment, the main image in the input image can be obtained first, and then the minimum bounding rectangle of the main image in the input image can be obtained. Subsequently, a pair of vertices on the diagonal of the minimum bounding rectangle are obtained: the first vertex and the second vertex, and the center point of the minimum bounding rectangle is determined based on the first vertex and the second vertex. For each pixel of the source image, the initial position of the pixel can be scaled based on the first vertex, the second vertex, and the center point to obtain the updated position of the pixel. The initial position of the pixel is the position corresponding to the pixel when the source image is tiled on the input image. After obtaining the updated position of each pixel, the source image can be added to the input image based on the updated positions of all pixels of the source image. Since the updated position of each pixel of the source image is determined by scaling operations based on the first vertex, the second vertex, and the center point, and the first vertex, the second vertex, and the center point are all generated based on the main image in the input image, the source image, after scaling, will be displayed around the main image, thus producing a better contrasting effect on the main image in the input image.

[0209] This application can improve user stickiness in image processing applications.

[0210] This application also provides an electronic device, which can be a terminal, a server, or other similar device. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers, etc.

[0211] In some embodiments, the image processing apparatus may also be integrated into multiple electronic devices, such as multiple servers, with the image processing method of this application being implemented by the multiple servers.

[0212] In this embodiment, the electronic device will be used as an example for detailed description, such as... Figure 4 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically:

[0213] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, an input module 404, and a communication module 405. Those skilled in the art will understand that... Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0214] The processor 401 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. In some embodiments, the processor 401 may include one or more processing cores; in some embodiments, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 401.

[0215] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0216] The electronic device also includes a power supply 403 that supplies power to the various components. In some embodiments, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0217] The electronic device may also include an input module 404, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0218] The electronic device may also include a communication module 405. In some embodiments, the communication module 405 may include a wireless module, through which the electronic device can perform short-range wireless transmission, thereby providing users with wireless broadband internet access. For example, the communication module 405 can be used to help users send and receive emails, browse web pages, and access streaming media.

[0219] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402 to realize various functions, as follows:

[0220] Obtain the main image from the input image; obtain the minimum bounding rectangle of the main image in the input image; obtain the first vertex and the second vertex of the minimum bounding rectangle, and determine the center point of the minimum bounding rectangle based on the first vertex and the second vertex, wherein the first vertex and the second vertex are a pair of vertices located on the diagonal of the minimum bounding rectangle; for each pixel of the source image, scale the initial position of the pixel based on the first vertex, the second vertex and the center point to obtain the updated position of the pixel; wherein the initial position is the position of the pixel when the source image is tiled on the input image; add the source image to the input image based on the updated position of each pixel of the source image.

[0221] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0222] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0223] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the image processing methods provided in embodiments of this application. For example, the instructions can execute the following steps:

[0224] Obtain the main image from the input image; obtain the minimum bounding rectangle of the main image in the input image; obtain the first vertex and the second vertex of the minimum bounding rectangle, and determine the center point of the minimum bounding rectangle based on the first vertex and the second vertex, wherein the first vertex and the second vertex are a pair of vertices located on the diagonal of the minimum bounding rectangle; for each pixel of the source image, scale the initial position of the pixel based on the first vertex, the second vertex and the center point to obtain the updated position of the pixel; wherein the initial position is the position of the pixel when the source image is tiled on the input image; add the source image to the input image based on the updated position of each pixel of the source image.

[0225] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0226] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments.

[0227] Since the instructions stored in the storage medium can execute the steps of any of the image processing methods provided in the embodiments of this application, the beneficial effects that any of the image processing methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0228] The foregoing has provided a detailed description of an image processing method, apparatus, electronic device, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An image processing method, characterized in that, The method includes: Obtain the main image from the input image; Obtain the smallest bounding rectangle of the main image in the input image; Obtain the first vertex and the second vertex of the minimum bounding rectangle, and determine the center point of the minimum bounding rectangle based on the first vertex and the second vertex, wherein the first vertex and the second vertex are a pair of vertices located on the diagonal of the minimum bounding rectangle; For each pixel in the source image, the initial position of the pixel is scaled based on the first vertex, the second vertex, and the center point to obtain the updated position of the pixel; wherein, the initial position is the position of the pixel when the source image is tiled on the input image; Based on the updated position of each pixel in the source image, the source image is added to the input image; For each pixel in the source image, the initial position of the pixel is scaled based on the first vertex, the second vertex, and the center point to obtain the updated position of the pixel, including: Get the scaling factor of the source material; Calculate a first difference between the initial position coordinates of the pixel and the coordinates of the first vertex; calculate a second difference between the coordinates of the second vertex and the coordinates of the first vertex; calculate the ratio of the first difference to the second difference; calculate a third difference between the ratio and the coordinates of the center point; calculate the product of the third difference and the scaling factor of the material; calculate the sum of the product and the coordinates of the center point, where the sum is the updated position coordinates of the pixel.

2. The method as described in claim 1, characterized in that, The step of obtaining the minimum bounding rectangle of the main image in the input image includes: Obtain the outline of the main image; Based on the outline, the initial minimum bounding rectangle of the main image is determined; In response to the movement operation of the border of the initially selected minimum bounding rectangle, the border of the initially selected minimum bounding rectangle is moved accordingly to obtain the final selected minimum bounding rectangle, wherein the final selected minimum bounding rectangle is the minimum bounding rectangle of the main image in the input image.

3. The method as described in claim 1, characterized in that, Obtaining the first and second vertices of the minimum bounding rectangle includes: Obtain the first and second vertices of the minimum bounding rectangle; Based on the resolution of the input graph, the initially selected first vertex is normalized to obtain the first vertex; Based on the resolution of the input graph, the initially selected second vertex is normalized to obtain the second vertex.

4. The method as described in claim 3, characterized in that, The process of obtaining the initial first vertex and the initial second vertex of the minimum bounding rectangle includes: Obtain the initial first vertex of the minimum bounding rectangle, the width value of the minimum bounding rectangle, and the height value, wherein the initial first vertex is any vertex of the minimum bounding rectangle; Based on the coordinates, width, and height of the initially selected first vertex, the coordinates of the vertex located diagonally opposite the initially selected first vertex are calculated to obtain the initially selected second vertex.

5. The method as described in claim 1, characterized in that, The source images include foreground and background images; The step of adding the source image to the input image based on the update position of each pixel of the source image includes: Based on the first occlusion relationship between the foreground material and the main image, and the second occlusion relationship between the background material and the main image, a material image is added to the main image to obtain a main image carrying the material image, which is denoted as the special effects main image; The main image of the special effect is overlaid with the input image to obtain the processed result image.

6. The method as described in claim 5, characterized in that, The step of adding a source image to the main image based on the first occlusion relationship between the foreground material and the main image, and the second occlusion relationship between the background material and the main image, to obtain a main image carrying the source image, includes: Based on the second occlusion relationship, the background material and the main image are combined to obtain a first combination result, wherein the second occlusion relationship is: for pixels in the main image and the background material that are in the same position, the pixels in the main image are displayed; Based on the first occlusion relationship, the foreground material and the first combination result are combined to obtain the main image of the special effects, wherein the first occlusion relationship is: for pixels in the foreground material and the first combination result that are in the same position, the pixels in the foreground material are displayed.

7. The method as described in claim 5, characterized in that, The process of overlaying the special effects subject image with the input image to obtain the processed result image includes: The main image in the input image is overlaid with the special effects main image to obtain the processed result image.

8. An image processing apparatus, characterized in that, The device includes: The main image acquisition unit is used to acquire the main image in the input image; A bounding rectangle acquisition unit is used to acquire the smallest bounding rectangle of the main image in the input image; The center point determination unit is used to obtain the first vertex and the second vertex of the minimum bounding rectangle, and determine the center point of the minimum bounding rectangle based on the first vertex and the second vertex, wherein the first vertex and the second vertex are a pair of vertices located on the diagonal of the minimum bounding rectangle; The position update unit is used to scale the initial position of each pixel in the source image based on the first vertex, the second vertex, and the center point to obtain the updated position of the pixel; wherein, the initial position is the position of the pixel when the source image is tiled on the input image; The material addition unit is used to add the material image to the input image based on the update position of each pixel of the material image; The scaling factor acquisition sub-unit is used to obtain the scaling factor of the source material; A first difference subunit is used to calculate a first difference between the coordinates of the initial position of the pixel and the coordinates of the first vertex; a second difference subunit is used to calculate a second difference between the coordinates of the second vertex and the coordinates of the first vertex; a ratio subunit is used to calculate the ratio between the first difference and the second difference; a third difference subunit is used to calculate a third difference between the ratio and the coordinates of the center point; a product subunit is used to calculate the product of the third difference and the scaling factor of the material; and a summation subunit is used to calculate the sum of the product and the coordinates of the center point, wherein the summation value is the coordinates of the updated position of the pixel.

9. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to perform the steps of the image processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the steps of the image processing method according to any one of claims 1 to 7.

Images