Object stroking method, device and equipment and storage medium

Abstract

The embodiment of the invention provides an object stroking method and device, equipment and a storage medium. The method comprises the following steps: according to depth information and normal information of each pixel point in a scene image, determining a depth deviation and a normal deviation of the pixel point under an associated offset; determining a target mask area corresponding to a specified object according to a spatial offset vector of each pixel point in the scene image facing a central point of the specified object in the scene image; and according to the depth deviation and the normal deviation of the target pixel point in the target mask area, determining corresponding stroking intensity information so as to generate a stroking effect of the specified object. According to the embodiment of the invention, personalized stroking of the specified object in any scene image can be realized, the flexibility of object stroking is improved, the fineness and naturalness of the object stroking effect are ensured, and the visual performance effect of the object after stroking is enhanced.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to a method, apparatus, device, and storage medium for outlining an object. Background Art

[0002] In the field of image rendering, object stroke can highlight the edge contour of any specified object during the image or video rendering process, making the specified object more clearly visible in the image or video and enhancing the visual effect of the specified object.

[0003] In related technologies, depth information and normal information are usually used to perform global stroke processing on any image based on the post-processing effect of screen space, but personalized stroke effects cannot be provided for specific objects in the image, which makes the existing image stroke method have certain limitations. Summary of the Invention

[0004] The embodiments of the present application provide a method, apparatus, device, and storage medium for object stroking, which enable personalized stroking of a specified object in any scene image and improve the flexibility of object stroking.

[0005] In a first aspect, an embodiment of the present application provides a method for outlining an object, the method comprising:

[0006] Determining the depth deviation and normal deviation of each pixel in the scene image under the associated offset according to the depth information and normal information of the pixel;

[0007] determining a target mask area corresponding to the specified object according to a spatial offset vector of each pixel point in the scene image facing a center point of the specified object in the scene image;

[0008] Corresponding stroke intensity information is determined according to the depth deviation and the normal deviation of the target pixel point in the target mask area to generate a stroke effect of the designated object.

[0009] In a second aspect, an embodiment of the present application provides a device for outlining an object, the device comprising:

[0010] A pixel deviation determination module is used to determine the depth deviation and normal deviation of each pixel under the associated offset based on the depth information and normal information of the pixel in the scene image;

[0011] a mask region determination module, configured to determine a target mask region corresponding to a specified object in the scene image based on a spatial offset vector of each pixel point in the scene image facing a center point of the specified object in the scene image;

[0012] The object stroke module is used to determine corresponding stroke intensity information according to the depth deviation and normal deviation of the target pixel point in the target mask area to generate the stroke effect of the specified object.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0014] A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the object tracing method provided in the first aspect of the present application.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program enables a computer to execute the object stroking method provided in the first aspect of the present application.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the object stroke method provided in the first aspect of the present application.

[0017] The technical solution provided in the embodiment of the present application determines the depth deviation and normal deviation of each pixel under the associated offset based on the depth information and normal information of each pixel in any scene image. And the target mask area corresponding to the specified object is determined based on the spatial offset vector of each pixel in the scene image facing the center point of the specified object in the scene image. Thus, according to the depth deviation and normal deviation of each target pixel in the target mask area, the corresponding stroke intensity information is determined to generate the stroke effect of the specified object, thereby realizing personalized stroke of the specified object in any scene image and improving the flexibility of object stroke. By comprehensively analyzing the depth deviation and normal deviation of each target pixel in the target mask area, the optimal stroke intensity information is set for the specified object to ensure the fineness and naturalness of the object stroke effect and enhance the visual performance of the object after stroke. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0019] Figure 1 A flowchart of a method for outlining an object provided in an embodiment of the present application;

[0020] Figure 2A flowchart of another method for object outlining provided by embodiments of the present application;

[0021] Figure 3 A principle block diagram of a device for object outlining provided by embodiments of the present application;

[0022] Figure 4 A schematic block diagram of an electronic device provided by embodiments of the present application. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0024] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or server including a series of steps or units does not necessarily have to include only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product, or device.

[0025] In view of the problem that the conventional image outlining method cannot provide personalized outlining effect for a specified object in an image and has certain limitations, embodiments of the present application design a new object outlining scheme. By analyzing the depth deviation and normal deviation of each pixel point in any scene image under the associated offset, and the target mask region corresponding to the specified object in the scene image, the depth deviation and normal deviation of each target pixel point in the target mask region are determined, and the optimal outlining intensity information of the specified object is comprehensively analyzed to provide personalized outlining effect for the specified object in the scene image, and the flexibility of object outlining is improved.

[0026] Figure 1A flowchart of a method for stroking an object provided in an embodiment of the present application. The method can be performed by an object stroking device provided in the present application. Among them, the object stroking device can be implemented by any software and / or hardware. Exemplarily, the object stroking device can be applied to any electronic device, which may include but is not limited to tablet computers, mobile phones (such as foldable screen mobile phones, large screen mobile phones, etc.), wearable devices, vehicle-mounted equipment, laptop computers, ultra-mobile personal computers (ultra-mobile personal computers, UMPCs), netbooks, personal digital assistants (personal digital assistants, PDAs), smart TVs, smart screens, high-definition TVs, 4K TVs, smart speakers, smart projectors and other types of computing devices. The present disclosure does not impose any restrictions on the specific types of electronic devices.

[0027] Specifically, such as Figure 1 As shown, the method may include the following steps:

[0028] S110 , determining a depth deviation and a normal deviation of each pixel under an associated offset according to the depth information and the normal information of the pixel in the scene image.

[0029] In this application, a scene image can be an image taken of the surrounding environment in any scene, and can be composed of multiple objects, environmental backgrounds, and action elements in any scene. Each pixel in the scene image refers to each individual pixel in the scene image. The depth information of each pixel refers to the distance from the scene point represented by the pixel to the camera, which can be a depth value. The normal information of each pixel refers to the normal direction of the surface where the scene point represented by the pixel is located, which can be a three-dimensional vector to represent the specific orientation of the surface where the scene point is located.

[0030] Considering that image stroke mainly adds corresponding lines to the edge contours in the scene image to make the edge contours in the scene image clearer and more visible, the edge contours in the scene image can be manifested as a sudden change in pixel depth (for example, there is a sudden change in depth at the junction between any object and the background), and a sudden change in the normal direction of the surface where the pixel is located (for example, there is a sudden change in the normal direction of multiple surfaces at the edge corners of a geometric object, while the depth change is not obvious).

[0031] Therefore, in order to ensure the accuracy of image stroking, this application first needs to combine the depth information and normal information of each pixel in the scene image to determine whether each pixel in the scene image is an edge point and needs to perform stroking processing.

[0032] It is understandable that in the field of image rendering, the geometric information of each pixel in any scene image is usually cached in a geometry buffer (G-buffer). The geometric information may include the color, depth, normal, etc. of the pixel to support delayed rendering of the scene image.

[0033] Therefore, for any scene image, the present application will first read the depth value and normal vector of each pixel in the scene image from the G-buffer, thereby obtaining the depth information and normal information of each pixel.

[0034] It should be understood that objects in the scene image conform to the perspective law of being larger when closer and smaller when farther away, which means that the smaller the depth value of a pixel in the scene image, the closer the scene point represented by the pixel is, and the larger the offset is to determine whether there is a depth mutation and a normal mutation in the pixel; and the larger the depth value of a pixel in the scene image, the farther the scene point represented by the pixel is, and then it is possible to determine whether there is a depth mutation and a normal mutation in the pixel under a smaller offset.

[0035] Therefore, in order to ensure the accuracy of edge detection in scene images, this application can set an associated offset for each pixel in the scene image according to its depth information, so as to accurately analyze whether there is a sudden change in depth and normal at each pixel. Among them, the smaller the depth value of any pixel, the larger the associated offset of the pixel. And the larger the depth value of any pixel, the smaller the associated offset of the pixel.

[0036] Furthermore, for each pixel in the scene image, the present application can determine the depth information and normal information of the pixel after it is offset according to the pixel's associated offset. Then, by analyzing the difference between the two depth values ​​before and after the offset and the angle between the two normal vectors of each pixel, the depth deviation and normal deviation of each pixel under the associated offset can be obtained.

[0037] It is understandable that, considering that the smaller the depth value of a pixel in a scene image, the closer the scene point represented by the pixel is, then if the scene point is stroked as an edge contour point, the required stroke width will be wider; and the larger the depth value of a pixel in a scene image, the farther the scene point represented by the pixel is, then if the scene point is stroked as an edge contour point, the required stroke width will be narrower, thereby ensuring the uniformity of the stroke effect at different distances in the scene image. It can be seen from this that the associated offset and stroke width of each pixel in the scene image have the same change pattern. Therefore, in order to ensure the accuracy of image stroking, the present application can use the associated offset of each pixel in the scene image as the stroke width of the pixel when stroking.

[0038] S120 , determining a target mask area corresponding to the designated object according to a spatial offset vector of each pixel point in the scene image facing the center point of the designated object in the scene image.

[0039] In this application, in order to achieve personalized stroking of any specified object in a scene image, it is first necessary to perform target detection on the specified object in the scene image, so as to frame the area where the specified object is located in the scene image. At this time, since the geometric shape of the specified object is uncertain, and the bounding box (i.e., the Bounding Box) of the specified object after target detection is a rectangular box, the bounding box of the specified object may include other elements in the scene image, so that when edge detection is performed on the bounding box image, other edge points other than the specified object will also be identified for stroking, and the accurate stroking of the specified object in the scene image cannot be guaranteed.

[0040] Considering that the distance from each boundary point to the center point of the specified object in three-dimensional space can be guaranteed to be within a certain spatial range, it can be known that scene points outside this spatial range will not be scene points on the specified object. Therefore, the present application can determine whether the scene point represented by each pixel in the scene image is a scene point on the specified object by analyzing whether the distance between the scene point represented by the pixel and the center point of the specified object is within the corresponding spatial range, thereby determining the target mask area corresponding to the specified object in the scene image, and more accurately marking the pixel range of the specified object in different shapes.

[0041] Therefore, for each pixel in the scene image, the present application can determine the spatial position information of the scene point represented by the pixel by analyzing the coordinate information of the pixel in the two-dimensional screen space and the depth value of the pixel. Moreover, by performing target detection on the specified object in the scene image and selecting the area where the specified object is located in the scene image, the present application can determine the center point of the bounding box of the specified object after target detection as the center point of the specified object. Then, by analyzing the coordinate information of the center point of the specified object in the two-dimensional screen space and the depth value of the center point, the spatial position information of the center point of the specified object can be determined.

[0042] Further, by analyzing the spatial variation between the spatial position information of the center point of the specified object and the spatial position information of the scene point corresponding to each pixel point in the scene image, the spatial offset vector of each pixel point in the scene image facing the center point of the specified object can be generated. Then, according to the vector values in the spatial offset vector of each pixel point in the scene image facing the center point of the specified object, the projection length of the spatial offset vector is calculated to represent the spatial distance between the scene point corresponding to each pixel point and the center point of the specified object. In this way, it is determined whether the projection length of the spatial offset vector of each pixel point is within the corresponding distance range set by the specified object, indicating whether the scene point corresponding to the pixel point is a scene point on the specified object. Thus, the target mask region corresponding to the specified object is determined in the scene image, and the pixel range of the specified object under different shapes is more accurately marked, so as to accurately limit the stroke range of the specified object.

[0043] In S130, according to the depth deviation and normal deviation of the target pixel point in the target mask region, the corresponding stroke intensity information is determined to generate the stroke effect of the specified object.

[0044] After determining the target mask region corresponding to the specified object in the scene image, the present application can determine each target pixel point in the target mask region in the scene image, and determine the depth deviation and normal deviation of each target pixel point. The scene point corresponding to the target pixel point can be a scene point on the specified object.

[0045] Then, in order to ensure the accurate stroke of the specified object in the scene image, the present application can analyze the specific difference size of the depth deviation and the normal deviation of each target pixel point to determine the possibility of the target pixel point belonging to the edge point of the specified object. Therefore, according to whether each target pixel point needs to be stroked, the present application can set a stroke transparency for representing the line depth of each target pixel point for stroke, so as to form different stroke effects.

[0046] It is understandable that if the difference in the depth deviation or normal deviation of a certain target pixel point is greater, it means that the depth mutation or normal mutation of the target pixel point is more obvious, and the target pixel point is more likely to be the edge contour point of the specified object and needs to be stroked. If the difference in the depth deviation or normal deviation of a certain target pixel point is smaller, it means that the depth mutation or normal mutation of the target pixel point is less obvious, and the target pixel point is more likely not to be the edge contour point of the specified object and does not need to be stroked. Therefore, the present application can set the stroke transparency of any target pixel point to vary inversely with the depth deviation and normal deviation of the target pixel point. Then, the greater the depth deviation and normal deviation of a certain target pixel point, the smaller the stroke transparency of the target pixel point, so that the stroke line of the target pixel point is darker, and the more obvious the stroke effect is at the target pixel point; and the smaller the depth deviation and normal deviation of a certain target pixel point, the greater the stroke transparency of the target pixel point, so that the stroke line of the target pixel point is lighter, and an inconspicuous stroke effect is presented at the target pixel point.

[0047] Therefore, for each target pixel, the present application can comprehensively analyze the depth deviation and normal deviation of the target pixel to determine the combined deviation of the target pixel, and thereby determine the stroke transparency of the target pixel according to the inverse proportional law between the stroke transparency and the depth deviation and the normal deviation, and combine the stroke width represented by the associated offset of the target pixel to determine the stroke intensity information of the target pixel.

[0048] In the same way as above, the stroke intensity information of each target pixel point can be determined, so as to more accurately stroke the specified object in the scene image and generate the stroke effect of the specified object, thereby realizing personalized stroke of the specified object in any scene image and improving the flexibility of object stroke.

[0049] In addition, in order to enhance the diversified stroke effects of object strokes, the present application can also optimize the stroke intensity information in the present application based on the dot texture and stroke color of the scene image to achieve an extensible stroke effect for a specified object in the scene image. In other words, the present application can use the preset dot density information to perform dot calculations at corresponding intervals in the horizontal and vertical directions of the two-dimensional screen space of the scene image to determine the dot texture of the scene image, which can form a dotted line stroke format. Moreover, the color of the stroke line is represented by setting the stroke color, such as a warning color, a technology color, etc.

[0050] Then, the dot texture of the scene image and the preset stroke color can be merged into the stroke intensity information corresponding to each target pixel in the target mask area to optimize the specific stroke effect of the specified object, thereby presenting a diversified stroke effect after stroking the specified object using dotted lines and stroke colors.

[0051] It should be noted that the present application can also add other stroke style parameters (such as flashing stroke) to the scene image to achieve personalized stroke effects of various visual styles for specified objects in the scene image.

[0052] The technical solution provided in the embodiment of the present application determines the depth deviation and normal deviation of each pixel under the associated offset based on the depth information and normal information of each pixel in any scene image. And the target mask area corresponding to the specified object is determined based on the spatial offset vector of each pixel in the scene image facing the center point of the specified object in the scene image. Thus, according to the depth deviation and normal deviation of each target pixel in the target mask area, the corresponding stroke intensity information is determined to generate the stroke effect of the specified object, thereby realizing personalized stroke of the specified object in any scene image and improving the flexibility of object stroke. By comprehensively analyzing the depth deviation and normal deviation of each target pixel in the target mask area, the optimal stroke intensity information is set for the specified object to ensure the fineness and naturalness of the object stroke effect and enhance the visual performance of the object after stroke.

[0053] As an optional implementation scheme in this application, in order to ensure the personalized stroking effect of a specified object in any scene image, this application can provide a detailed explanation of the specific process of determining the depth deviation and normal deviation of any pixel point in the scene image, the target mask area corresponding to the specified object, and the stroke intensity information of the target pixel point.

[0054] Figure 2 A flowchart of another method for outlining an object provided in an embodiment of the present application, the method may specifically include the following steps:

[0055] S210 , for each pixel in the scene image, determine an associated offset of the pixel according to depth information of the pixel.

[0056] For any scene image, objects in the scene image will conform to the perspective law of being larger when closer and smaller when farther away, which means that the smaller the depth value of a pixel in the scene image, the closer the scene point represented by the pixel is, and the larger the offset is to determine whether there is a depth mutation and a normal mutation in the pixel; and the larger the depth value of a pixel in the scene image, the farther the scene point represented by the pixel is, and then it is possible to determine whether there is a depth mutation and a normal mutation in the pixel under a smaller offset.

[0057] Therefore, in order to ensure accurate tracing in the scene image, the present application can set an associated offset for each pixel that matches the depth information of the pixel based on the inverse proportional change law between the depth information of each pixel in the scene image and the associated offset.

[0058] In some possible implementations, the present application can pre-set an ideal depth value (denoted as standardDepth) according to the scene type of any scene image, and set a suitable offset range [minOffset, maxOffset] for each pixel point in the scene image, so that the middle offset of the offset range [minOffset, maxOffset] (denoted as desiredOffsetAtStandard) can be used as the associated offset matched under the ideal depth value standardDepth.

[0059] Then, based on the inversely proportional variation between the depth information and the associated offset of each pixel in the scene image, a first ratio between the set ideal depth value standardDepth and the depth information of each pixel and a second ratio between the initial associated offset of each pixel and the intermediate offset desiredOffsetAtStandard that matches the ideal depth value standardDepth can be determined to be equal, thereby calculating the initial associated offset of each pixel. Furthermore, the final associated offset of each pixel is determined by determining whether the initial associated offset of each pixel is within the offset range [minOffset, maxOffset]. For example, if the initial associated offset of a pixel is within the offset range [minOffset, maxOffset], the initial associated offset is used as the final associated offset of the pixel. If the initial associated offset of a pixel is less than the minimum offset minOffset of the offset range [minOffset, maxOffset], the minimum offset minOffset is used as the final associated offset of the pixel. If the initial associated offset of a pixel point is greater than the maximum offset maxOffset of the offset range [minOffset, maxOffset], the maximum offset maxOffset will be used as the final associated offset of the pixel point.

[0060] For example, assuming that the depth information of each pixel in the scene image is recorded as planeDepth, the calculation formula of the association offset associationOffset of each pixel can be: associationOffset=clamp(desiredOffsetAtStandard*(standardDepth / planeDepth),minOffset,maxOffset).

[0061] S220: Determine the offset pixel of the pixel in multiple offset directions according to the associated offset of the pixel.

[0062] Considering that when a pixel point in a scene image is an edge contour point, depth mutation or normal mutation may occur in multiple different directions. Therefore, in order to ensure accurate edge detection of each pixel point in the scene image, this application first sets multiple offset directions, such as upward offset, left offset, downward offset, or right offset.

[0063] Therefore, after determining the associated offset of each pixel point in the scene image, for each pixel point, the present application can offset the pixel point in each offset direction according to the associated offset of the pixel point, and determine the offset pixel point of the pixel point in each offset direction.

[0064] In the same manner as above, multiple offset pixel points of each pixel point in each offset direction can be determined.

[0065] S230, determining the depth deviation of the pixel point based on the depth information of the pixel point and the depth information of the corresponding offset pixel point; determining the normal deviation of the pixel point based on the normal information of the pixel point and the normal information of the corresponding offset pixel point.

[0066] After determining the multiple offset pixel points for each pixel in each offset direction, the present application can read the depth information and normal information of each offset pixel point of each pixel point. Then, for each pixel point, the present application can obtain multiple depth deviations of the pixel point by analyzing the depth value difference between the pixel point and the various offset pixel points of the pixel point, and select the maximum depth deviation from them as the final depth deviation of the pixel point.

[0067] Moreover, for each pixel point, the present application can obtain multiple normal deviations of the pixel point by analyzing the angle difference between the normal vectors of the pixel point and each offset pixel point of the pixel point, and select the maximum normal deviation as the final normal deviation of the pixel point.

[0068] In the same way as above, the final depth deviation and normal deviation of each pixel can be determined.

[0069] S240: Generate a corresponding particle patch for a specified object in the scene image.

[0070] In this application, in order to achieve personalized stroking of any specified object in a scene image, it is first necessary to perform target detection on the specified object in the scene image, so as to frame the area where the specified object is located in the scene image. At this time, since the geometric shape of the specified object is uncertain, and the bounding box (i.e., the Bounding Box) of the specified object after target detection is a rectangular box, the bounding box of the specified object may include other elements in the scene image, so that when edge detection is performed on the bounding box image, other edge points other than the specified object will also be identified for stroking, and the accurate stroking of the specified object in the scene image cannot be guaranteed.

[0071] Considering that the particle patch is a dynamic surface or geometric structure formed by the dense arrangement or interpolation of a large number of particles as a particle system, it can support the construction of the surface shape of a specified object in the scene image. Moreover, considering that the distance from each boundary point to the center point of the specified object in the three-dimensional space can be guaranteed to be within a certain spatial range, it can be known that the scene points beyond the spatial range will not be scene points on the specified object. Therefore, the present application can determine whether the scene point represented by each pixel point in the scene image is a scene point on the specified object by analyzing whether the distance between the scene point represented by the pixel point and the center point of the specified object is within the corresponding spatial range.

[0072] Therefore, in order to ensure accurate detection of specified objects in the scene image, the present application can select the specified object in the scene image, and then generate a particle patch at the center point of the specified object by analyzing the position of the center point of the specified object. The particle patch, as a two-dimensional plane, will always face the camera shooting the scene image, so that the center point of the particle patch is almost consistent with the center point of the specified object.

[0073] In some possible implementations, the present application may use the following method to generate a particle patch corresponding to a specified object: determine the bounding box of the specified object in the scene image and the maximum geometric length of the bounding box; and generate a particle patch corresponding to the specified object based on the center point of the bounding box and the maximum geometric length.

[0074] Specifically, in order to achieve personalized stroking of any specified object in the scene image, the present application will first use a bounding box algorithm to detect the specified object in the scene image to determine the bounding box in the scene image used to represent the geometric range of the specified object, so as to completely mark the specified object. Then, in order to more accurately mark the pixel range of the specified object in different shapes, the present application can determine the center point and maximum geometric length of the bounding box of the specified object, and the maximum geometric length can be the diagonal length of the bounding box. Furthermore, since the center point of the bounding box is approximately the center point of the specified object, the present application can use the center point of the bounding box as the center point of the particle patch, and the maximum geometric length of the bounding box as the length of the particle patch to generate the particle patch corresponding to the specified object, so that the particle patch can completely cover the specified object.

[0075] Exemplarily, the present application can generate a particle emitter at the center point of the bounding box, and set the emission range of the particle emitter to be determined by the maximum geometric length of the bounding box, and the particle emitter emits a particle with a life cycle of N seconds every N seconds, and sets the movement speed of the particle to 0, to generate a particle patch corresponding to the specified object, so that the particle patch is always facing the camera that shoots the scene image, and ensures that there is only one particle patch at each moment, and the particle patch can cover all pixels in the bounding box to completely cover the specified object.

[0076] S250 , determining a target mask area corresponding to the designated object according to a spatial offset vector of associated pixel points covered by the particle patch facing the center point of the particle patch.

[0077] After generating the corresponding particle patch for the specified object, in order to ensure the accurate detection of the specified object in the scene image, the present application will first determine the various pixel points covered by the particle patch in the scene image as the associated pixel points in the present application. It can be seen from this that the non-associated pixel points in the scene image will definitely not be the relevant pixel points of the specified object, and there is no need to process the non-associated pixel points in the scene image when subsequently analyzing the target mask area of ​​the specified object. Therefore, based on the consideration that the distance from each boundary point to the center point of the specified object in three-dimensional space can be guaranteed to be within a certain spatial range, the present application can determine whether the scene point represented by the associated pixel point is a scene point on the specified object by analyzing whether the distance between the scene point represented by each associated pixel point covered by the particle patch and the center point of the specified object is within the corresponding spatial range.

[0078] Considering that the center point of the particle patch is almost the same as the center point of the specified object, and the particle patch is always facing the camera that captured the scene image, the depth information of all particles in the particle patch is almost the same as the depth information of the center point of the particle patch (that is, the center point of the specified object), and the depth error is within a very small error range, so that the depth information of all particles in the particle patch can be approximated to the depth information of the center point of the particle patch.

[0079] Then, the distance between the scene point represented by each associated pixel point covered by the particle patch and the center point of the specified object can be approximately expressed as the distance between the scene point represented by each associated pixel point covered by the particle patch and the center point of the particle patch. Therefore, the present application can determine the spatial position information of the scene point represented by the associated pixel point by analyzing the coordinate information of each associated pixel point in the two-dimensional screen space and the depth value of the associated pixel point. Moreover, by analyzing the coordinate information of the center point of the particle patch in the two-dimensional screen space and the depth value of the center point, the spatial position information of the center point of the particle patch can be determined.

[0080] Furthermore, by analyzing the spatial changes between the spatial position information of the center point of the particle patch and the spatial position information of the scene point corresponding to each associated pixel point, a spatial offset vector of each associated pixel point facing the center point of the particle patch can be generated. The vector value of the spatial offset vector of each associated pixel point facing the center point of the particle patch can at least include the depth difference between the depth information of the associated pixel point and the depth information of the center point of the particle patch, as well as the screen position deviation between the associated pixel point and the center point of the particle patch. The screen position deviation can be two position deviation values ​​of the associated pixel point and the center point of the particle patch in the horizontal and vertical directions of the two-dimensional screen space, respectively.

[0081] Then, based on the vector values ​​of the spatial offset vector of each associated pixel point facing the center point of the particle patch, the projection length of the spatial offset vector is calculated to represent the spatial distance between the scene point represented by each associated pixel point and the center point of the specified object. In this way, it is determined whether the projection length of the spatial offset vector corresponding to each associated pixel point is within the corresponding distance range set for the specified object, to indicate whether the scene point represented by the associated pixel point is a scene point on the specified object, thereby determining the target mask area corresponding to the specified object to accurately limit the stroke range of the specified object.

[0082] In some implementations, to simplify the calculation of the spatial offset vector of each associated pixel point facing the center point of the particle patch, the present application can normalize the coordinate information of each associated pixel point and the center point of the particle patch in the two-dimensional screen space and remap it to the unified coordinate system [-1, 1]. Then, the center point of the particle patch is the coordinate origin (0, 0) in the unified coordinate system [-1, 1], so that the normalized coordinate information (remapUV_U, remapUV_V) of each associated pixel point is the screen position deviation between the associated pixel point and the center point of the particle patch.

[0083] Then, by analyzing the depth information of each associated pixel point (denoted as screenDepth) and the particle patch depth (denoted as particleDepth) represented by the depth information of the center point of the particle patch, the depth difference between the depth information of each associated pixel point and the particle patch depth represented by the depth information of the center point of the particle patch can be calculated as depthDiff = abs(screenDepth-particleDepth).

[0084] Based on the depth difference between each associated pixel's depth information and the particle patch's depth, and combined with the screen position deviation between each associated pixel and the particle patch's center point, a spatial offset vector for each associated pixel facing the particle patch's center point can be determined. This spatial offset vector can be expressed as particleDepthData = (depthDiff, remapUV_U, remapUV_V).

[0085] Then, the target mask area corresponding to the specified object can be expressed as:

[0086] sphereMask=max(1-sqrt(dot(particleDepthData,particleDepthData)),0)

[0087] Among them, the projection length of the spatial offset vector of each associated pixel point facing the center point of the particle patch can be calculated through sqrt(dot(particleDepthData,particleDepthData)). Moreover, in the normalized unified coordinate system [-1, 1], it can be known that the maximum distance of the corresponding spatial range between the relevant pixels and the center point of the specified object is 1. Then, by determining whether the projection length of the spatial offset vector of each associated pixel point facing the center point of the particle patch exceeds the maximum distance 1, the target mask area sphereMask corresponding to the specified object is generated.

[0088] S260 , determining a depth stroke parameter of the target pixel point according to the depth deviation of the target pixel point in the target mask area.

[0089] In order to ensure the accurate tracing of the specified object in the scene image, the present application can use different methods to judge the possibility that the target pixel point belongs to the edge point of the specified object by analyzing the specific difference in depth deviation and normal deviation of each target pixel point respectively, so as to subsequently judge whether the target pixel point belongs to the edge point of the specified object and needs to be stroked by comprehensively analyzing the combined deviation size of the target pixel points.

[0090] Moreover, in response to whether each target pixel needs to be stroked, the present application can set a stroke transparency to indicate the depth of the line used for stroking each target pixel to form different stroke effects. At this time, if the difference in the depth deviation of a target pixel is greater, it means that the depth mutation of the target pixel is more obvious, and the target pixel is more likely to be the edge contour point of the specified object and need to be stroked. If the difference in the depth deviation of a target pixel is smaller, it means that the depth mutation of the target pixel is less obvious, and the target pixel is more likely not to be the edge contour point of the specified object and does not need to be stroked. Therefore, the present application can set the stroke transparency of any target pixel to vary inversely with the depth deviation of the target pixel. Then, the greater the depth deviation of a target pixel, the smaller the stroke transparency of the target pixel, so that the stroke line of the target pixel is darker, and the more obvious the stroke effect is at the target pixel; and the smaller the depth deviation of a target pixel, the greater the stroke transparency of the target pixel, so that the stroke line of the target pixel is lighter, and an inconspicuous stroke effect is presented.

[0091] Therefore, when determining each target pixel point within the target mask area, this application will also determine the depth deviation of each target pixel point, and determine the first stroke transparency used by the target pixel point under the depth deviation according to the inverse proportional change law between the stroke transparency and the depth deviation, as the depth stroke parameter of the target pixel point.

[0092] S270 , determining a normal stroke parameter of the target pixel point according to the normal deviation of the target pixel point in the target mask area.

[0093] For the transparency of the stroke of each target pixel, if the difference in the normal deviation of a target pixel is greater, it means that the normal mutation of the target pixel is more obvious, and the target pixel is more likely to be the edge contour point of the specified object and need to be stroked. If the difference in the normal deviation of a target pixel is smaller, it means that the normal mutation of the target pixel is less obvious, and the target pixel is more likely not to be the edge contour point of the specified object and does not need to be stroked. Therefore, the present application can set the transparency of the stroke of any target pixel to vary inversely with the normal deviation of the target pixel. Then, the greater the normal deviation of a target pixel, the smaller the stroke transparency of the target pixel, so that the stroke line of the target pixel is darker, and a more obvious stroke effect is presented at the target pixel; and the smaller the normal deviation of a target pixel, the greater the stroke transparency of the target pixel, so that the stroke line of the target pixel is lighter, and an inconspicuous stroke effect is presented.

[0094] Therefore, when determining each target pixel point within the target mask area, this application will also determine the normal deviation of each target pixel point, and determine the second stroke transparency used by the target pixel point under the normal deviation according to the inverse proportional change law between the stroke transparency and the normal deviation, as the normal stroke parameter of the target pixel point.

[0095] It should be understood that S260 and S270 in this application are two different determination methods for determining the corresponding stroke parameters of the target pixel points using the depth deviation and normal deviation of the target pixel points in the target mask area respectively. There is no order of execution and they can be executed in parallel.

[0096] S280 , determining corresponding stroke intensity information according to the depth stroke parameter and the normal stroke parameter of each target pixel point, so as to generate a stroke effect of the designated object.

[0097] After determining the depth stroke parameters and normal stroke parameters of each target pixel, in order to ensure the accurate stroke of the specified object in the scene image, the present application can comprehensively analyze the optimal stroke transparency of the target pixel based on the depth stroke parameters and normal stroke parameters of each target pixel. For example, by analyzing the depth of the stroke line represented by the depth stroke parameters and normal stroke parameters of each target pixel, a stroke parameter with a darker stroke line is selected as the optimal stroke transparency of the target pixel. Alternatively, by analyzing the depth of the stroke line represented by the depth stroke parameters and normal stroke parameters of each target pixel, two stroke lines of different depths represented by the two stroke parameters are superimposed, and the stroke transparency corresponding to the superposition of the stroke lines is determined as the optimal stroke transparency of the target pixel.

[0098] Then, by using the optimal stroke transparency of each target pixel point and combining it with the stroke width represented by the associated offset of the target pixel point, the stroke intensity information of the target pixel point can be determined. The stroke intensity information of each target pixel point can be used to more accurately stroke the specified object in the scene image to generate the stroke effect of the specified object, thereby achieving personalized stroke of the specified object in any scene image and improving the flexibility of object stroke.

[0099] The technical solution provided in the embodiment of the present application determines the depth deviation and normal deviation of each pixel under the associated offset based on the depth information and normal information of each pixel in any scene image. And the target mask area corresponding to the specified object is determined based on the spatial offset vector of each pixel in the scene image facing the center point of the specified object in the scene image. Thus, according to the depth deviation and normal deviation of each target pixel in the target mask area, the corresponding stroke intensity information is determined to generate the stroke effect of the specified object, thereby realizing personalized stroke of the specified object in any scene image and improving the flexibility of object stroke. By comprehensively analyzing the depth deviation and normal deviation of each target pixel in the target mask area, the optimal stroke intensity information is set for the specified object to ensure the fineness and naturalness of the object stroke effect and enhance the visual performance of the object after stroke.

[0100] Figure 3 This is a block diagram of the principle of an object outlining device provided in an embodiment of the present application. Figure 3 As shown, the apparatus 300 may include:

[0101] A pixel deviation determination module 310 is configured to determine the depth deviation and normal deviation of each pixel in the scene image under an associated offset based on the depth information and normal information of the pixel;

[0102] a mask region determining module 320 for determining a target mask region corresponding to a designated object in the scene image based on a spatial offset vector of each pixel point in the scene image facing a center point of the designated object in the scene image;

[0103] The object stroke module 330 is configured to determine corresponding stroke intensity information according to the depth deviation and normal deviation of the target pixel point in the target mask area, so as to generate a stroke effect of the designated object.

[0104] In some implementations, the pixel deviation determination module 310 may be specifically configured to:

[0105] For each pixel in the scene image, determining an associated offset of the pixel based on depth information of the pixel;

[0106] Determining, according to the associated offset amount of the pixel point, an offset pixel point of the pixel point in multiple offset directions;

[0107] The depth deviation of the pixel point is determined according to the depth information of the pixel point and the depth information of the corresponding offset pixel point; the normal deviation of the pixel point is determined according to the normal information of the pixel point and the normal information of the corresponding offset pixel point.

[0108] In some implementations, the mask region determination module 320 may include:

[0109] A particle patch generating unit, configured to generate a corresponding particle patch for a specified object in the scene image;

[0110] The mask area determination unit is used to determine the target mask area corresponding to the designated object according to the spatial offset vector of the associated pixel points covered by the particle patch facing the center point of the particle patch.

[0111] In some implementations, the vector value of the spatial offset vector includes at least a depth difference between the depth information of the associated pixel point and the depth of the particle patch and a screen position deviation between the associated pixel point and a center point of the particle patch.

[0112] In some implementations, the particle sheet generation unit may be specifically configured to:

[0113] Determining a bounding box of a specified object in the scene image and a maximum geometric length of the bounding box;

[0114] Generate a particle patch corresponding to the designated object according to the center point of the bounding box and the maximum geometric length.

[0115] In some implementations, the object tracing module 330 may be specifically configured to:

[0116] Determining a depth stroke parameter of the target pixel point according to a depth deviation of the target pixel point within the target mask area;

[0117] Determining a normal stroke parameter of the target pixel point according to a normal deviation of the target pixel point within the target mask area;

[0118] The corresponding stroke intensity information is determined according to the depth stroke parameter and the normal stroke parameter of each target pixel.

[0119] In some implementations, the object stroke apparatus 300 may further include a stroke strength optimization module. The stroke strength optimization module may be used to:

[0120] The stroke intensity information is optimized according to the dot texture and stroke color of the scene image.

[0121] In an embodiment of the present application, the depth deviation and normal deviation of each pixel under the associated offset are determined based on the depth information and normal information of each pixel in any scene image. And the target mask area corresponding to the specified object is determined based on the spatial offset vector of each pixel in the scene image facing the center point of the specified object in the scene image. Thus, the corresponding stroke intensity information is determined based on the depth deviation and normal deviation of each target pixel in the target mask area to generate the stroke effect of the specified object, thereby achieving personalized stroke of the specified object in any scene image and improving the flexibility of object stroke. By comprehensively analyzing the depth deviation and normal deviation of each target pixel in the target mask area, the optimal stroke intensity information is set for the specified object to ensure the fineness and naturalness of the object stroke effect and enhance the visual performance of the object after stroke.

[0122] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, Figure 3 The device 300 shown can execute any method embodiment provided in the present application, and the aforementioned and other operations and / or functions of each module in the device 300 are respectively for implementing the corresponding processes in each method of the embodiment of the present application. For the sake of brevity, they will not be repeated here.

[0123] The above describes the device 300 of the embodiment of the present application from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. Optionally, the software module can be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps in the above method embodiment in conjunction with its hardware.

[0124] Figure 4 A schematic block diagram of an electronic device provided in an embodiment of the present application.

[0125] like Figure 4 As shown, the electronic device 400 may include:

[0126] The memory 410 and the processor 420 are configured to store computer programs and transmit the program code to the processor 420. In other words, the processor 420 can call and run the computer program from the memory 410 to implement the method in the embodiment of the present application.

[0127] For example, the processor 420 may be configured to execute the above method embodiments according to instructions in the computer program.

[0128] In some embodiments of the present application, the processor 420 may include but is not limited to:

[0129] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0130] In some embodiments of the present application, the memory 410 includes but is not limited to:

[0131] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0132] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 410 and executed by the processor 420 to implement the method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0133] like Figure 4 As shown, the electronic device may further include:

[0134] The transceiver 430 may be connected to the processor 420 or the memory 410 .

[0135] The processor 420 may control the transceiver 430 to communicate with other devices. Specifically, the processor 420 may send information or data to other devices or receive information or data sent by other devices. The transceiver 430 may include a transmitter and a receiver. The transceiver 430 may further include one or more antennas.

[0136] It should be understood that the various components in the electronic device are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0137] The present application also provides a computer storage medium having a computer program stored thereon, which, when executed by a computer, enables the computer to perform the method of the above-mentioned method embodiment. In other words, the present application also provides a computer program product containing instructions, which, when executed by a computer, enables the computer to perform the method of the above-mentioned method embodiment.

[0138] When implemented in software, the functions can be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media can be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired computer program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, or a twisted pair, as examples, then the coaxial cable, fiber optic cable, or twisted pair are included in the definition of medium. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), and Blu-Ray® disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0139] In one embodiment, the techniques described herein can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the software can be executed in a computer system, which can include one or more computers. The software can be stored on one or more computer readable media, such as a magnetic disk, optical disk, or solid state memory. The computer readable media can be distributed among one or more computer systems.

[0140] In several embodiments provided in the present application, it should be understood that the disclosed system, device, and method can be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and the division of the modules is merely a logical function division. In actual implementation, another division manner can be used, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed modules can be indirect coupling or communication connection through some interfaces, devices or modules, and can be electrical, mechanical or other forms.

[0141] Modules described as separate components may or may not be physically separate, and components displayed as modules may or may not be physical modules, i.e., they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected based on actual needs to achieve the purpose of the present embodiment. For example, the functional modules in the various embodiments of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module.

[0142] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for outlining an object, characterized in that: include: Determining the depth deviation and normal deviation of each pixel in the scene image under the associated offset according to the depth information and normal information of the pixel; determining a target mask area corresponding to the specified object according to a spatial offset vector of each pixel point in the scene image facing a center point of the specified object in the scene image; Corresponding stroke intensity information is determined according to the depth deviation and the normal deviation of the target pixel point in the target mask area to generate a stroke effect of the designated object.

2. The method according to claim 1, characterized in that The step of determining the depth deviation and the normal deviation of each pixel under the associated offset according to the depth information and the normal information of each pixel in the scene image includes: For each pixel in the scene image, determining an associated offset of the pixel based on the depth information of the pixel; Determining, according to the associated offset amount of the pixel point, an offset pixel point of the pixel point in multiple offset directions; The depth deviation of the pixel point is determined according to the depth information of the pixel point and the depth information of the corresponding offset pixel point; the normal deviation of the pixel point is determined according to the normal information of the pixel point and the normal information of the corresponding offset pixel point.

3. The method according to claim 1, characterized in that The determining, based on a spatial offset vector of each pixel point in the scene image facing a center point of the specified object in the scene image, a target mask area corresponding to the specified object includes: Generate a corresponding particle patch for a specified object in the scene image; The target mask area corresponding to the designated object is determined according to a spatial offset vector of the associated pixel points covered by the particle patch facing the center point of the particle patch.

4. The method according to claim 3, characterized in that The vector value of the spatial offset vector includes at least a depth difference between the depth information of the associated pixel point and the depth of the particle patch and a screen position deviation between the associated pixel point and the center point of the particle patch.

5. The method according to claim 3, characterized in that Generating a corresponding particle patch for a specified object in the scene image includes: Determining a bounding box of a specified object in the scene image and a maximum geometric length of the bounding box; Generate a particle patch corresponding to the specified object according to the center point of the bounding box and the maximum geometric length.

6. The method according to claim 1, characterized in that The determining corresponding stroke intensity information according to the depth deviation and the normal deviation of the target pixel point in the target mask area includes: Determining a depth stroke parameter of the target pixel point according to a depth deviation of the target pixel point within the target mask area; Determining a normal stroke parameter of the target pixel point according to a normal deviation of the target pixel point within the target mask area; The corresponding stroke intensity information is determined according to the depth stroke parameter and the normal stroke parameter of each target pixel.

7. The method according to claim 1, characterized in that The method further comprises: The stroke intensity information is optimized according to the dot texture and stroke color of the scene image.

8. A device for outlining an object, characterized in that: include: A pixel deviation determination module is used to determine the depth deviation and normal deviation of each pixel under the associated offset based on the depth information and normal information of the pixel in the scene image; a mask region determination module, configured to determine a target mask region corresponding to a specified object in the scene image based on a spatial offset vector of each pixel point in the scene image facing a center point of the specified object in the scene image; The object stroke module is used to determine corresponding stroke intensity information according to the depth deviation and normal deviation of the target pixel point in the target mask area to generate the stroke effect of the specified object.

9. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the object tracing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that Used to store a computer program, wherein the computer program enables a computer to execute the object stroke method according to any one of claims 1 to 7.

Images