A method, apparatus, device and storage medium for object sketching

Abstract

Embodiments of the present application provide a method, device and equipment for object outlining, and a storage medium. The method comprises: determining, according to depth information and normal information of each pixel point in a scene image, a depth deviation and a normal deviation of the pixel point under an associated offset; determining a target mask region corresponding to a specified object in the scene image according to a spatial offset vector of each pixel point in the scene image facing a center point of the specified object; and determining corresponding outlining intensity information according to the depth deviation and the normal deviation of a target pixel point in the target mask region, to generate an outlining effect of the specified object. Embodiments of the present application can realize personalized outlining of a specified object in any scene image, improve the flexibility of object outlining, ensure the fineness and naturalness of the object outlining effect, and enhance the visual performance effect after object outlining.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to a method, apparatus, device, and storage medium for outlining objects. Background Technology

[0002] In the field of image rendering, object outlining can highlight the edge contour of any specified object during the image or video rendering process, making the specified object more clearly visible in the image or video and enhancing the visual effect of the specified object.

[0003] In related technologies, post-processing effects based on screen space are typically used to perform global outlining on any image using depth and normal information. However, this approach cannot provide personalized outlining effects for specific objects in the image, thus limiting the effectiveness of existing image outlining methods. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for object outlining, enabling personalized outlining of a specified object in any scene image and improving the flexibility of object outlining.

[0005] In a first aspect, embodiments of this application provide a method for outlining an object, the method comprising:

[0006] Based on the depth and normal information of each pixel in the scene image, determine the depth deviation and normal deviation of the pixel under the associated offset;

[0007] The target mask region corresponding to the specified object is determined based on the spatial offset vector of each pixel in the scene image toward the center point of the specified object in the scene image.

[0008] Based on the depth deviation and normal deviation of the target pixels within the target mask area, the corresponding stroke intensity information is determined to generate the stroke effect of the specified object.

[0009] Secondly, embodiments of this application provide an object outlining device, the device comprising:

[0010] The pixel deviation determination module is used to determine the depth deviation and normal deviation of each pixel under the associated offset based on the depth information and normal information of each pixel in the scene image.

[0011] The mask region determination module is used to determine the target mask region corresponding to the specified object based on the spatial offset vector of each pixel in the scene image facing the center point of the specified object in the scene image.

[0012] The object outlining module is used to determine the corresponding outlining intensity information based on the depth deviation and normal deviation of the target pixels within the target mask area, so as to generate the outlining effect of the specified object.

[0013] Thirdly, embodiments of this application provide an electronic device, which includes:

[0014] A processor and a memory, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to perform the object outlining method provided in the first aspect of this application.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium for storing a computer program that causes a computer to perform the object outlining method provided in the first aspect of this application.

[0016] Fifthly, embodiments of this application provide a computer program product, including a computer program / instructions that, when executed by a processor, implement the object outlining method provided in the first aspect of this application.

[0017] The technical solution provided in this application determines the depth deviation and normal deviation of each pixel under the associated offset based on the depth and normal information of each pixel in any scene image. Furthermore, it determines the target mask region corresponding to the specified object based on the spatial offset vector of each pixel facing the center point of a specified object in the scene image. Therefore, based on the depth deviation and normal deviation of each target pixel within the target mask region, the corresponding stroke intensity information is determined to generate the stroke effect of the specified object, thereby achieving personalized strokes for a specified object in any scene image and improving the flexibility of object strokes. By comprehensively analyzing the depth deviation and normal deviation of each target pixel within the target mask region, the optimal stroke intensity information is set for the specified object, ensuring the fineness and naturalness of the object stroke effect and enhancing the visual performance of the object after stroke. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a method for outlining an object, as provided in an embodiment of this application;

[0020] Figure 2A flowchart illustrating another method for outlining an object, as provided in an embodiment of this application;

[0021] Figure 3 A schematic block diagram of an object outlining device provided in an embodiment of this application;

[0022] Figure 4 A schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0025] To address the limitations of traditional image outlining methods, which cannot provide personalized outlining effects for specific objects in an image, this application proposes a novel object outlining scheme. By analyzing the depth and normal deviations of each pixel in any scene image under associated offsets, and the target mask region corresponding to the specified object in the scene image, the depth and normal deviations of each target pixel within the target mask region are determined. This comprehensive analysis yields the optimal outlining intensity information for the specified object, providing a personalized outlining effect and enhancing the flexibility of object outlining.

[0026] Figure 1This is a flowchart illustrating a method for outlining an object, as provided in an embodiment of this application. This method can be executed by the object outlining apparatus provided in this application. The object outlining apparatus can be implemented in any software and / or hardware manner. Exemplarily, the object outlining apparatus can be applied to any electronic device, including but not limited to tablet computers, mobile phones (such as foldable phones, large-screen phones, etc.), wearable devices, in-vehicle devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), smart TVs, smart screens, high-definition TVs, 4K TVs, smart speakers, smart projectors, and other computing devices. This disclosure does not impose any limitations on the specific type of electronic device.

[0027] Specifically, such as Figure 1 As shown, the method may include the following steps:

[0028] S110, based on the depth information and normal information of each pixel in the scene image, determine the depth deviation and normal deviation of the pixel under the associated offset.

[0029] In this application, the scene image can be an image captured facing the surrounding environment of any scene, and can be composed of multiple objects, environmental backgrounds, and action elements in any scene. Each pixel in the scene image refers to each individual pixel in the scene image. The depth information of each pixel refers to the distance from the scene point it represents to the camera, and can be a depth value. The normal information of each pixel refers to the normal direction of the surface on which the scene point it represents is located, and can be a three-dimensional vector to represent the specific orientation of the surface on which the scene point is located.

[0030] Image outlining primarily involves adding lines to the edge contours of a scene image to make them clearer and more visible. The edge contours in a scene image can be represented by abrupt changes in pixel depth (e.g., abrupt changes in depth at the boundary between any object and the background) and abrupt changes in the normal direction of the surface on which the pixel is located (e.g., abrupt changes in the normal direction of multiple surfaces at the edge corners of a geometric object, while the depth change is not obvious).

[0031] Therefore, in order to ensure the accuracy of image outlining, this application first needs to combine the depth information and normal information of each pixel in the scene image to determine whether each pixel in the scene image belongs to an edge point and needs to be outlining.

[0032] Understandably, in the field of image rendering, the geometry information of each pixel in any scene image is usually cached in the geometry buffer (G-buffer). This geometry information can include the pixel's color, depth, normals, etc., to support deferred rendering of the scene image.

[0033] Therefore, for any scene image, this application first reads the depth value and normal vector of each pixel in the scene image from the G-buffer, thereby obtaining the depth information and normal information of each pixel.

[0034] It should be understood that objects in a scene image follow the perspective rule of objects appearing larger when closer and smaller when farther away. This means that the smaller the depth value of a pixel in a scene image, the closer the scene point it represents, and the larger the offset is required to determine whether there are abrupt changes in depth and normals. On the other hand, the larger the depth value of a pixel in a scene image, the farther the scene point it represents, and the smaller the offset is required to determine whether there are abrupt changes in depth and normals.

[0035] Therefore, to ensure the accuracy of edge detection in the scene image, this application sets an associated offset for each pixel based on its depth information, thereby accurately analyzing whether there are abrupt changes in depth and normals at each pixel. Specifically, the smaller the depth value of any pixel, the larger its associated offset; conversely, the larger the depth value of any pixel, the smaller its associated offset.

[0036] Furthermore, for each pixel in the scene image, this application can determine the depth information and normal information of that pixel after offset according to its associated offset. Then, by analyzing the difference between the two depth values ​​of each pixel before and after the offset and the angle between the two normal vectors, the depth deviation and normal deviation of each pixel under the associated offset can be obtained.

[0037] It is understandable that, considering the smaller the depth value of a pixel in a scene image, the closer the scene point it represents, the wider the stroke width required when that scene point is used as an edge contour point for outlining; conversely, the larger the depth value of a pixel in a scene image, the farther away the scene point it represents, and the narrower the stroke width required when that scene point is used as an edge contour point for outlining, thus ensuring the consistency of the outlining effect at different distances in the scene image. Therefore, it can be seen that the correlation offset of each pixel in the scene image and the change pattern of the stroke width are the same. Therefore, to ensure the accuracy of image outlining, this application can use the correlation offset of each pixel in the scene image as the stroke width for that pixel when performing outlining.

[0038] S120: Determine the target mask region corresponding to the specified object based on the spatial offset vector of each pixel in the scene image facing the center point of the specified object in the scene image.

[0039] In this application, to achieve personalized outlining of any specified object in a scene image, it is first necessary to perform object detection on the specified object in the scene image, thereby selecting the region where the specified object is located in the scene image. At this time, since the geometry of the specified object is indeterminate, and the bounding box of the specified object after object detection is a rectangle, the bounding box of the specified object may include other elements in the scene image. This means that when performing edge detection on the bounding box image, other edge points besides the specified object will also be identified and outlined, thus failing to guarantee accurate outlining of the specified object in the scene image.

[0040] Considering that the distances from each boundary point to the center point of a specified object in 3D space are guaranteed to be within a certain spatial range, and that scene points outside this spatial range are unlikely to be scene points on the specified object, this application can determine whether the scene point represented by each pixel in the scene image is a scene point on the specified object by analyzing whether the distance between the scene point represented by that pixel and the center point of the specified object is within the corresponding spatial range. This allows for the determination of the target mask region corresponding to the specified object in the scene image, thus more accurately marking the pixel range of the specified object under different shapes.

[0041] Therefore, for each pixel in the scene image, this application can determine the spatial location information of the scene point represented by the pixel by analyzing the coordinate information of the pixel in the two-dimensional screen space and the depth value of the pixel. Furthermore, by performing target detection on a specified object in the scene image to select the region where the specified object is located, this application can determine the center point of the bounding box of the specified object after target detection, which is taken as the center point of the specified object. Then, by analyzing the coordinate information of the center point of the specified object in the two-dimensional screen space and the depth value of the center point, the spatial location information of the center point of the specified object can be determined.

[0042] Furthermore, by analyzing the spatial changes between the spatial location information of the center point of the specified object and the spatial location information of the scene point represented by each pixel in the scene image, a spatial offset vector of each pixel in the scene image facing the center point of the specified object can be generated. Then, based on the vector values ​​of each vector in the spatial offset vector of each pixel in the scene image facing the center point of the specified object, the projection length of the spatial offset vector is calculated to represent the spatial distance between the scene point represented by each pixel in the scene image and the center point of the specified object. This determines whether the projection length of the spatial offset vector corresponding to each pixel is within the corresponding distance range set for the specified object, indicating whether the scene point represented by the pixel is a scene point on the specified object. Thus, the target mask region corresponding to the specified object is determined in the scene image, allowing for more accurate marking of the pixel range of the specified object under different shapes, thereby precisely limiting the outline range of the specified object.

[0043] S130: Based on the depth deviation and normal deviation of the target pixels within the target mask area, determine the corresponding stroke intensity information to generate the stroke effect of the specified object.

[0044] After determining the target mask region corresponding to a specified object from the scene image, this application can determine each target pixel point in the scene image that is located within the target mask region, and determine the depth deviation and normal deviation of each target pixel point. The scene point represented by the target pixel point can be a scene point on the specified object.

[0045] Then, to ensure accurate outlining of specified objects in the scene image, this application can determine the likelihood that a target pixel belongs to the edge point of the specified object by analyzing the specific differences in depth deviation and normal deviation of each target pixel. To this end, for each target pixel, this application can set an outlining transparency to indicate the line depth used for outlining each target pixel, thereby creating different outlining effects.

[0046] Understandably, the greater the difference in depth deviation or normal deviation of a target pixel, the more pronounced the abrupt change in depth or normal at that pixel, and the more likely it is to be an edge contour point of the specified object, thus requiring an outline. Conversely, the smaller the difference in depth deviation or normal deviation, the less pronounced the abrupt change in depth or normal at that pixel, and the less likely it is to be an edge contour point of the specified object, thus requiring no outline. Therefore, this application can set the outline transparency of any target pixel to be inversely proportional to its depth deviation and normal deviation. Thus, the greater the depth deviation and normal deviation of a target pixel, the smaller its outline transparency, resulting in a darker outline line and a more pronounced outline effect at that pixel; conversely, the smaller the depth deviation and normal deviation, the greater its outline transparency, resulting in a lighter outline line and a less noticeable outline effect.

[0047] Therefore, for each target pixel, this application can comprehensively analyze the depth deviation and normal deviation of the target pixel to determine the merging deviation of the target pixel. Based on the inverse relationship between the stroke transparency and the depth deviation and normal deviation, the stroke transparency of the target pixel can be determined. Combined with the stroke width represented by the associated offset of the target pixel, the stroke intensity information of the target pixel can be determined.

[0048] By following the same method described above, the outline intensity information of each target pixel can be determined, thereby enabling more precise outlining of a specified object in the scene image to generate the outline effect of the specified object. This allows for personalized outlining of a specified object in any scene image, improving the flexibility of object outlining.

[0049] Furthermore, to enhance the diversity of object outline effects, this application can optimize the outline intensity information based on the pixel texture and outline color of the scene image, thereby achieving an expandable outline effect for specified objects in the scene image. In other words, this application can use preset pixel density information to perform pixel calculations at corresponding intervals in the horizontal and vertical directions of the scene image in two-dimensional screen space to determine the pixel texture of the scene image. This pixel texture can form a dotted dashed line outline format. Moreover, the outline color can be set to represent the color of the outline line, such as a warning color or a technology color.

[0050] Then, the bitmap texture of the scene image and the preset stroke color can be merged into the stroke intensity information corresponding to each target pixel in the target mask area to optimize the specific stroke effect of the specified object, thereby presenting a variety of stroke effects after the specified object is stroked with dotted dashed lines and stroke color.

[0051] It should be noted that this application can also add other stroke style parameters (such as blinking stroke) to the scene image to achieve personalized stroke effects for various visual styles of specified objects in the scene image.

[0052] The technical solution provided in this application determines the depth deviation and normal deviation of each pixel under the associated offset based on the depth and normal information of each pixel in any scene image. Furthermore, it determines the target mask region corresponding to the specified object based on the spatial offset vector of each pixel facing the center point of a specified object in the scene image. Therefore, based on the depth deviation and normal deviation of each target pixel within the target mask region, the corresponding stroke intensity information is determined to generate the stroke effect of the specified object, thereby achieving personalized strokes for a specified object in any scene image and improving the flexibility of object strokes. By comprehensively analyzing the depth deviation and normal deviation of each target pixel within the target mask region, the optimal stroke intensity information is set for the specified object, ensuring the fineness and naturalness of the object stroke effect and enhancing the visual performance of the object after stroke.

[0053] As an optional implementation of this application, in order to ensure the personalized outline effect of a specified object in any scene image, this application can provide a detailed explanation of the specific determination process of the depth deviation and normal deviation of any pixel in the scene image, the target mask area corresponding to the specified object, and the outline intensity information of the target pixel.

[0054] Figure 2 A flowchart illustrating another method for stroking an object, provided in an embodiment of this application, is provided. This method may specifically include the following steps:

[0055] S210: For each pixel in the scene image, determine the associated offset of the pixel based on the pixel's depth information.

[0056] For any scene image, objects in the scene image will conform to the perspective law of near objects appearing larger and distant objects appearing smaller. This means that the smaller the depth value of a pixel in the scene image, the closer the scene point it represents, and the larger the offset is required to determine whether there are abrupt changes in depth and normals. On the other hand, the larger the depth value of a pixel in the scene image, the farther the scene point it represents, and the smaller the offset is required to determine whether there are abrupt changes in depth and normals.

[0057] Therefore, in order to ensure accurate outlining in the scene image, this application can set an associated offset for each pixel that matches the depth information of that pixel based on the inverse relationship between the depth information and the associated offset of each pixel in the scene image.

[0058] In some implementations, this application can pre-define an ideal depth value (denoted as standardDepth) according to the scene type of any scene image, and set a suitable offset range [minOffset, maxOffset] for each pixel in the scene image. In this way, the intermediate offset of the offset range [minOffset, maxOffset] (denoted as desiredOffsetAtStandard) can be used as the associated offset matched under the ideal depth value standardDepth.

[0059] Then, based on the inverse relationship between the depth information and associated offset of each pixel in the scene image, the first ratio between the set ideal depth value standardDepth and the depth information of each pixel, and the second ratio between the initial associated offset of each pixel and the intermediate offset desiredOffsetAtStandard that matches the ideal depth value standardDepth, can be determined. This allows the initial associated offset of each pixel to be calculated. Furthermore, by determining whether the initial associated offset of each pixel is within the offsettable range [minOffset, maxOffset], the final associated offset of each pixel is determined. For example, if the initial associated offset of a pixel is within the offsettable range [minOffset, maxOffset], then this initial associated offset is taken as the final associated offset of that pixel. If the initial associated offset of a pixel is less than the minimum offset minOffset of the offsettable range [minOffset, maxOffset], then this minimum offset minOffset is taken as the final associated offset of that pixel. If the initial associated offset of a pixel is greater than the maximum offset maxOffset of the offset range [minOffset, maxOffset], then the maximum offset maxOffset is used as the final associated offset of the pixel.

[0060] For example, assuming the depth information of each pixel in the scene image is denoted as planeDepth, the formula for calculating the association offset of each pixel can be: associationOffset = clamp(desiredOffsetAtStandard*(standardDepth / planeDepth),minOffset,maxOffset).

[0061] S220, based on the associated offset of the pixel, determine the offset pixel in multiple offset directions.

[0062] Considering that when a pixel in a scene image is an edge contour point, depth abrupt changes or normal abrupt changes may occur in multiple different directions, this application first sets multiple offset directions, such as upward offset, leftward offset, downward offset, or rightward offset, in order to ensure accurate edge detection for each pixel in the scene image.

[0063] Therefore, after determining the associated offset of each pixel in the scene image, for each pixel, this application can offset the pixel in each offset direction according to the associated offset of the pixel, and determine the offset pixel in each offset direction.

[0064] Following the same method described above, multiple offset pixels for each pixel in each offset direction can be determined.

[0065] S230, determine the depth deviation of the pixel based on the depth information of the pixel and the depth information of the corresponding offset pixel; determine the normal deviation of the pixel based on the normal information of the pixel and the normal information of the corresponding offset pixel.

[0066] After determining the multiple offset pixels of each pixel in each offset direction, this application can read the depth information and normal information of each offset pixel of each pixel. Then, for each pixel, this application can obtain multiple depth deviations of the pixel by analyzing the depth value differences between the pixel and each offset pixel of the pixel, and select the largest depth deviation as the final depth deviation of the pixel.

[0067] Furthermore, for each pixel, this application can obtain multiple normal deviations of the pixel by analyzing the angle difference between the normal vectors of the pixel and each offset pixel, and select the largest normal deviation as the final normal deviation of the pixel.

[0068] Following the same method described above, the final depth deviation and normal deviation of each pixel can be determined.

[0069] S240 generates corresponding particle patches for specified objects in the scene image.

[0070] In this application, to achieve personalized outlining of any specified object in a scene image, it is first necessary to perform object detection on the specified object in the scene image, thereby selecting the region where the specified object is located in the scene image. At this time, since the geometry of the specified object is indeterminate, and the bounding box of the specified object after object detection is a rectangle, the bounding box of the specified object may include other elements in the scene image. This means that when performing edge detection on the bounding box image, other edge points besides the specified object will also be identified and outlined, thus failing to guarantee accurate outlining of the specified object in the scene image.

[0071] Considering that particle patches, as dynamic surfaces or geometric structures formed by the dense arrangement or interpolation of a large number of particles in a particle system, can support the construction of the surface shape of a specified object in a scene image. Furthermore, considering that the distances from each boundary point of the specified object to its center point in three-dimensional space can be guaranteed to be within a certain spatial range, it can be known that scene points exceeding this spatial range are not scene points on the specified object. Therefore, this application can determine whether the scene point represented by each pixel in the scene image is a scene point on the specified object by analyzing whether the distance between the scene point represented by each pixel and the center point of the specified object is within the corresponding spatial range.

[0072] Therefore, in order to ensure accurate detection of a specified object in the scene image, this application can select the specified object in the scene image and then generate a particle patch at the center point of the specified object by analyzing the location of the center point of the specified object. The particle patch, as a two-dimensional plane, will always face the camera capturing the scene image, so that the center point of the particle patch is almost consistent with the center point of the specified object.

[0073] In some implementations, this application may use the following method to generate particle patches corresponding to a specified object: determine the bounding box of the specified object in the scene image and the maximum geometric length of the bounding box; generate particle patches corresponding to the specified object based on the center point of the bounding box and the maximum geometric length.

[0074] Specifically, to achieve personalized outlining of any specified object in a scene image, this application first uses a bounding box algorithm to detect the specified object in the scene image, determining the bounding box representing the geometric range of the specified object to completely mark it. Then, to more accurately mark the pixel range of the specified object under different shapes, this application can determine the center point and maximum geometric length of the bounding box of the specified object, where the maximum geometric length can be the diagonal length of the bounding box. Furthermore, since the center point of the bounding box is approximately the center point of the specified object, this application can use the center point of the bounding box as the center point of the particle patch, and the maximum geometric length of the bounding box as the length of the particle patch, to generate the particle patch corresponding to the specified object, ensuring that the particle patch completely covers the specified object.

[0075] For example, this application can generate a particle emitter at the center point of the bounding box, and set the emission range of the particle emitter to be determined by the maximum geometric length of the bounding box. The particle emitter emits a particle with a lifespan of N seconds every N seconds and sets the velocity of the particle to 0 to generate a particle patch corresponding to the specified object. This ensures that the particle patch always faces the camera capturing the scene image, and that only one particle patch appears at any given moment. The particle patch can cover all the pixels in the bounding box to completely cover the specified object.

[0076] S250: Determine the target mask region corresponding to the specified object based on the spatial offset vector of the associated pixels covered by the particle patch facing the center point of the particle patch.

[0077] After generating the corresponding particle patch for the specified object, to ensure accurate detection of the specified object in the scene image, this application first determines each pixel point covered by the particle patch in the scene image as the associated pixel point in this application. Therefore, it is known that unassociated pixels in the scene image will not be associated pixels of the specified object, and there is no need to process unassociated pixels in the scene image when analyzing the target mask region of the specified object. Therefore, based on the consideration that the distance from each boundary point to the center point of the specified object in 3D space can be guaranteed to be within a certain spatial range, this application can determine whether the scene point represented by each associated pixel point covered by the particle patch is a scene point on the specified object by analyzing whether the distance between the scene point represented by each associated pixel point and the center point of the specified object is within the corresponding spatial range.

[0078] Considering that the center point of the particle patch is almost identical to the center point of the specified object, and the particle patch always faces the camera capturing the scene image, the depth information of all particles in the particle patch is almost identical to the depth information of the center point of the particle patch (i.e., the center point of the specified object). The depth error is within a very small range, making the depth information of all particles in the particle patch approximate the depth information of the center point of the particle patch.

[0079] Therefore, the distance between the scene point represented by each associated pixel covered by the particle patch and the center point of the specified object can be approximately expressed as the distance between the scene point represented by each associated pixel covered by the particle patch and the center point of the particle patch. Thus, this application can determine the spatial location information of the scene point represented by each associated pixel by analyzing the coordinate information of each associated pixel in the two-dimensional screen space and the depth value of that associated pixel. Furthermore, by analyzing the coordinate information of the center point of the particle patch in the two-dimensional screen space and the depth value of that center point, the spatial location information of the center point of the particle patch can be determined.

[0080] Furthermore, by analyzing the spatial changes between the spatial location information of the center point of the particle patch and the spatial location information of the scene point represented by each associated pixel, a spatial offset vector of each associated pixel facing the center point of the particle patch can be generated. The vector value of the spatial offset vector of each associated pixel facing the center point of the particle patch can include at least the depth difference between the depth information of the associated pixel and the depth of the particle patch represented by the depth information of the center point of the particle patch, as well as the screen position deviation between the associated pixel and the center point of the particle patch. This screen position deviation can be two positional deviation values ​​of the associated pixel and the center point of the particle patch in the horizontal and vertical directions of the two-dimensional screen space, respectively.

[0081] Then, based on the vector values ​​in the spatial offset vector of each associated pixel facing the center point of the particle patch, the projection length of the spatial offset vector is calculated to represent the spatial distance between the scene point represented by each associated pixel and the center point of the specified object. This is used to determine whether the projection length of the spatial offset vector corresponding to each associated pixel is within the corresponding distance range set by the specified object, and whether the scene point represented by the associated pixel is a scene point on the specified object. This determines the target mask area corresponding to the specified object, and precisely limits the outline range of the specified object.

[0082] In some implementations, to simplify the calculation of the spatial offset vector of each associated pixel toward the center point of the particle patch, this application can normalize the coordinate information of each associated pixel and the center point of the particle patch in the two-dimensional screen space and remap them to a unified coordinate system [-1, 1]. Then, the center point of the particle patch is the origin (0, 0) of the unified coordinate system [-1, 1], so that the normalized coordinate information (remapUV_U, remapUV_V) of each associated pixel is the screen position deviation between the associated pixel and the center point of the particle patch.

[0083] Then, by analyzing the depth information of each associated pixel (denoted as screenDepth) and the particle depth represented by the depth information of the center point of the particle patch (denoted as particleDepth), the depth difference between the depth information of each associated pixel and the particle depth represented by the depth information of the center point of the particle patch can be calculated as depthDiff = abs(screenDepth - particleDepth).

[0084] Therefore, based on the depth difference between the depth information of each associated pixel and the depth of the particle patch, and combined with the screen position deviation between each associated pixel and the center point of the particle patch, the spatial offset vector of each associated pixel facing the center point of the particle patch can be determined. This spatial offset vector can be represented as particleDepthData = (depthDiff, remapUV_U, remapUV_V).

[0085] Therefore, the target mask region corresponding to a specified object can be represented as:

[0086] sphereMask=max(1-sqrt(dot(particleDepthData,particleDepthData)),0)

[0087] The `sqrt(dot(particleDepthData,particleDepthData))` method calculates the projected length of the spatial offset vector of each associated pixel facing the center point of the particle patch. Furthermore, in the normalized coordinate system [-1, 1], the maximum distance between the associated pixels of a specified object and its center point is determined to be 1. Therefore, by checking whether the projected length of the spatial offset vector of each associated pixel facing the center point of the particle patch exceeds the maximum distance of 1, the target mask region `sphereMask` corresponding to the specified object is generated.

[0088] S260, determine the depth tracing parameters of the target pixels based on the depth deviation of the target pixels within the target mask area.

[0089] To ensure accurate outlining of a specified object in a scene image, this application can analyze the specific differences in depth deviation and normal deviation of each target pixel to determine the likelihood that the target pixel belongs to the edge point of the specified object in different ways. This allows for subsequent comprehensive analysis of the merged deviation of the target pixels to determine whether the target pixel belongs to the edge point of the specified object and needs to be outlined.

[0090] Furthermore, this application allows setting a stroke transparency to indicate the depth of the stroke used for each target pixel, thus creating different stroke effects. In this case, a larger difference in depth deviation for a target pixel indicates a more pronounced depth change, making it more likely that the target pixel is an edge contour point of the specified object and requires a stroke. Conversely, a smaller difference in depth deviation for a target pixel indicates a less pronounced depth change, making it less likely that the target pixel is not an edge contour point of the specified object and does not require a stroke. Therefore, this application allows setting the stroke transparency of any target pixel to be inversely proportional to its depth deviation. A larger depth deviation for a target pixel results in a smaller stroke transparency, making the stroke line darker and the stroke effect more pronounced; conversely, a smaller depth deviation for a target pixel results in a larger stroke transparency, making the stroke line lighter and the stroke effect less pronounced.

[0091] Therefore, when determining each target pixel within the target mask area, this application also determines the depth deviation of each target pixel and, according to the inverse relationship between the stroke transparency and the depth deviation, determines the first stroke transparency used by the target pixel under the depth deviation, which is then used as the depth stroke parameter of the target pixel.

[0092] S270, determine the normal stroke parameters of the target pixel based on the normal deviation of the target pixel within the target mask area.

[0093] Regarding the stroke opacity of each target pixel, a larger difference in the normal deviation of a target pixel indicates a more pronounced abrupt change in the normal of that pixel, making it more likely that the pixel is an edge contour point of the specified object and requires stroke. Conversely, a smaller difference in the normal deviation of a target pixel indicates a less pronounced abrupt change in the normal of that pixel, making it less likely that the pixel is not an edge contour point of the specified object and does not require stroke. Therefore, this application can set the stroke opacity of any target pixel to be inversely proportional to its normal deviation. A larger normal deviation results in a smaller stroke opacity, leading to a darker stroke line and a more pronounced stroke effect at that pixel; conversely, a smaller normal deviation results in a larger stroke opacity, leading to a lighter stroke line and a less noticeable stroke effect.

[0094] Therefore, when determining each target pixel within the target mask area, this application also determines the normal deviation of each target pixel and, according to the inverse relationship between stroke transparency and normal deviation, determines the second stroke transparency used by the target pixel under the normal deviation, which is then used as the normal stroke parameter of the target pixel.

[0095] It should be understood that S260 and S270 in this application are two different methods of determining the corresponding outline parameters of the target pixel using the depth deviation and normal deviation of the target pixel within the target mask area, respectively. There is no order of execution between them, and they can be executed in parallel.

[0096] S280: Based on the depth stroke parameters and normal stroke parameters of each target pixel, determine the corresponding stroke intensity information to generate the stroke effect of the specified object.

[0097] After determining the depth and normal stroke parameters for each target pixel, to ensure accurate outlining of a specified object in the scene image, this application can comprehensively analyze the optimal stroke transparency of the target pixel based on its depth and normal stroke parameters. For example, by analyzing the depth and normal stroke parameters of each target pixel and the depth of the stroke lines they represent, the darker stroke parameter can be selected as the optimal stroke transparency for that target pixel. Alternatively, by analyzing the depth and normal stroke parameters of each target pixel and the depth of the stroke lines they represent, two stroke lines of different depths can be superimposed, and the stroke transparency used after the superposition can be determined as the optimal stroke transparency for that target pixel.

[0098] Then, by using the optimal stroke transparency of each target pixel and combining it with the stroke width represented by the associated offset of that target pixel, the stroke intensity information of that target pixel can be determined. This allows for more precise stroke of a specified object in the scene image using the stroke intensity information of each target pixel, thereby generating the stroke effect of the specified object and realizing personalized stroke of a specified object in any scene image, thus improving the flexibility of object stroke.

[0099] The technical solution provided in this application determines the depth deviation and normal deviation of each pixel under the associated offset based on the depth and normal information of each pixel in any scene image. Furthermore, it determines the target mask region corresponding to the specified object based on the spatial offset vector of each pixel facing the center point of a specified object in the scene image. Therefore, based on the depth deviation and normal deviation of each target pixel within the target mask region, the corresponding stroke intensity information is determined to generate the stroke effect of the specified object, thereby achieving personalized strokes for a specified object in any scene image and improving the flexibility of object strokes. By comprehensively analyzing the depth deviation and normal deviation of each target pixel within the target mask region, the optimal stroke intensity information is set for the specified object, ensuring the fineness and naturalness of the object stroke effect and enhancing the visual performance of the object after stroke.

[0100] Figure 3 This is a schematic block diagram of an object outlining device provided in an embodiment of this application. Figure 3 As shown, the device 300 may include:

[0101] The pixel deviation determination module 310 is used to determine the depth deviation and normal deviation of the pixel under the associated offset based on the depth information and normal information of each pixel in the scene image.

[0102] The mask region determination module 320 is used to determine the target mask region corresponding to the specified object based on the spatial offset vector of each pixel in the scene image toward the center point of the specified object in the scene image.

[0103] The object outlining module 330 is used to determine the corresponding outlining intensity information based on the depth deviation and normal deviation of the target pixels within the target mask area, so as to generate the outlining effect of the specified object.

[0104] In some implementations, the pixel deviation determination module 310 can be specifically used for:

[0105] For each pixel in the scene image, the associated offset of the pixel is determined based on the depth information of the pixel;

[0106] Based on the associated offset of the pixel, determine the offset pixel in multiple offset directions;

[0107] The depth deviation of the pixel is determined based on the depth information of the pixel and the depth information of the corresponding offset pixel; the normal deviation of the pixel is determined based on the normal information of the pixel and the normal information of the corresponding offset pixel.

[0108] In some implementations, the mask region determination module 320 may include:

[0109] The particle patch generation unit is used to generate corresponding particle patches for specified objects in the scene image.

[0110] The mask region determination unit is used to determine the target mask region corresponding to the specified object based on the spatial offset vector of the associated pixels covered by the particle patch facing the center point of the particle patch.

[0111] In some implementations, the vector value of the spatial offset vector includes at least the depth difference between the depth information of the associated pixel and the depth of the particle patch, as well as the screen position deviation between the associated pixel and the center point of the particle patch.

[0112] In some implementations, the particle patch generation unit can be specifically used for:

[0113] Determine the bounding box of a specified object in the scene image and the maximum geometric length of the bounding box;

[0114] Based on the center point of the bounding box and the maximum geometric length, generate the particle patch corresponding to the specified object.

[0115] In some implementations, the object outlining module 330 can be specifically used for:

[0116] The depth stroke parameters of the target pixels are determined based on the depth deviation of the target pixels within the target mask area;

[0117] The normal stroke parameters of the target pixel are determined based on the normal deviation of the target pixel within the target mask region.

[0118] The corresponding stroke intensity information is determined based on the depth stroke parameters and normal stroke parameters of each target pixel.

[0119] In some implementations, the object outlining device 300 may further include an outlining intensity optimization module. This outlining intensity optimization module can be used for:

[0120] The stroke intensity information is optimized based on the pixel texture and stroke color of the scene image.

[0121] In this embodiment, the depth and normal information of each pixel in any scene image are used to determine the depth deviation and normal deviation of each pixel under the associated offset. Furthermore, based on the spatial offset vector of each pixel in the scene image facing the center point of a specified object in the scene image, the target mask region corresponding to the specified object is determined. Therefore, based on the depth and normal deviation of each target pixel within the target mask region, the corresponding stroke intensity information is determined to generate the stroke effect of the specified object, thereby achieving personalized strokes for a specified object in any scene image and improving the flexibility of object strokes. By comprehensively analyzing the depth and normal deviations of each target pixel within the target mask region, the optimal stroke intensity information is set for the specified object, ensuring the precision and naturalness of the object stroke effect and enhancing the visual performance of the stroked object.

[0122] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, further details will not be provided here. Specifically, Figure 3 The apparatus 300 shown can execute any of the method embodiments provided in this application, and the foregoing and other operations and / or functions of each module in the apparatus 300 are respectively for implementing the corresponding processes in the various methods of the embodiments of this application. For the sake of brevity, they will not be described in detail here.

[0123] The apparatus 300 of this application embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in this application embodiment can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can be located in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.

[0124] Figure 4 A schematic block diagram of an electronic device provided in an embodiment of this application.

[0125] like Figure 4 As shown, the electronic device 400 may include:

[0126] The system includes a memory 410 and a processor 420. The memory 410 stores computer programs and transfers the program code to the processor 420. In other words, the processor 420 can retrieve and run the computer program from the memory 410 to implement the methods described in the embodiments of this application.

[0127] For example, the processor 420 can be used to execute the above-described method embodiments according to instructions in the computer program.

[0128] In some embodiments of this application, the processor 420 may include, but is not limited to:

[0129] General-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0130] In some embodiments of this application, the memory 410 includes, but is not limited to:

[0131] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0132] In some embodiments of this application, the computer program may be divided into one or more modules, which are stored in the memory 410 and executed by the processor 420 to perform the method provided in this application. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0133] like Figure 4 As shown, the electronic device may also include:

[0134] Transceiver 430, which can be connected to processor 420 or memory 410.

[0135] The processor 420 can control the transceiver 430 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 430 may include a transmitter and a receiver. The transceiver 430 may further include antennas, and the number of antennas may be one or more.

[0136] It should be understood that the various components in the electronic device are connected through a bus system, which includes a data bus, a power bus, a control bus, and a status signal bus.

[0137] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, this application also provides a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.

[0138] When implemented using software, it can be implemented entirely or partially as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0139] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0140] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0141] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0142] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for outlining an object, characterized in that, include: Based on the depth and normal information of each pixel in the scene image, the depth deviation and normal deviation of the pixel under the associated offset are determined. Specifically, an ideal depth value is preset according to the scene type of any scene image, and an offset range is set for each pixel in the scene image. The middle offset of the offset range is used as the associated offset matched under the ideal depth value. Based on the spatial offset vector of each pixel in the scene image facing the center point of the specified object in the scene image, the target mask region corresponding to the specified object is determined. In this process, by analyzing the spatial change between the spatial location information of the center point of the specified object and the spatial location information of the scene point represented by each pixel in the scene image, the spatial offset vector of each pixel in the scene image facing the center point of the specified object is generated. Based on the depth deviation and normal deviation of the target pixels within the target mask area, the corresponding stroke intensity information is determined to generate the stroke effect of the specified object.

2. The method according to claim 1, characterized in that, The step of determining the depth deviation and normal deviation of each pixel under the associated offset based on the depth information and normal information of each pixel in the scene image includes: For each pixel in the scene image, the associated offset of the pixel is determined based on the depth information of the pixel; Based on the associated offset of the pixel, determine the offset pixel in multiple offset directions; The depth deviation of the pixel is determined based on the depth information of the pixel and the depth information of the corresponding offset pixel; the normal deviation of the pixel is determined based on the normal information of the pixel and the normal information of the corresponding offset pixel.

3. The method according to claim 1, characterized in that, The step of determining the target mask region corresponding to the specified object based on the spatial offset vector of each pixel in the scene image facing the center point of the specified object in the scene image includes: Generate corresponding particle patches for specified objects in the scene image; The target mask region corresponding to the specified object is determined based on the spatial offset vector of the associated pixels covered by the particle patch facing the center point of the particle patch.

4. The method according to claim 3, characterized in that, The vector value of the spatial offset vector includes at least the depth difference between the depth information of the associated pixel and the depth of the particle patch, as well as the screen position deviation between the associated pixel and the center point of the particle patch.

5. The method according to claim 3, characterized in that, The step of generating corresponding particle patches for specified objects in the scene image includes: Determine the bounding box of a specified object in the scene image and the maximum geometric length of the bounding box; Based on the center point of the bounding box and the maximum geometric length, generate the particle patch corresponding to the specified object.

6. The method according to claim 1, characterized in that, The step of determining the corresponding stroke intensity information based on the depth deviation and normal deviation of the target pixels within the target mask region includes: The depth stroke parameters of the target pixels are determined based on the depth deviation of the target pixels within the target mask area; The normal stroke parameters of the target pixel are determined based on the normal deviation of the target pixel within the target mask region. The corresponding stroke intensity information is determined based on the depth stroke parameters and normal stroke parameters of each target pixel.

7. The method according to claim 1, characterized in that, The method further includes: The stroke intensity information is optimized based on the pixel texture and stroke color of the scene image.

8. A device for outlining an object, characterized in that, include: The pixel deviation determination module is used to determine the depth deviation and normal deviation of each pixel in the scene image based on the depth information and normal information of each pixel. The ideal depth value is preset according to the scene type of any scene image, and an offset range is set for each pixel in the scene image. The middle offset of the offset range is used as the associated offset matched under the ideal depth value. The mask region determination module is used to determine the target mask region corresponding to the specified object based on the spatial offset vector of each pixel in the scene image facing the center point of the specified object in the scene image. The module generates the spatial offset vector of each pixel in the scene image facing the center point of the specified object by analyzing the spatial change between the spatial location information of the center point of the specified object and the spatial location information of the scene point represented by each pixel in the scene image. The object outlining module is used to determine the corresponding outlining intensity information based on the depth deviation and normal deviation of the target pixels within the target mask area, so as to generate the outlining effect of the specified object.

9. An electronic device, characterized in that, include: A processor and a memory, the memory for storing a computer program, the processor for calling and running the computer program stored in the memory to perform the method of tracing an object according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the method of tracing an object as described in any one of claims 1-7.

Images