Special effects image rendering methods, devices, equipment and media

By performing trajectory tracking and meshing processing on target objects in the target video, special effects images that match the user's movement trajectory are generated. This solves the problem of the lack of real-time interactivity in existing special effects images and realizes real-time association of special effects images and expansion of application scenarios.

CN115578495BActive Publication Date: 2026-11-17BEIJING ZITIAO NETWORK TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202211098070.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-08
Publication Date
2026-11-17
Estimated Expiration
2042-09-08

AI Technical Summary

Technical Problem

In existing technologies, special effects images lack real-time interactivity with user behavior, which limits their application scenarios.

Method used

By tracking the trajectory of target objects in the target video, a 3D point set is obtained and meshed to generate a target mesh structure. Then, texture processing is performed to obtain a special effects image that matches the user's motion trajectory.

Benefits of technology

It enables real-time correlation between special effects images and user behavior, expands the application scenarios of special effects, and improves the efficiency of special effects usage.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115578495B_ABST
    Figure CN115578495B_ABST
Patent Text Reader

Abstract

This disclosure provides a method, apparatus, device, and medium for rendering special effects images. The method includes: responding to a user-triggered special effects rendering request; performing trajectory tracking on a target video provided by the user to obtain a three-dimensional point set corresponding to the motion trajectory of a target object in the target video; performing meshing processing on the three-dimensional point set to obtain a target mesh structure; performing texture mapping processing on the target mesh structure to obtain a target special effects image; and displaying the target special effects image corresponding to the motion trajectory of the target object in the target video. The technical solution of this disclosure generates special effects images using three-dimensional point sets, obtaining three-dimensional special effects images and improving the displayability of the special effects images.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method, apparatus, device, and medium for drawing special effects images. Background Technology

[0002] Currently, special effects can be displayed to users in fields such as AR (Augmented Reality) virtual interaction and short video playback. For example, while a user is playing a video, pre-generated special effects images, such as flowers and rockets, can be added to the faces in the video. However, these special effects images are pre-generated and lack interaction with the user, resulting in low real-time correlation between the special effects and user behavior. Summary of the Invention

[0003] This disclosure provides a method, apparatus, device, and medium for drawing special effects images, overcoming the technical problems that special effects images are pre-generated, lack user interaction, and have low real-time correlation with user behavior.

[0004] In a first aspect, embodiments of this disclosure provide a method for drawing special effects images, including:

[0005] In response to a user-triggered special effects rendering request, the system performs trajectory tracking on the target video provided by the user to obtain a set of three-dimensional points corresponding to the motion trajectory of the target object in the target video.

[0006] The three-dimensional point set is meshed to obtain the target mesh structure;

[0007] The target mesh structure is textured to obtain a target special effects image;

[0008] The target effect image corresponding to the motion trajectory of the target object is displayed in the target video.

[0009] Secondly, embodiments of this disclosure provide a special effects image rendering apparatus, comprising:

[0010] The trajectory tracking unit is used to respond to a user-triggered special effects rendering request, perform trajectory tracking on the target video provided by the user, and obtain a set of three-dimensional points corresponding to the motion trajectory of the target object in the target video;

[0011] A mesh generation unit is used to perform meshing processing on the three-dimensional point set to obtain the target mesh structure;

[0012] The special effects generation unit is used to perform texture processing on the target mesh structure to obtain a target special effects image;

[0013] The special effects display unit is used to display the target special effects image corresponding to the motion trajectory of the target object in the target video.

[0014] Thirdly, embodiments of this disclosure provide an electronic device, including: a processor, a memory, and an output device;

[0015] The memory stores computer-executed instructions;

[0016] The processor executes computer execution instructions stored in the memory, such that the processor is configured with the special effects image rendering method described in the first aspect and various possible designs of the first aspect, and the output device is used to output a target video with target special effects images.

[0017] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the special effects image rendering method described in the first aspect and various possible designs of the first aspect.

[0018] The technical solution provided in this embodiment obtains a corresponding target video through user interaction. Trajectory tracking is performed on the target video to obtain a set of three-dimensional points corresponding to the motion trajectory of the target object in the video. The three-dimensional point set is then meshed to obtain a target mesh structure. After obtaining the target mesh structure, texturing processing can be performed on the target mesh structure to obtain a target special effects image. This target special effects image is then displayed corresponding to the motion trajectory of the target object, and subsequently, the target special effects image can be displayed corresponding to the motion trajectory of the video. This technical solution, by tracking the trajectory of the target object in the video, achieves the generation of a special effects mesh structure based on the motion trajectory, obtaining a target special effects image that matches the user's trajectory, and generating special effects based on the user's movement behavior. This expands the application scenarios of special effects and improves the efficiency of special effects usage. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 An example diagram illustrating an application of a special effects image drawing method provided in this disclosure embodiment;

[0021] Figure 2 A flowchart of one embodiment of a special effects image drawing method provided by this disclosure;

[0022] Figure 3 A flowchart of yet another embodiment of a special effects image drawing method provided in this disclosure;

[0023] Figure 4 An example diagram of resampling provided for an embodiment of this disclosure;

[0024] Figure 5 A sampling example diagram provided for an embodiment of this disclosure;

[0025] Figure 6 A flowchart of yet another embodiment of a special effects image drawing method provided in this disclosure;

[0026] Figure 7 An example diagram of the triangulation of a three-dimensional point set provided in an embodiment of this disclosure;

[0027] Figure 8 A flowchart of yet another embodiment of a special effects image drawing method provided in this disclosure;

[0028] Figure 9 An example diagram of a ring-shaped strip effect image provided in an embodiment of this disclosure;

[0029] Figure 10 This is a schematic diagram of the structure of one embodiment of a special effects image rendering apparatus provided in this disclosure.

[0030] Figure 11 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0032] The technical solution disclosed herein can be applied to augmented reality (AR) technology scenarios. By tracking the trajectory of a target object in a video, and performing mesh processing and texture processing based on the motion trajectory of the target object, a target special effects image can be obtained, thereby realizing the output of special effects corresponding to the trajectory and improving the real-time performance and accuracy of the special effects output.

[0033] In related technologies, in conventional special effects output scenarios, pre-generated special effects images are typically fused with image frames in a video, or special effects images are directly generated from areas of interest such as faces within the image frames. After generating image frames with special effects, the video with the effects can be displayed through output. However, the above methods of special effects processing add effects to image frames, which is largely unrelated to user behavior and lacks user interaction. This limits the application scenarios of special effects images and lacks real-time interaction between the effects and the user.

[0034] To address the aforementioned technical issues, this disclosure proposes trajectory tracking of target objects in a video, utilizing the tracked trajectory for special effects generation, and generating a 3D point set based on the motion trajectory of the target object in the video. By generating special effects images using these 3D point sets, a stereoscopic special effects image is obtained, enhancing the visual appeal of the special effects image.

[0035] This disclosure relates to the fields of computer, cloud computing, virtual reality and other technical fields, and specifically to a method, apparatus, device and medium for rendering special effects images.

[0036] In this disclosed technical solution, a target video is obtained through user interaction. The target video is then tracked to obtain a set of 3D points corresponding to the motion trajectory of the target object. This 3D point set is then meshed to obtain a target mesh structure. After obtaining the target mesh structure corresponding to the motion trajectory, a texture mapping process is performed on the target mesh structure to obtain a target special effects image. This target special effects image is then displayed on the motion trajectory of the target object, and subsequently, it can be displayed on the motion trajectory of the video. This technical solution, by tracking the trajectory of the target object in the video, achieves the generation of a mesh structure for special effects based on the motion trajectory, obtaining a target special effects image that matches the user's trajectory, and generating special effects based on the user's movement behavior. This expands the application scenarios of special effects and improves the efficiency of special effects usage.

[0037] The technical solutions of this disclosure and how they solve the aforementioned technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0038] Figure 1This diagram illustrates an application example of a special effects image rendering method provided in this disclosure. This method can be applied to an electronic device 1. The electronic device 1 may include, for example, an AR device. The electronic device 1 can detect special effects rendering requests triggered by a user. The electronic device 1 can be configured with the technical solution of this disclosure to respond to special effects rendering requests, acquire target special effects images, and display the target special effects images corresponding to the motion trajectory of the target object.

[0039] For example, assuming the user's motion trajectory in the target video is a curve 2, a moon 3 can be generated based on curve 2. The moon 3 can then be output at the curve 2 corresponding to the motion trajectory.

[0040] It should be noted that, Figure 1 In the application examples shown, the trajectory of the target object U changes as the target object moves. The target object U can remain stationary during trajectory tracking. The multiple target objects shown in each figure are only to illustrate the change in the trajectory of target object U and do not represent multiple trajectory objects. (Reference) Figure 1 During the movement of the target object U, a corresponding motion trajectory, namely curve 2, can be generated.

[0041] refer to Figure 2 , Figure 2 This flowchart illustrates one embodiment of a special effects image rendering method provided by this disclosure. The method can be configured as a special effects image rendering device, which can be located in an electronic device. The special effects image rendering method may include the following steps:

[0042] 201: In response to a user-triggered special effects rendering request, perform trajectory tracking on the target video provided by the user to obtain the 3D point set corresponding to the motion trajectory of the target object in the target video.

[0043] Optionally, the effects rendering request can refer to a URL (Uniform Resource Locator) request for effects rendering. Effects rendering controls can be set on the video playback page, and the effects rendering request is triggered when the user activates the controls. The video playback page can refer to the webpage playing the target video. The effects rendering request triggered by the user can be detected during the playback of the target video. The target video can be a live video being played on an electronic device.

[0044] The technical solution disclosed herein can also be applied to web page technology, enabling the playback of target videos via web pages and their players. Web pages can be written in programming languages ​​such as HTLM5 (HyperText Markup Language 5), Objective-C, and Java.

[0045] After the special effects rendering request is triggered, a target video selection page or dialog box can be provided, allowing users to select the target video. The system can receive target videos uploaded by users and can receive target videos from the user's device. This embodiment does not impose excessive limitations on the specific method of uploading the target video. The target video can include ordinary two-dimensional videos, as well as videos played through augmented reality devices.

[0046] The target object can include at least one of the following object types: moving objects, vehicles, pedestrians, or facial features of a person. During the tracking process, the target object can be any point on the target object. Taking a face as an example, the target object can refer to any easily identifiable key point on the face, such as the center of the forehead, the center of the eyes, or the tip of the nose.

[0047] Motion trajectories are generally obtained by sampling key points from image frames in a video, and connecting these key points to form a curve. Obtaining the 3D point set corresponding to the motion trajectory of a target object in a video specifically involves identifying the target object's location points (key points or trajectory points) from image frames containing the target object. By forming a motion trajectory from the key points of multiple image frames, secondary sampling can be performed on the motion trajectory to obtain sampling points with a higher density than the original trajectory points. These sampling points can then be used to determine the 3D point set.

[0048] 202: Mesh the 3D point set to obtain the target mesh structure.

[0049] Optionally, the target mesh structure may include a curved mesh structure or a ring-shaped strip mesh structure.

[0050] The target mesh structure can be a mesh surface determined according to the three-dimensional point set corresponding to the trajectory. It can include a surface with a normal surface structure or a toroidal surface formed by the trajectory contour.

[0051] 203: Apply texture mapping to the target mesh structure to obtain the target special effects image.

[0052] The target mesh structure does not have a texture set. You can paste a texture image into the target mesh structure. The target mesh structure with the pasted texture image will then have the target effect image.

[0053] Texture mapping of mesh structures can be achieved using texture mapping software.

[0054] 204: Displays a target effect image in the target video that corresponds to the motion trajectory of the target object.

[0055] Among them, the target effect image can be displayed at the position corresponding to the motion trajectory of the target object in the target video.

[0056] The target special effects image can be a three-dimensional image. It can be displayed through the output device of an electronic device. Specifically, when the electronic device displays the target video, the target special effects image can be displayed in association with the location of the target object's motion trajectory. Specifically, the center position of the motion trajectory can be used as the center position of the target special effects image, and the image can be displayed at that center position.

[0057] Electronic devices may include mobile phones, augmented reality devices, virtual reality devices, etc., and this embodiment does not impose too many limitations on the specific type of electronic device. Electronic devices can display three-dimensional target effect images. Of course, in practical applications, the target effect image can be converted into a two-dimensional effect image or effect point cloud through coordinate transformation to perform effect display.

[0058] In this embodiment, trajectory tracking is performed on the target video to obtain a set of three-dimensional points corresponding to the motion trajectory of the target object in the video. By meshing the three-dimensional point set, a target mesh structure is obtained. The generation of this target mesh structure enables the generation of the target mesh structure for the motion trajectory. After obtaining the target mesh structure corresponding to the motion trajectory, texture mapping can be performed on the target mesh structure to obtain a target special effects image. This target special effects image is then displayed on the motion trajectory of the target object, and subsequently, it can be displayed on the motion trajectory of the video. This technical solution, by tracking the trajectory of the target object in the video, achieves the generation of a mesh structure for special effects based on the motion trajectory, obtaining a target special effects image that matches the user's trajectory, and generating special effects that correspond to the user's movement behavior. This expands the application scenarios of special effects and improves the efficiency of special effects usage.

[0059] like Figure 3 The diagram shown is a flowchart of yet another embodiment of a special effects image drawing method provided in this disclosure. Figure 2 The difference in the illustrated embodiment lies in that trajectory tracking is performed on the target video provided by the user to obtain a set of three-dimensional target points corresponding to the motion trajectory of the target object in the target video, including:

[0060] 301: Identify target objects in the target video.

[0061] Optionally, a target tracking algorithm can be used to identify target objects in the target video.

[0062] After target object recognition, key points of the target object can be determined. Taking a face as an example, any point on the face can be used as a key point, such as the tip of the nose or the center of the eyes.

[0063] 302: Sample the motion trajectory of the target object in the target video to obtain at least one target sampling point.

[0064] 303: Map at least one target sampling point to a three-dimensional spatial coordinate system to obtain a three-dimensional point set.

[0065] Mapping the target sampling points to a three-dimensional spatial coordinate system can include mapping the target sampling points from a two-dimensional spatial coordinate system to a three-dimensional spatial coordinate system.

[0066] In this embodiment of the disclosure, a target object in a target video can be identified, and the motion trajectory of the target object can be sampled to obtain a two-dimensional point set. By mapping the two-dimensional point set to a three-dimensional spatial coordinate system, a three-dimensional point set is obtained. Through trajectory sampling and spatial mapping, accurate extraction of the three-dimensional point set corresponding to the motion trajectory can be achieved.

[0067] As one example, sampling the motion trajectory of the target object in the target video to obtain at least one target sampling point may include:

[0068] The motion trajectory of the target object is sampled from the target video to obtain a two-dimensional point set; the two-dimensional point set includes multiple first sampling points;

[0069] Resampling of the two-dimensional point set yields multiple second sampling points;

[0070] Determine at least one target sampling point from a plurality of second sampling points that satisfies the trajectory smoothing condition.

[0071] Optionally, resampling a two-dimensional point set to obtain multiple second sampling points may include: sampling between pairs of adjacent sampling points among multiple first sampling points in the two-dimensional point set to obtain second sampling points. Multiple second sampling points can be obtained when the two-dimensional point set resampling is completed.

[0072] For ease of understanding, such as Figure 4 The resampling example shown assumes that the first sampling points collected in the target object's motion trajectory are 401 to 404, as referenced. Figure 4 Since the distance between two adjacent sampling points is relatively large, directly drawing effects based on these sampling points may lead to significant contour errors. To avoid this, the technical solution disclosed in this paper employs resampling. That is, the fitted curve based on the first sampling point can be segmented, and the segmented curves can be resampled. The sampling points obtained from the resampling are the second sampling points. (Reference) Figure 4Piecewise curve sampling corresponding to the first sampling points 401 and 402 yields the second sampling point 405; piecewise curve sampling corresponding to the first sampling points 402 and 403 yields the second sampling point 406; and piecewise curve sampling corresponding to the first sampling points 403 and 404 yields three second sampling points 407. Of course, directly fitting adjacent sampling points to a curve is merely exemplary and should not constitute a limitation on the technical solution of this disclosure. Alternatively, entire curve fitting and segmentation can be used to resample the piecewise curves, as described in the following embodiments.

[0073] The first sampling point can be a key point obtained from the target video. The second sampling point can be a sampling point obtained by resampling based on the first sampling point, and the second sampling point can include the first sampling point.

[0074] The trajectory smoothing condition can refer to the distance between the sampling point and the trajectory being less than a distance threshold or being located on the trajectory. By using the trajectory smoothing condition, sampling points that differ significantly from the trajectory can be removed, making the trajectory formed by at least one target sampling point smoother, avoiding the appearance of sharp points, and improving the accuracy of the trajectory.

[0075] In this embodiment, after sampling the motion trajectory of the target object in the target video, a two-dimensional point set consisting of multiple first sampling points is obtained. The two-dimensional point set is then resampled to obtain denser second sampling points; resampling enhances the density of the sampling points. After obtaining the second sampling points, at least one target sampling point satisfying the trajectory smoothing condition can be determined from them. The second sampling points are then optimized using the trajectory smoothing condition to make the trajectory corresponding to the target sampling point smoother, thereby improving the acquisition accuracy and precision of the target sampling points.

[0076] In some embodiments, sampling the motion trajectory of the target object from the target video to obtain a two-dimensional point set includes:

[0077] Based on a preset sampling frequency, the trajectory points of the target object are collected from the image frames of the target video;

[0078] Based on a preset distance threshold, determine whether the trajectory points meet the distance constraint conditions;

[0079] If so, then determine the first sampling point based on the trajectory points;

[0080] If not, replace the trajectory point with the last first sampling point in the two-dimensional point set;

[0081] Obtain the two-dimensional point set corresponding to the multiple first sampling points obtained at the end of the target video acquisition.

[0082] Optionally, acquiring trajectory points of the target object from image frames of the target video according to a preset sampling frequency may include: performing image sampling from the target video according to the preset sampling frequency to obtain image frames, identifying key points of the target object from the image frames, and determining the key points of the target object as trajectory points.

[0083] There can be multiple trajectory points, which can be acquired from multiple image frames.

[0084] Optionally, the sampling frequency can be preset, specifically based on the sampling time interval. The product of the sampling time interval and the sampling frequency can be 1, and they are reciprocals of each other.

[0085] Optionally, replacing the trajectory point with the last first sampling point in the two-dimensional point set includes: deleting the existing last first sampling point in the two-dimensional point set and using the trajectory point as the new last first sampling point in the two-dimensional point set.

[0086] In this embodiment of the disclosure, when sampling the motion trajectory of the target object, trajectory points can be initially obtained based on the sampling of image frames in the target video. By judging whether the trajectory points meet the distance constraint conditions, the trajectory points collected from different image frames are filtered in more detail under the constraint conditions, thereby improving the sampling accuracy of the first sampling point.

[0087] In one possible design, based on a preset distance threshold, it is determined whether the trajectory points satisfy the distance constraint conditions, including:

[0088] Determine the first distance between the trajectory point and the last first sampling point in the two-dimensional point set;

[0089] If the first distance is greater than the distance threshold, then the trajectory point is determined to satisfy the distance constraint condition;

[0090] If the first distance is less than or equal to the distance threshold, then the trajectory point is determined not to meet the distance constraint condition.

[0091] Determining the first distance between a trajectory point and the last first sampling point in the 2D point set can include: calculating the pixel coordinate distance based on the pixel coordinates of the trajectory point and the last first sampling point in the 2D point set, and then determining the first distance based on the pixel coordinate distance. The unit of the pixel coordinate distance can be different from or the same as the unit of the first distance. In different situations, the two distance units can be converted to achieve distance comparison in the same unit, thus achieving accurate distance comparison.

[0092] The distance threshold can be used to limit the distance between two adjacent trajectory points.

[0093] In this embodiment of the disclosure, a first distance is determined between the trajectory point and the last first sampling point in the two-dimensional point set. The trajectory point in the two-dimensional point set is constrained by a distance threshold to avoid the situation where the distance to the collected sampling point is too abnormal, thereby achieving accurate extraction of the sampling point.

[0094] If the trajectory points meet the distance constraint conditions, further sampling can be performed on the trajectory points.

[0095] Therefore, in one possible design, determining the first sampling point based on the trajectory points includes:

[0096] Get the number of sampling points of the first existing sampling point in the two-dimensional point set.

[0097] If the number of sampling points is determined to be less than or equal to the sampling number threshold, then the trajectory point is added as the last first sampling point in the two-dimensional point set;

[0098] If the number of sampling points is greater than the sampling number threshold, the mean point is obtained by weighted summation of the first N first sampling points and trajectory points in the two-dimensional point set; the mean point is added as the last first sampling point in the two-dimensional point set; N is a positive integer greater than 0.

[0099] If two adjacent points are too close, a closure threshold judgment may be triggered. That is, if the distance between the current trajectory point and any previous point is less than the distance threshold, the trajectory can be considered closed, leading to invalid sampling or sampling stopping. Therefore, after the distance judgment, the number of sampling points can be judged. By constraining the number of sampling points, weighted sampling can be performed only after the number of currently drawn sampling points reaches the sampling point threshold. This ensures that the sampling process is subject to both the distance threshold and the sampling number threshold, improving sampling accuracy.

[0100] The mean point is obtained by weighted summation of the first N sampling points and trajectory points in the two-dimensional point set. This allows for some control over the trajectory point drawing process, ensuring that the final sampling points are not limited to the trajectory points. This avoids the problem of uneven curves caused by large fluctuations in the actual trajectory points. Weighted summation makes the collected sampling points smoother and more accurate.

[0101] In this embodiment of the disclosure, the number of sampling points is determined using a sampling quantity threshold, which can confirm whether the trajectory points can be directly used. By using a weighted summation method, the sampling point and the sampling points before it can be weighted and summed, resulting in higher accuracy for the first sampling point and accurate sampling of the first sampling point.

[0102] As another alternative implementation, the two-dimensional point set is resampled to obtain multiple second sampling points, including:

[0103] Based on the curve fitting algorithm, curve fitting is performed on multiple first sampling points in the two-dimensional point set to obtain the first sample line;

[0104] Curve fitting is performed on the first and last sampling points in the two-dimensional point set to obtain the second spline.

[0105] A closed curve is obtained by splicing the first and second splines;

[0106] Based on the segmented interval sampling strategy, the closed curve is resampled to obtain multiple second sampling points.

[0107] Optionally, the curve fitting algorithm may include any fitting algorithm such as the Bézier curve algorithm or the Catmull-Rom curve algorithm.

[0108] To obtain a second spline by curve fitting between the first and last sampling points in a two-dimensional point set, the following methods can be used: First, based on the average curvature of the first spline, perform curve fitting between the first and last sampling points in the two-dimensional point set to obtain the second spline. Alternatively, the first and last sampling points in the two-dimensional point set can be directly connected to form a line segment, thus obtaining the second spline corresponding to that line segment.

[0109] For ease of understanding, such as Figure 5 The sampling example diagram shown shows that the first spline 503 obtained by fitting the sampling points between the first sampling point 501 and the last sampling point 502 is not closed. The second spline 504 can be obtained by curve fitting. The first spline 503 and the second spline 504 can be spliced ​​together to form a closed curve.

[0110] In this embodiment, curve fitting is performed on multiple first sampling points in a two-dimensional point set to obtain a first spline. To obtain a closed curve, curve fitting is performed on the first and last sampling points in the two-dimensional point set to obtain a second spline. A closed curve is obtained by splicing the first and second splines. The closed curve corresponds to the motion trajectory; segmenting the closed curve allows for the acquisition of multiple more detailed second sampling points.

[0111] In one possible design, the closed curve is resampled according to a segmented interval sampling strategy to obtain multiple second sampling points, including:

[0112] Determine the number of segments corresponding to the segmented interval sampling strategy;

[0113] The closed curve is segmented according to the number of segments to obtain at least one segmented curve.

[0114] Collect second sampling points from the piecewise curve to obtain multiple second sampling points corresponding to at least one piecewise curve.

[0115] Optionally, acquiring second sampling points from the piecewise curve may include determining at least one curve sampling point from the piecewise curve, and designating the at least one curve sampling point as a second sampling point. Alternatively, the original first sampling point may be designated as the second sampling point. Multiple second sampling points may include multiple first sampling points from a two-dimensional point set.

[0116] Optionally, segmenting a closed curve according to the number of segments may include: determining the segment length of the closed curve based on the number of segments, and segmenting the closed curve into at least one segmented curve according to the segment length and the number of segments. The number of curves in the at least one segmented curve may be equal to the number of segments.

[0117] In this embodiment of the disclosure, the closed curve is segmented to obtain at least one segmented curve. The segmented curve is a line segment. Sampling from the segmented curve can make multiple second sampling points belong to the same line segment, resulting in higher accuracy.

[0118] As an optional implementation, determining the number of segments corresponding to the segmented interval sampling strategy may include:

[0119] Determine the maximum number of segments, the maximum number of input points, and the minimum segment value corresponding to the spline interpolation algorithm;

[0120] The number of segments corresponding to the segmented interval sampling strategy is determined based on the maximum number of segments, the maximum number of input points, and the minimum number of segments.

[0121] Among them, parameters such as the maximum number of segments, the maximum number of input points, and the minimum number of segments can be set.

[0122] In this embodiment of the disclosure, the accurate number of segments is obtained based on the constraints of the maximum number of segments, the maximum number of input points, and the minimum number of segments.

[0123] As another optional implementation, determining the number of segments corresponding to the segmented interval sampling strategy may include:

[0124] The curve length is determined based on the sampling frequency and the number of samplings.

[0125] The curve length is used as the number of sampling intervals;

[0126] Calculate the number of segments based on the number of sampling intervals and the curve length.

[0127] In this embodiment of the disclosure, the number of segments is calculated using a determined curve length, which enables the number of segments to be matched with the curve length in real time, thereby achieving dynamic adjustment of the number of segments and obtaining an accurate number of segments.

[0128] As one embodiment, at least one target sampling point is mapped to a three-dimensional spatial coordinate system to obtain a three-dimensional point set, including:

[0129] Based on the camera transformation matrix corresponding to the camera rendering target video, the target sampling points are transformed into a three-dimensional spatial coordinate system to obtain the three-dimensional transformed coordinates corresponding to each target sampling point.

[0130] Obtain a three-dimensional point set consisting of the three-dimensional transformed coordinates corresponding to each target sampling point.

[0131] Optionally, the camera transformation matrix can be obtained by matrix calculation using the camera parameters, which may include camera intrinsic parameters and camera extrinsic parameters.

[0132] The target sampling point can be a point in a two-dimensional image coordinate system. It can be transformed into a three-dimensional spatial coordinate system using data such as the camera transformation matrix, thus obtaining the three-dimensional transformed coordinates. For details on the transformation between image coordinate systems, camera coordinate systems, and spatial coordinate systems, please refer to the descriptions of relevant techniques.

[0133] In this embodiment of the disclosure, the target sampling points can be transformed into a three-dimensional spatial coordinate system based on the depth information of each target sampling point and the camera transformation distance of the camera coordinate system, thereby achieving accurate transformation of the pixel points in the three-dimensional spatial coordinate system.

[0134] After obtaining multiple second sampling points, it can be confirmed whether the multiple second sampling points are prominent.

[0135] In one possible design, determining at least one target sampling point that satisfies the trajectory smoothing condition from a plurality of second sampling points includes:

[0136] Combine two adjacent sampling points from multiple second sampling points to obtain at least one set of sampling points;

[0137] Determine the tangent vectors corresponding to the two second sampling points in each group of sampling points;

[0138] For at least one set of sampling points, if the tangent vector of the target set of sampling points is less than zero, then any second sampling point among the two second sampling points in the target set is determined as the target sampling point;

[0139] If the tangent vector of the target group sampling point is greater than or equal to zero, then both second sampling points in the target group are determined to be target sampling points.

[0140] At least one set of sampling points is obtained, and all target sampling points are at least one target sampling point.

[0141] The calculation steps for the tangent vectors of the two second sampling points may include arc fitting based on the two sampling points, and using the fitted arc to calculate the tangent vectors.

[0142] In this embodiment, for the second sampling point, adjacent sampling points can be grouped into a single group. The tangent vector of each group is then checked to see if it is greater than zero. If the tangent vector is greater than zero, it indicates that the two second sampling points in the group are relatively similar, and both can be retained. If the tangent vector is less than zero, it indicates that the two second sampling points in the group are significantly different, with one sampling point being more prominent and potentially affecting the display of special effects. Therefore, only one of the second sampling points needs to be retained. By judging the tangent vector, prominent points among adjacent sampling points can be processed, improving the processing efficiency of sampling points.

[0143] Of course, to improve the efficiency of screening smoothing conditions, in another possible design, at least one target sampling point satisfying the trajectory smoothing condition is determined from the second sampling point, including:

[0144] Determine whether the second sampling point belongs to a point on the smooth curve;

[0145] If so, then the second sampling point is determined to satisfy the trajectory smoothing condition;

[0146] If not, it is determined that the second sampling point does not meet the trajectory smoothing condition, and the process returns to the step of determining whether the second sampling point belongs to a point on the smooth curve based on the edge sampling algorithm.

[0147] When the second sampling point traversal ends, the target sampling point that satisfies the trajectory smoothing condition is obtained.

[0148] Optionally, determining whether a second sampling point belongs to a point on a smooth curve may include: performing curve fitting on the second sampling point to obtain a fitted curve, calculating the absolute distance from each second sampling point to the fitted curve, and if the absolute distance is greater than an absolute threshold, then the second sampling point is determined to meet the trajectory smoothing condition; if the absolute distance is less than or equal to the absolute threshold, then the second sampling point is determined not to meet the trajectory smoothing condition.

[0149] In this embodiment of the disclosure, the edge average sampling algorithm can be used to determine whether the second sampling point belongs to a point on the smooth curve. By judging the smooth curve, it is possible to detect whether the sampling point is a relatively prominent cusp, so that the target sampling point is a point along the smooth curve. Thus, when generating special effects based on at least one target sampling point, the contour of the special effects image is smoother and there are no cusp parts, thereby obtaining at least one target sampling point with a smoother contour.

[0150] like Figure 6The flowchart shown is for another embodiment of a special effects image drawing method provided by this disclosure. The difference between this special effects image drawing method and the previous embodiments lies in that it performs meshing processing on the three-dimensional point set to obtain a target mesh structure, including:

[0151] 601: The entire region contained in the three-dimensional point set is triangularly divided to obtain a triangular mesh surface;

[0152] 602: Determine the triangular mesh surface as the target mesh structure corresponding to the surface effect type.

[0153] The target mesh structure is textured to obtain the target effect image, including:

[0154] 603: Perform texture mapping on a 3D mesh surface to obtain the target texture image for 3D rendering;

[0155] 604: Perform edge smoothing on the target texture image of the 3D rendering to obtain the target special effects image.

[0156] Alternatively, a triangular mesh surface can be obtained by connecting multiple points in a three-dimensional point set to form a mash.

[0157] Edge smoothing of a target texture image in 3D rendering can refer to smoothing the mesh edges of the target texture image in 3D rendering.

[0158] In this embodiment, a triangular mesh surface can be generated from a 3D point set according to the network topology. This triangular mesh surface is then used as the target mesh structure for mapping to obtain a 3D texture map. Through network processing, an accurate triangular mesh surface can be obtained. After obtaining the 3D rendered target texture image, edge smoothing processing can be performed on the 3D rendered target texture image to obtain the target special effects image. Edge smoothing processing makes the target special effects image smoother and improves its display effect.

[0159] As one example, the entire region contained in the three-dimensional point set is triangulated to obtain a triangular mesh surface, including:

[0160] Determine the center point corresponding to the 3D point set;

[0161] Based on the 3D point set and center point, a sector mesh topology is constructed to obtain the 3D topological structure;

[0162] Transform the boundary values ​​of the 3D topology to the calculated values ​​corresponding to the percentage coordinates of the image;

[0163] Based on the three-dimensional topology and the calculated values ​​of each point on the three-dimensional topology, the triangular mesh surface is determined.

[0164] Optionally, determining the center point corresponding to the three-dimensional point set may include the following implementation methods:

[0165] Implementation Method 1: Solve for the internal centroid of the convex polygon corresponding to the 3D point set to obtain the center point.

[0166] Implementation Method 2: Perform circular fitting on the contour of the three-dimensional point set to obtain the center point of the fitted circle.

[0167] Implementation Method 3: Perform Delaunay triangulation on the 3D point set. If a point in the 3D point set is not inside any of the triangular parts, it means that the center / centroid of the circle is outside the convex polygon corresponding to the 3D point set. The mean of all points in the 3D point set can be calculated as the center point.

[0168] A convex polygon can be a polygonal shape formed by connecting the edge points of a three-dimensional point set.

[0169] For ease of understanding, Figure 7 An example diagram of the triangulation of a 3D point set is shown. Mapping the 3D point set to a planar diagram is referenced. Figure 7 The point in the 3D point set is 701, and the center point is 702. Points 701 and the center point 702 of any two adjacent 3D point sets can be connected pairwise to form a triangle, resulting in the corresponding 3D topological structure 703. Of course, Figure 7 The 3D point set shown is merely exemplary; in practical applications, the 3D point set can change with the motion trajectory. The 3D topology 703 can be mapped to the UV coordinate system to obtain the UV values ​​of each point, thereby obtaining a triangular mesh surface.

[0170] Boundary values ​​of a 3D topology can refer to the size of the boundary of the 3D topology, such as its length and width. The length and width are actually calibration values.

[0171] The percentage coordinates of an image can refer to the UV coordinate system. When mapping a 3D topology, to ensure a closer fit between the image and the 3D topology and to avoid over-fitting, the boundary values ​​of the 3D topology can be converted into calculated values ​​in the UV coordinate system. The UV coordinates are the percentage coordinates of the image, with U for the horizontal direction and V for the vertical direction. For example, suppose the 3D topology includes four boundary points: top-left, top-right, bottom-left, and bottom-right. We can determine that the top-left point has U=0 and V=0, the bottom-right point has U=1 and V=1, the top-right point has U=1 and V=0, and the bottom-left point has U=0 and V=1. Specifically, these can be the calculated UV coordinate values ​​of the four corners of the 3D topology, while the UV value of the center point of the 3D topology can be U=0.5 and V=0.5.

[0172] When applying textures to a 3D mesh surface, the texture can be applied using the calculated values ​​of each point, i.e., the UV values.

[0173] In this embodiment, when generating a triangular mesh surface, a sector network topology can be constructed based on the center points of a 3D point set to obtain a 3D topological structure. By converting the boundary values ​​of the 3D topological structure into UV calculation values, the 3D topological structure can be represented in the UV coordinate system, resulting in a triangular mesh surface with a coordinate structure. Accurate triangular mesh surfaces can be obtained through surface construction and coordinate transformation, ensuring the accurate execution of subsequent special effects mapping.

[0174] In one possible design, the target texture image of the 3D rendering is subjected to edge smoothing to obtain the target special effects image, including:

[0175] Each pixel of the target texture image in the 3D rendering is rendered into a rendering texture with an initial value of zero to obtain the first texture map.

[0176] The alpha channel of the first texture map is drawn onto the grayscale rendering texture to obtain a grayscale mask image;

[0177] The grayscale mask image is subjected to erosion and blurring processing to obtain a smooth mask image;

[0178] The first texture map is image fusion processed using a smooth mask image to obtain the target special effects image.

[0179] Optionally, the texture image and the rendered texture may include four channels: R (Red), G (Green), B (Blue), and A (alpha), and each channel may have a corresponding channel value at the pixel.

[0180] Optionally, the channel values ​​of each channel of the render texture (RT) can all be 0, that is, the initial value of each pixel is (0,0,0,0). When the channel values ​​of each pixel are all 0, the background of the render texture is transparent.

[0181] A grayscale rendering texture refers to a rendering texture where the pixel values ​​are grayscale values ​​between 0 and 1. The alpha channel value can be used to convert the color value of the grayscale rendering texture. For example, the color value corresponding to an alpha channel value of 0 is (0,0,0), which is pure black; the color value corresponding to an alpha channel value of 1 is (1,1,1); and alpha channel values ​​in the range of 0-1 correspond to grayscale color values ​​of (alpha,alpha,alpha).

[0182] A grayscale mask can be used to mask the first texture map, so that the outline of the target effect image is affected by the grayscale mask and transformed into a polygonal outline.

[0183] Optionally, performing erosion and blurring on the grayscale mask to obtain a smooth mask may include eroding the grayscale mask to obtain an eroded mask, and then blurring the eroded mask to obtain a smooth mask. Blurring the eroded mask may include blurring the eroded mask based on a Gaussian function.

[0184] In this embodiment, a first texture map is obtained by drawing the target texture image of the 3D rendering onto a rendering texture with an initial value of zero. A grayscale mask image is obtained by drawing the alpha channel value of the first texture map onto a grayscale rendering texture. Erosion and blurring processing of the grayscale mask image can yield a smoother smooth mask image. Image fusion processing of the first texture map using the smooth mask image can smooth the image contours, resulting in a target effect image with smoother contours.

[0185] like Figure 8 The flowchart shown is for another embodiment of a special effects image drawing method provided by this disclosure. The difference between this special effects image drawing method and the previous embodiments lies in that it performs meshing processing on the three-dimensional point set to obtain a target mesh structure, including:

[0186] 801: Interpolate the pairwise adjacent sampling points in the three-dimensional point set to obtain the target point set.

[0187] Optionally, interpolation can be performed on two adjacent sampling points in the three-dimensional point set. This can refer to interpolating between two adjacent sampling points in the three-dimensional point set until the sampling of two adjacent sampling points in the three-dimensional point set is completed.

[0188] One approach is to use interpolation based on the Catmull Curve to obtain a denser set of target points.

[0189] 802: Generate a strip network for the contour region corresponding to the target point set to obtain the target mesh structure;

[0190] The target mesh structure is textured to obtain the target effect image, including:

[0191] 803: Render the trajectory of the target mesh structure to obtain a 3D bar trajectory map;

[0192] 804: Determine the target special effects image based on the 3D bar trajectory diagram.

[0193] For ease of understanding, Figure 9 An example image of a ring-shaped bar effect is shown. The three-dimensional bar trajectory diagram 901 can be used to determine the target effect image, demonstrating a ring-shaped effect.

[0194] Optionally, the target effect image can have semi-transparent properties. The rendering transparency can be preset according to usage requirements, and can be a value greater than 0 and less than or equal to 1, or a decimal between 0 and 1.

[0195] In this embodiment, a striped network corresponding to a set of target points can be generated, resulting in a striped target network structure and enabling the generation of a striped mesh. By applying texture mapping to the striped target mesh structure, a three-dimensional striped trajectory map can be obtained. This three-dimensional striped trajectory map can then be used to determine the target special effects image. By establishing a striped target mesh structure using the mesh, the accurate generation of a striped target special effects image is achieved.

[0196] As one example, determining the target special effects image based on the three-dimensional bar trajectory includes:

[0197] The three-dimensional strip trajectory is rendered onto a rendering texture with an initial value of zero to obtain a second texture map;

[0198] For the second texture map, the trajectory blending image is made transparent according to the rendering transparency to obtain the target special effect image.

[0199] Optionally, the second texture map can be a rendered image displaying a three-dimensional bar trajectory.

[0200] In this embodiment of the disclosure, RT can be used to smooth the edges of the three-dimensional bar trajectory, resulting in a smoother bar shape and better display effect in the target special effect image.

[0201] One possible design also includes:

[0202] Determine the trigger time for user-triggered effect rendering requests;

[0203] Based on the time periods corresponding to at least one festival type, determine the target festival type corresponding to the trigger time;

[0204] Obtain the texture image pre-associated with the target holiday type;

[0205] The target mesh structure is textured to obtain the target effect image, including:

[0206] Texture images are used to apply textures to the target mesh structure to obtain the target special effects image.

[0207] Optionally, at least one festival type can be set to be available, and the time period corresponding to each festival type is known. For example, the time period corresponding to the Mid-Autumn Festival is known. Texture images can be pre-associated for each festival type. For example, the texture image for the Mid-Autumn Festival can be a moon texture image, and the texture image for the Dragon Boat Festival can be a zongzi leaf texture image.

[0208] In this embodiment, the target holiday type is determined by the trigger time of the special effects rendering request, and the texture image corresponding to the target holiday type is used for mapping, so that the texture of the target special effects image matches the target holiday type in real time, the applicable time is longer, and higher display efficiency can be obtained.

[0209] The technical solution disclosed herein can be applied to a variety of practical application scenarios. The specific application scenarios of this disclosure will be described in detail below.

[0210] Application Scenario 1: In traditional holiday scenarios, such as Dragon Boat Festival and Mid-Autumn Festival, when users initiate special effects rendering requests, texture images adapted to the holidays can be obtained based on the technical solution disclosed herein. The target mesh structure obtained based on some steps of this disclosure can be processed by mapping, such as mapping the moon texture, to obtain a special effects moon, which can be displayed in the corresponding motion trajectory of the video.

[0211] Application Scenario 2: In augmented reality scenarios, based on the technical solution disclosed herein, special effects images can be generated for the tracked vehicle or person, and the generated special effects images are three-dimensional special effects images. These three-dimensional special effects images can be displayed in augmented reality devices to achieve augmented reality and improve the utilization rate of special effects.

[0212] Furthermore, the technical solutions disclosed herein can also be applied to video playback and augmented reality, specifically including scene-based special effects for special holidays and user trajectory tracking effects.

[0213] like Figure 10 The diagram shown is a structural schematic of one embodiment of a special effects image drawing device provided in this disclosure. This device can be located in an electronic device and can be configured with the aforementioned special effects image drawing method. The special effects image drawing device 1000 may include:

[0214] Tracking unit 1001: In response to a user-triggered special effects rendering request, it performs trajectory tracking on the target video provided by the user to obtain the three-dimensional point set corresponding to the motion trajectory of the target object in the target video.

[0215] Mesh generation unit 1002: Used to mesh a 3D point set to obtain the target mesh structure.

[0216] Special effects generation unit 1003: Used to perform texture processing on the target mesh structure to obtain the target special effects image.

[0217] Special effects display unit 1004: used to display target special effects images corresponding to the motion trajectory of the target object in the target video.

[0218] As one embodiment, the trajectory tracking unit includes:

[0219] The object recognition module is used to identify target objects in the target video;

[0220] The trajectory sampling module is used to sample the motion trajectory of the target object in the target video to obtain at least one target sampling point;

[0221] The spatial mapping module is used to map at least one target sampling point to a three-dimensional spatial coordinate system to obtain a three-dimensional point set.

[0222] In some embodiments, the trajectory sampling module includes:

[0223] The trajectory sampling submodule is used to sample the motion trajectory of the target object from the target video to obtain a two-dimensional point set; the two-dimensional point set includes multiple first sampling points;

[0224] The resampling submodule is used to resample the two-dimensional point set to obtain multiple second sampling points;

[0225] The sampling selection submodule is used to determine at least one target sampling point that satisfies the trajectory smoothing condition from a plurality of second sampling points.

[0226] In one possible design, the trajectory sampling submodule is specifically used for:

[0227] Based on a preset sampling frequency, the trajectory points of the target object are collected from the image frames of the target video;

[0228] Based on a preset distance threshold, determine whether the trajectory points meet the distance constraint conditions;

[0229] If so, then determine the first sampling point based on the trajectory points;

[0230] If not, replace the trajectory point with the last first sampling point in the two-dimensional point set;

[0231] Obtain the two-dimensional point set corresponding to the multiple first sampling points obtained at the end of the target video acquisition.

[0232] In one possible design, the trajectory sampling submodule is specifically used for:

[0233] Determine the first distance between the trajectory point and the last first sampling point in the two-dimensional point set;

[0234] If the first distance is greater than the distance threshold, then the trajectory point is determined to satisfy the distance constraint condition;

[0235] If the first distance is less than or equal to the distance threshold, then the trajectory point is determined not to meet the distance constraint condition.

[0236] In one possible design, the trajectory sampling submodule is specifically used for:

[0237] Obtain the number of sampling points of the first existing sampling point in the two-dimensional point set;

[0238] If the number of sampling points is determined to be less than or equal to the sampling number threshold, then the trajectory point is added as the last first sampling point in the two-dimensional point set;

[0239] If the number of sampling points is greater than the sampling number threshold, the mean point is obtained by weighted summation of the first N first sampling points and trajectory points in the two-dimensional point set; the mean point is added as the last first sampling point in the two-dimensional point set; N is a positive integer greater than 0.

[0240] In some embodiments, the resampling submodule is specifically used for:

[0241] Based on the curve fitting algorithm, curve fitting is performed on multiple first sampling points in the two-dimensional point set to obtain the first sample line;

[0242] Curve fitting is performed on the first and last sampling points in the two-dimensional point set to obtain the second spline.

[0243] A closed curve is obtained by splicing the first and second splines;

[0244] Based on the segmented interval sampling strategy, the closed curve is resampled to obtain multiple second sampling points.

[0245] In some embodiments, the resampling submodule is specifically used for:

[0246] Determine the number of segments corresponding to the segmented interval sampling strategy;

[0247] The closed curve is segmented according to the number of segments to obtain at least one segmented curve.

[0248] Collect second sampling points from the piecewise curve to obtain multiple second sampling points corresponding to at least one piecewise curve.

[0249] As one example, the sampling selection submodule can be specifically used for:

[0250] Combine two adjacent sampling points from multiple second sampling points to obtain at least one set of sampling points;

[0251] Determine the dot product result between the two second sampling points in each group of sampling points. The dot product result is obtained by calculating the dot product of the tangent vectors of the two second sampling points.

[0252] For at least one set of sampling points, if the dot product of the two second sampling points in the target set is less than zero, then any second sampling point among the two second sampling points in the target set is determined as the target sampling point.

[0253] If the dot product of the two second sampling points in the target group is greater than or equal to zero, then both second sampling points in the target group are determined to be target sampling points.

[0254] At least one set of sampling points is obtained, and all target sampling points are at least one target sampling point.

[0255] In some embodiments, the space mapping module may include:

[0256] The matrix transformation submodule is used to transform the target sampling points into a spatial coordinate system based on the camera transformation matrix corresponding to the camera rendering the target video, and obtain the three-dimensional transformed coordinates corresponding to each target sampling point.

[0257] The point set determination submodule is used to obtain the three-dimensional point set composed of the three-dimensional transformed coordinates corresponding to each target sampling point.

[0258] As another embodiment, the mesh generation unit may include:

[0259] The first processing module is used to triangulate the entire region contained in the three-dimensional point set to obtain a triangular mesh surface;

[0260] The first determination module is used to determine the target mesh structure corresponding to the surface effect type of the triangular mesh surface.

[0261] Special effects generation unit, including:

[0262] The first mapping module is used to perform texture mapping on the 3D mesh surface to obtain the target texture image for 3D rendering.

[0263] The edge smoothing module is used to smooth the edges of the target texture image in 3D rendering to obtain the target special effects image.

[0264] In some embodiments, the first processing module includes:

[0265] The center determination submodule is used to determine the center point corresponding to a 3D point set;

[0266] The topology construction submodule is used to construct a sector mesh topology based on a 3D point set and center point to obtain a 3D topological structure.

[0267] The coordinate calculation submodule is used to calculate the boundary values ​​of the 3D topology and transform them to the percentage coordinate system of the image.

[0268] The surface determination submodule is used to determine the triangular mesh surface based on the three-dimensional topology and the calculated values ​​of each point on the three-dimensional topology.

[0269] As one embodiment, the edge smoothing module includes:

[0270] The first rendering submodule is used to render each pixel of the target texture image of the 3D rendering to a rendering texture where each channel value is zero, so as to obtain the first texture map.

[0271] The second rendering submodule is used to draw the alpha channel of the first texture map onto the grayscale rendering texture to obtain a grayscale mask map.

[0272] The erosion and blurring submodule is used to perform erosion and blurring processing on the grayscale mask image to obtain a smooth mask image;

[0273] The image fusion submodule is used to perform image fusion processing on the smooth mask image and the first texture map to obtain the target special effect image.

[0274] As another embodiment, the mesh generation unit includes:

[0275] The point set interpolation module is used to interpolate the pairwise adjacent sampling points in a 3D point set to obtain the target point set;

[0276] The mesh generation module is used to generate a strip network for the contour region corresponding to the target point set to obtain the target mesh structure.

[0277] Special effects generation unit, including:

[0278] The trajectory rendering module is used to render the trajectory of the target mesh structure and obtain a three-dimensional bar trajectory map.

[0279] The trajectory processing module is used to determine the target special effects image based on the 3D bar trajectory map.

[0280] In one possible design, the trajectory processing module includes:

[0281] The third rendering submodule is used to render the three-dimensional bar trajectory to a rendering texture in which all channel values ​​are zero, and obtain the second texture map.

[0282] The transparency processing submodule is used to process the second texture map to be transparent according to the rendering transparency, so as to obtain the target special effect image.

[0283] One possible design also includes:

[0284] The time determination unit is used to determine the trigger time of the special effect rendering request initiated by the user;

[0285] The festival determination unit is used to determine the target festival type corresponding to the trigger time based on the time period corresponding to at least one festival type.

[0286] Texture association unit, used to obtain the texture image pre-associated for the target holiday type;

[0287] Special effects generation unit, including:

[0288] The second mapping module is used to apply texture images to the target mesh structure to obtain target special effects images.

[0289] The apparatus provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0290] To implement the above embodiments, this disclosure also provides an electronic device.

[0291] refer to Figure 11 The diagram illustrates a structural schematic of an electronic device 1100 suitable for implementing embodiments of the present disclosure. The electronic device 1100 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 11 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0292] like Figure 11 As shown, electronic device 1100 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 1101, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1102 or a program loaded from storage device 1108 into random access memory (RAM) 1103. RAM 1103 also stores various programs and data required for the operation of electronic device 1100. The processing unit 1101, ROM 1102, and RAM 1103 are interconnected via bus 1104. Input / output (I / O) interface 1105 is also connected to bus 1104.

[0293] Typically, the following devices can be connected to I / O interface 1105: input devices 1106 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 1107 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1108 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1109. Communication device 1109 allows electronic device 1100 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 11 An electronic device 1100 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0294] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 1309, or installed from storage device 1308, or installed from ROM 1302. When the computer program is executed by processing device 1301, it performs the functions defined in the methods of embodiments of this disclosure.

[0295] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0296] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0297] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0298] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0299] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0300] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0301] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0302] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0303] In a first aspect, according to one or more embodiments of this disclosure, a method for drawing special effects images is provided, comprising:

[0304] In response to a user-triggered special effects rendering request, the system performs trajectory tracking on the target video provided by the user to obtain a set of 3D points corresponding to the motion trajectory of the target object in the target video.

[0305] The three-dimensional point set is meshed to obtain the target mesh structure;

[0306] Apply texture mapping to the target mesh structure to obtain the target special effects image;

[0307] Display target effect images in the target video that correspond to the motion trajectory of the target object.

[0308] According to one or more embodiments of this disclosure, trajectory tracking is performed on a target video provided by a user to obtain a set of three-dimensional target points corresponding to the motion trajectory of a target object in the target video, including:

[0309] Identify target objects in a target video;

[0310] Sample the motion trajectory of the target object in the target video to obtain at least one target sampling point;

[0311] At least one target sampling point is mapped to a three-dimensional spatial coordinate system to obtain a three-dimensional point set.

[0312] According to one or more embodiments of this disclosure, sampling the motion trajectory of a target object in a target video to obtain at least one target sampling point includes:

[0313] The motion trajectory of the target object is sampled from the target video to obtain a two-dimensional point set; the two-dimensional point set includes multiple first sampling points;

[0314] Resampling of the two-dimensional point set yields multiple second sampling points;

[0315] Determine at least one target sampling point from a plurality of second sampling points that satisfies the trajectory smoothing condition.

[0316] According to one or more embodiments of this disclosure, sampling the motion trajectory of a target object from a target video to obtain a two-dimensional point set includes:

[0317] Based on a preset sampling frequency, the trajectory points of the target object are collected from the image frames of the target video;

[0318] Based on a preset distance threshold, determine whether the trajectory points meet the distance constraint conditions;

[0319] If so, then determine the first sampling point based on the trajectory points;

[0320] If not, replace the trajectory point with the last first sampling point in the two-dimensional point set;

[0321] Obtain the two-dimensional point set corresponding to the multiple first sampling points obtained at the end of the target video acquisition.

[0322] According to one or more embodiments of this disclosure, determining whether a trajectory point satisfies a distance constraint condition based on a preset distance threshold includes:

[0323] Determine the first distance between the trajectory point and the last first sampling point in the two-dimensional point set;

[0324] If the first distance is greater than the distance threshold, then the trajectory point is determined to satisfy the distance constraint condition;

[0325] If the first distance is less than or equal to the distance threshold, then the trajectory point is determined not to meet the distance constraint condition.

[0326] According to one or more embodiments of this disclosure, determining a first sampling point based on trajectory points includes:

[0327] Obtain the number of sampling points of the first existing sampling point in the two-dimensional point set;

[0328] If the number of sampling points is determined to be less than or equal to the sampling number threshold, then the trajectory point is added as the last first sampling point in the two-dimensional point set;

[0329] If the number of sampling points is greater than the sampling number threshold, the mean point is obtained by weighted summation of the first N first sampling points and trajectory points in the two-dimensional point set; the mean point is added as the last first sampling point in the two-dimensional point set; N is a positive integer greater than 0.

[0330] According to one or more embodiments of this disclosure, a two-dimensional point set is resampled to obtain a plurality of second sampling points, including:

[0331] Based on the curve fitting algorithm, curve fitting is performed on multiple first sampling points in the two-dimensional point set to obtain the first sample line;

[0332] Curve fitting is performed on the first and last sampling points in the two-dimensional point set to obtain the second spline.

[0333] A closed curve is obtained by splicing the first and second splines;

[0334] Based on the segmented interval sampling strategy, the closed curve is resampled to obtain multiple second sampling points.

[0335] According to one or more embodiments of this disclosure, a closed curve is resampled according to a segmented interval sampling strategy to obtain multiple second sampling points, including:

[0336] Determine the number of segments corresponding to the segmented interval sampling strategy;

[0337] The closed curve is segmented according to the number of segments to obtain at least one segmented curve.

[0338] Collect second sampling points from the piecewise curve to obtain multiple second sampling points corresponding to at least one piecewise curve.

[0339] According to one or more embodiments of this disclosure, determining at least one target sampling point that satisfies a trajectory smoothing condition from a plurality of second sampling points includes:

[0340] Combine two adjacent sampling points from multiple second sampling points to obtain at least one set of sampling points;

[0341] Determine the dot product result between the two second sampling points in each group of sampling points. The dot product result is obtained by calculating the dot product of the tangent vectors of the two second sampling points.

[0342] For at least one set of sampling points, if the dot product of the two second sampling points in the target set is less than zero, then any second sampling point among the two second sampling points in the target set is determined as the target sampling point.

[0343] If the dot product of the two second sampling points in the target group is greater than or equal to zero, then both second sampling points in the target group are determined to be target sampling points.

[0344] According to one or more embodiments of this disclosure, at least one target sampling point is mapped to a three-dimensional spatial coordinate system to obtain a three-dimensional point set, including:

[0345] Based on the camera transformation matrix corresponding to the camera rendering the target video, the target sampling points are transformed into a spatial coordinate system to obtain the three-dimensional transformed coordinates corresponding to each target sampling point.

[0346] Obtain a three-dimensional point set consisting of the three-dimensional transformed coordinates corresponding to each target sampling point.

[0347] According to one or more embodiments of this disclosure, a three-dimensional point set is meshed to obtain a target mesh structure, including:

[0348] The entire region contained in the three-dimensional point set is triangularly divided to obtain a triangular mesh surface;

[0349] Determine the triangular mesh surface as the target mesh structure corresponding to the surface effect type.

[0350] The target mesh structure is textured to obtain the target effect image, including:

[0351] Texture mapping is performed on a 3D mesh surface to obtain the target texture image for 3D rendering;

[0352] Edge smoothing is performed on the target texture image of the 3D rendering to obtain the target special effects image.

[0353] According to one or more embodiments of this disclosure, the entire region encompassed by a three-dimensional point set is triangulated to obtain a triangular mesh surface, including:

[0354] Determine the center point corresponding to the 3D point set;

[0355] Based on the 3D point set and center point, a sector mesh topology is constructed to obtain the 3D topological structure;

[0356] Calculated values ​​of boundary values ​​of a 3D topology transformed to the percentage coordinate system of the image;

[0357] Based on the three-dimensional topology and the calculated values ​​of each point on the three-dimensional topology, the triangular mesh surface is determined.

[0358] According to one or more embodiments of this disclosure, edge smoothing processing is performed on a target texture image rendered in three dimensions to obtain a target special effects image, including:

[0359] Each pixel of the target texture image in the 3D rendering is rendered into a rendering texture where all channel values ​​are zero, thus obtaining the first texture map.

[0360] The alpha channel of the first texture map is drawn onto the grayscale rendering texture to obtain a grayscale mask image;

[0361] The grayscale mask image is subjected to erosion and blurring processing to obtain a smooth mask image;

[0362] The smooth mask image and the first texture map are fused together to obtain the target special effect image.

[0363] According to one or more embodiments of this disclosure, a three-dimensional point set is meshed to obtain a target mesh structure, including:

[0364] Interpolate between pairs of adjacent sampling points in the 3D point set to obtain the target point set;

[0365] A strip network is generated for the contour region corresponding to the target point set to obtain the target mesh structure;

[0366] The target mesh structure is textured to obtain the target effect image, including:

[0367] The target mesh structure is rendered to obtain a three-dimensional bar trajectory map;

[0368] Based on the 3D bar trajectory diagram, determine the target special effects image.

[0369] According to one or more embodiments of this disclosure, determining a target special effects image based on a three-dimensional bar trajectory includes:

[0370] The three-dimensional bar trajectory is rendered onto a rendering texture where all channel values ​​are zero, thus obtaining a second texture map;

[0371] The edge blending process of the 3D strip trajectory is performed based on the second texture map to obtain a trajectory blending image;

[0372] The trajectory blending image is made transparent according to the rendering transparency to obtain the target special effect image.

[0373] According to one or more embodiments of this disclosure, it further includes:

[0374] Determine the trigger time for user-triggered effect rendering requests;

[0375] Based on the time periods corresponding to at least one festival type, determine the target festival type corresponding to the trigger time;

[0376] Obtain the texture image pre-associated with the target holiday type;

[0377] The target mesh structure is textured to obtain the target effect image, including:

[0378] Texture images are used to apply textures to the target mesh structure to obtain the target special effects image.

[0379] Secondly, according to one or more embodiments of this disclosure, a special effects image rendering apparatus is provided, comprising:

[0380] The trajectory tracking unit is used to respond to the user-triggered special effects rendering request, track the trajectory of the target video provided by the user, and obtain the three-dimensional point set corresponding to the motion trajectory of the target object in the target video;

[0381] Mesh generation unit, used to mesh a 3D point set to obtain the target mesh structure;

[0382] The special effects generation unit is used to process the texture of the target mesh structure to obtain the target special effects image;

[0383] The special effects display unit is used to display target special effects images corresponding to the motion trajectory of the target object in the target video.

[0384] Thirdly, according to one or more embodiments of the present disclosure, an electronic device is provided, including: a processor, a memory, and an output device;

[0385] The memory stores the instructions that the computer executes;

[0386] The processor executes computer execution instructions stored in memory, causing the processor to be configured with the special effects image rendering method of the first aspect and various possible designs of the first aspect, and the output device is used to output a target video with the target special effects image.

[0387] Fourthly, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, which stores computer-executable instructions that, when executed by a processor, implement the special effects image drawing method described above and various possible designs of the first aspect.

[0388] Fifthly, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the special effects image drawing method as described in the first aspect above and various possible designs of the first aspect.

[0389] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0390] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0391] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for drawing special effects images, characterized in that, include: During the playback of the target video, in response to the user-triggered special effects rendering request, the trajectory of the target object in the target video provided by the user is tracked to obtain the three-dimensional point set corresponding to the motion trajectory of the target object in the target video; The three-dimensional point set is meshed to obtain the target mesh structure; The target mesh structure is textured to obtain a target special effects image; Display the target special effects image corresponding to the motion trajectory of the target object in the target video; The step of meshing the three-dimensional point set to obtain the target mesh structure includes: The entire region contained in the three-dimensional point set is triangularly divided to obtain a triangular mesh surface; The triangular mesh surface is determined to be the target mesh structure; The process of applying texture mapping to the target mesh structure to obtain the target special effects image includes: Texture mapping is performed on the triangular mesh surface to obtain a target texture image for 3D rendering; The target texture image of the 3D rendering is subjected to edge smoothing processing to obtain the target special effects image; The step of triangulating the entire region contained in the three-dimensional point set to obtain a triangular mesh surface includes: The center point corresponding to the three-dimensional point set is determined; based on the three-dimensional point set and the center point, a sector mesh topology is constructed to obtain a three-dimensional topological structure; the boundary values ​​of the three-dimensional topological structure are converted to the calculated values ​​of the percentage coordinate system of the image; based on the three-dimensional topological structure and the calculated values ​​of each point on the three-dimensional topological structure, the triangular mesh surface is determined.

2. The method according to claim 1, characterized in that, The step of tracking the trajectory of a target object in the target video provided by the user to obtain a set of three-dimensional points corresponding to the motion trajectory of the target object in the target video includes: Identify the target object in the target video; The motion trajectory of the target object in the target video is sampled to obtain at least one target sampling point; At least one of the target sampling points is mapped to a three-dimensional spatial coordinate system to obtain the three-dimensional point set.

3. The method according to claim 2, characterized in that, The step of sampling the motion trajectory of the target object in the target video to obtain at least one target sampling point includes: The motion trajectory of the target object is sampled from the target video to obtain a two-dimensional point set; the two-dimensional point set includes multiple first sampling points. The two-dimensional point set is resampled to obtain multiple second sampling points; Determine at least one target sampling point from a plurality of second sampling points that satisfies the trajectory smoothing condition.

4. The method according to claim 3, characterized in that, The step of sampling the motion trajectory of the target object from the target video to obtain a two-dimensional point set includes: According to a preset sampling frequency, the trajectory points of the target object are collected from the image frames of the target video; Based on a preset distance threshold, determine whether the trajectory points meet the distance constraint conditions; If so, then the first sampling point is determined based on the trajectory points; If not, then replace the trajectory point with the last first sampling point in the two-dimensional point set; Obtain the two-dimensional point set corresponding to the multiple first sampling points obtained at the end of the target video acquisition.

5. The method according to claim 3, characterized in that, The resampling of the two-dimensional point set to obtain multiple second sampling points includes: Based on the curve fitting algorithm, curve fitting is performed on multiple first sampling points in the two-dimensional point set to obtain the first sample line; Curve fitting is performed on the first and last first sampling points in the two-dimensional point set to obtain the second spline; A closed curve is obtained by splicing the first spline and the second spline; According to the segmented interval sampling strategy, the closed curve is resampled to obtain the plurality of second sampling points.

6. The method according to claim 3, characterized in that, Determining at least one target sampling point that satisfies the trajectory smoothing condition from a plurality of second sampling points includes: Combine two adjacent sampling points from a plurality of second sampling points to obtain at least one set of sampling points; Determine the dot product result of the two second sampling points in each group of sampling points. The dot product result is obtained by calculating the dot product of the tangent vectors of the two second sampling points. For the at least one set of sampling points, if the dot product of the two second sampling points in the target set is less than zero, then any second sampling point among the two second sampling points in the target set is determined as the target sampling point. If the dot product of the two second sampling points in the target group is greater than or equal to zero, then both second sampling points in the target group are determined to be the target sampling points. The at least one set of the sampling points corresponds to at least one target sampling point.

7. The method according to claim 1, characterized in that, The step of performing edge smoothing processing on the target texture image of the 3D rendering to obtain the target special effects image includes: Each pixel of the target texture image of the three-dimensional rendering is rendered into a rendering texture in which all channel values ​​are zero, to obtain the first texture map. The alpha channel of the first texture map is drawn onto the grayscale rendering texture to obtain a grayscale mask image; The grayscale mask image is subjected to erosion and blurring processing to obtain a smooth mask image; The smooth mask image and the first texture map are fused together to obtain the target special effects image.

8. The method according to claim 1, characterized in that, The step of meshing the three-dimensional point set to obtain the target mesh structure includes: Interpolate the pairwise adjacent sampling points in the three-dimensional point set to obtain the target point set; A strip mesh is generated for the contour region corresponding to the target point set to obtain the target mesh structure; The process of applying texture mapping to the target mesh structure to obtain the target special effects image includes: The target mesh structure is rendered to obtain a three-dimensional bar trajectory map; The target special effects image is determined based on the three-dimensional bar trajectory diagram.

9. The method according to claim 8, characterized in that, Determining the target special effects image based on the three-dimensional bar chart includes: The three-dimensional bar trajectory map is rendered onto a rendering texture where all channel values ​​are zero to obtain a second texture map. The second texture map is made transparent according to the rendering transparency to obtain the target special effect image.

10. The method according to claim 1, characterized in that, Also includes: Determine the trigger time of the special effects rendering request initiated by the user; Based on the time periods corresponding to at least one festival type, determine the target festival type corresponding to the trigger time; Obtain the texture image pre-associated with the target holiday type; The process of applying texture mapping to the target mesh structure to obtain the target special effects image includes: The target mesh structure is mapped using the texture image to obtain the target special effects image.

11. A special effects image drawing device, characterized in that, include: The trajectory tracking unit is used to track the trajectory of the target object in the target video provided by the user in response to the user-triggered special effects rendering request during the playback of the target video, and obtain the three-dimensional point set corresponding to the motion trajectory of the target object in the target video. A mesh generation unit is used to perform meshing processing on the three-dimensional point set to obtain the target mesh structure; The special effects generation unit is used to perform texture processing on the target mesh structure to obtain a target special effects image; A special effects display unit is used to display the target special effects image corresponding to the motion trajectory of the target object in the target video; The mesh generation unit is specifically used to triangulate the entire region contained in the three-dimensional point set to obtain a triangular mesh surface; and to determine the triangular mesh surface as the target mesh structure. The special effects generation unit is specifically used to perform texture mapping on the triangular mesh surface to obtain a target texture image for 3D rendering; and to perform edge smoothing processing on the target texture image for 3D rendering to obtain the target special effects image. The mesh generation unit is specifically used to determine the center point corresponding to the three-dimensional point set; construct a sector mesh topology based on the three-dimensional point set and the center point to obtain a three-dimensional topological structure; convert the boundary values ​​of the three-dimensional topological structure to the calculated values ​​of the percentage coordinate system of the image; and determine the triangular mesh surface based on the three-dimensional topological structure and the calculated values ​​of each point on the three-dimensional topological structure.

12. An electronic device, characterized in that, include: Processor, memory, and output devices; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to implement the special effects image drawing method as described in any one of claims 1 to 10, and the output device is used to output a target video having a target special effects image.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the special effects image drawing method as described in any one of claims 1 to 10.

Citation Information

Patent Citations

  • Planar trajectory drawing method and device, equipment, and storage medium

    CN112634401A

  • Special effect video processing method and device, electronic equipment and storage medium

    CN114401443A