Material presentation method, device, electronic device, storage medium, and program product

Abstract

The present disclosure relates to a material presentation method, device, electronic device, storage medium, and program product, in which a captured video image is obtained, a trajectory image is obtained based on the position of a target identification object in the video image, and a superimposition process is performed based on the trajectory image and a material image corresponding to a target element to obtain a plurality of target unit effect images. The plurality of target unit effect images are merged to obtain a target element. The main body of the target three-dimensional effect is a three-dimensional geometric structure including a plurality of surfaces, and the plurality of target unit effect images are pasted onto the corresponding geometric surfaces, respectively, to present the plurality of target unit effect images on the plurality of geometric surfaces of the three-dimensional geometric structure.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application is based on and claims priority from a Chinese application having application number CN202210089981.6 and filed on January 25, 2022, the entire disclosure of which is incorporated herein by reference in its entirety.

[0002] The present disclosure relates to the field of Internet technology, and in particular to a material presentation method, device, electronic device, storage medium, and program product. [Background technology]

[0003] With the development of Internet technology, the technology of human-machine interaction between smart terminals and users has become more mature, and there are many applications related to human-machine interaction. Meanwhile, human-machine interaction in entertainment scenes can bring more enjoyment to users.

[0004] In existing man-machine interaction scenarios, applications can provide effects to users for them to use and present. Summary of the Invention

[0005] According to some embodiments of the present disclosure, Acquiring a captured video image and obtaining a trajectory image based on position information of a target identification object in the captured video image; obtaining a plurality of target unit effect images based on the trajectory image; A material presentation method is provided, which includes generating a target element based on the plurality of target unit effect images and presenting the target element in a captured video image, wherein the body of the target element is a three-dimensional geometric structure including a plurality of geometric surfaces, and pasting the plurality of target unit effect images onto the corresponding geometric surfaces to present the plurality of target unit effect images on the plurality of geometric surfaces.

[0006] In some embodiments, obtaining a plurality of target unit effect images based on the trajectory image as described above includes: superimposing the trajectory image and a material image corresponding to the target element to obtain a unit effect image; and performing a copy process and a mirror process on the unit effect image to obtain the plurality of target unit effect images.

[0007] In some embodiments, the above-described method of superimposing the trajectory image and the material image corresponding to the target element to obtain the unit effect image includes: performing a mirror image process on the trajectory image to obtain a mirror image corresponding to the trajectory image; The method includes superimposing the trajectory image and the mirror image, respectively, on different regions of a material image corresponding to the target material to obtain the unit effect image.

[0008] In some embodiments, obtaining a trajectory image based on position information of a target object in the captured video image as described above includes: Draw two circular regions on a pre-constructed canvas based on the positions of the target identification object in each of two consecutively captured video images; Draw a rectangular area on the canvas based on a connecting line between the positions of the target identification object in each of the two consecutively captured video images, where the size of the short side of the rectangular area is the same as the diameter of the circular area, the short side of the rectangular area is the side perpendicular to the connecting line, and the center point of the width of the rectangular area overlaps with the center of the corresponding circular area; and drawing the trajectory based on the continuously captured video images until the trajectory drawing is completed, thereby obtaining the trajectory image.

[0009] In some embodiments, the method further comprises: The method further includes performing a blurring process on the trajectory image so that the trajectory in the trajectory image has a luminous effect.

[0010] In some embodiments, the size of the long side of the rectangular area is proportional to the length of the connecting line, and the long side of the rectangular area is the side parallel to the connecting line.

[0011] In some embodiments, the blurring process is a Gaussian blurring process.

[0012] In some embodiments, the plurality of target unit effect images in a tiled state are moved along corresponding paths, and the plurality of target unit effect images are sequentially pasted onto corresponding geometric surfaces to obtain the target material.

[0013] In some embodiments, the step of presenting the target element in the captured video image comprises: The method includes presenting the target element in a captured video image by rotating it around a rotation axis corresponding to the three-dimensional geometric structure.

[0014] In some embodiments, the target element is a paper lantern.

[0015] In some embodiments, presenting the target element in a captured video image as described above includes presenting the target element in a subsequently captured video image.

[0016] In some embodiments, the method further includes adding at least one of a foreground image, a filter, text, or a stamp that matches the target element to a subsequently captured video image.

[0017] According to some further embodiments of the present disclosure, an image acquisition module for acquiring the captured video images; a trajectory generation module for obtaining a trajectory image based on position information of a target identification object in the captured video image; a material generation module for obtaining a plurality of target unit effect images based on the trajectory image, and generating target elements based on the plurality of target unit effect images; a presentation module for presenting the target element in a captured video image; The body of the target element is a three-dimensional geometric structure including a plurality of geometric surfaces, and a material presentation device is provided in which the plurality of target unit effect images are pasted onto the corresponding geometric surfaces, and the plurality of target unit effect images are presented on the plurality of geometric surfaces.

[0018] According to yet another embodiment of the present disclosure, there is provided an electronic device including a memory and a processor, the memory configured to store computer program instructions; The processor provides an electronic device configured to execute the computer program instructions to cause the electronic device to implement the method for presenting material according to any of the embodiments described above.

[0019] According to yet another embodiment of the present disclosure, there is provided a readable storage medium containing computer program instructions, comprising: A readable storage medium is provided in which the computer program instructions, when executed by at least one processor of an electronic device, cause the electronic device to implement the method of presenting material of any of the embodiments described above.

[0020] According to yet another embodiment of the present disclosure, there is provided a computer program product that, when executed by a computer, causes the computer to implement the material presentation method of any of the above-described embodiments.

[0021] According to still further embodiments of the present disclosure, there is provided a computer program comprising instructions that, when executed by a processor, implement the material presentation method of any of the above-described embodiments. [Brief explanation of the drawings]

[0022] The drawings herein, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

[0023] In order to more clearly describe the technical solutions in the embodiments of the present disclosure or the prior art, the following briefly introduces drawings that are required to be used in the description of the embodiments or the prior art. Obviously, those skilled in the art can obtain other drawings based on these drawings without any creative efforts. [Figure 1] 1 is a schematic diagram showing the flow of a material presentation method according to an embodiment of the present disclosure. [Figure 2A] FIG. 10 is an effect presentation diagram according to an embodiment of the present disclosure. [Figure 2B] FIG. 10 is an effect presentation diagram according to an embodiment of the present disclosure. [Figure 2C] FIG. 10 is an effect presentation diagram according to an embodiment of the present disclosure. [Figure 2D] FIG. 10 is an effect presentation diagram according to an embodiment of the present disclosure. [Figure 2E] FIG. 10 is an effect presentation diagram according to an embodiment of the present disclosure. [Figure 2F]FIG. 10 is an effect presentation diagram according to an embodiment of the present disclosure. [Figure 3] FIG. 10 is a schematic diagram illustrating the effect of a trajectory pattern according to an embodiment of the present disclosure. [Figure 4] 1 is a schematic diagram illustrating the configuration of a material presentation device according to an embodiment of the present disclosure. [Figure 5] 1 is a schematic diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0024] In order to make the above objects, features and advantages of the present disclosure more clearly understandable, the following further describes the aspects of the present disclosure. It should be noted that, unless contradictory, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0025] In the following description, many specific details are set forth in order to fully understand the present disclosure; however, the present disclosure may be embodied in other forms different from those described herein. Obviously, the embodiments in the specification are only some of the embodiments of the present disclosure, but not all of the embodiments.

[0026] The material presentation method according to the present disclosure may be implemented by a material presentation device according to the present disclosure, which may be implemented in any software and / or hardware form. In some embodiments, the material presentation device may be an electronic device such as a tablet, a mobile phone (e.g., a flip phone, a large-screen phone, etc.), a wearable device, an augmented reality (AR) / virtual reality (VR) device, a laptop, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), and the present disclosure does not limit the specific type of electronic device.

[0027] In the following embodiments, the material presentation method according to the present disclosure will be described in detail with reference to an electronic device as an example, along with drawings and application scenarios.

[0028] 1 is a schematic diagram showing the flow of a material presentation method according to some embodiments of the present disclosure. As shown in FIG. 1, the method according to this embodiment includes steps S101 to S104.

[0029] In step S101, a captured video image is acquired, and a trajectory image is obtained based on the position information of the target identification object in the captured video image.

[0030] The captured video image is an image currently captured or taken by the electronic device within a suitable image capture range using an image capture module (e.g., a camera). The present disclosure does not limit the size, clarity, or other parameters of the captured video image.

[0031] Each captured video image may include a target identification object. The present disclosure is not limited to the target identification object. For example, the target identification object may be a part of a human body, such as a finger, a nose, an eye, a mouth, etc.

[0032] The electronic device may obtain captured video images, perform image classification on each video image, determine whether each video image includes a target object, and obtain location information of the target object in each video image. The present disclosure is not limited to an implementation in which the electronic device performs image classification. For example, the electronic device may perform image classification on each captured video image using an image classification model. The image classification model may be, but is not limited to, a neural network model, a convolutional network model, or other types of models.

[0033] Of course, the electronic device may obtain position information of the target to be identified in each video image by performing image identification on each captured video image in another manner.

[0034] In the following example, the user's face is located within an image collection range corresponding to the image collection module of the electronic device, the image collection module of the electronic device collects images of the person's face, and the target identification object is the nose on the person's face. This example explains how the electronic device draws and obtains a trajectory image based on the position information of the target identification object in each video image.

[0035] After obtaining captured video images, the electronic device draws a circular element on a pre-constructed canvas based on the position information of a key point corresponding to the user's nose on each video image, and a rectangular element may connect the newly drawn circular element to the circular element drawn based on the previous video image, or the rectangular element and the circular element may overlap. Video images captured by the electronic device are typically captured continuously. Therefore, as the number of captured video images increases, more circular elements and rectangular elements are drawn in real time to dynamically present a trajectory image generated in accordance with the user's movements.

[0036] Here, the position information of the target identification object in each video image may be represented by the coordinates of the target identification object in each video image. A corresponding coordinate system may be constructed according to actual needs. For example, a two-dimensional coordinate system may be constructed by setting the lower left corner of each captured video image as the origin of the coordinate system, the horizontal axis as the abscissa, and the vertical axis as the ordinate. Of course, two-dimensional coordinate systems may also be constructed in other forms.

[0037] Here, the canvas can be considered as a base on which a trajectory image is placed. The present disclosure does not limit parameters such as the size and color of the canvas. For example, the size of the canvas may be related to the size of each face of the target object included in the target 3D effect, and the color of the canvas may be any preset color, such as white, gray, or black.

[0038] When drawing a circular area (also referred to as a circular material) and a rectangular area (also referred to as a rectangular material) on a canvas, the size (e.g., diameter or radius) of the circular material may be a preset value in the initial state, and the size of the circular material may then change according to the moving speed of the nose on the person's face. The initial state referred to here means the state of the circular material drawn based on the first captured video image. When drawing the circular material and the rectangular material on the canvas, the circular material and the rectangular material have different colors from the canvas. This allows the drawn trajectory to be clearly presented to the user. For example, the color of the canvas may be black, and the color of the circular material and the rectangular material may be white.

[0039] In some embodiments, when drawing a trajectory image, the trajectory may be presented using a specific camera. Note that the specific camera referred to here is not a real camera, but a virtual camera. Before starting to draw the trajectory image, the camera's ClearType is set to Amaz.CameraClearType.Done. In this way, the screen of the trajectory image of the previous frame drawn is not cleared, and the effect of drawing the trajectory with a paintbrush is displayed. Here, ClearType represents the clear mode corresponding to the virtual camera, and Amaz.CameraClearType.Done represents setting the clear mode of the virtual camera to not clear, i.e., not clearing the contents of the previously drawn trajectory image. This allows the drawn trajectory image to be presented to the user.

[0040] S102: Obtain a plurality of target unit effect images based on the trajectory image.

[0041] In some embodiments, the target element is an element included in the effect to be realized. The body of the target element may be a three-dimensional geometric structure, such as a cone, a cylinder, a prism, or a spherical structure including multiple arcuate surfaces. The three-dimensional geometric structure typically includes multiple geometric surfaces, and the target unit effect image is an image to be presented by being pasted onto each geometric surface included in the body of the target element.

[0042] The size, shape, and other parameters of the multiple target unit effect images may be the same or different, and are related to the thematic structure of the target element and are determined according to the specific situation.

[0043] In some embodiments, a unit effect image may be obtained by performing a superimposition process on the material image and the trajectory image corresponding to the target element, and then the electronic device may perform a copy process and a mirror process on the unit effect image to obtain multiple target unit effect images.

[0044] The superimposition process referred to here means replacing the background of the trajectory image with a material image corresponding to the target element. If the target element is a lantern, the material image corresponding to the target element is an initial material image to be pasted onto each geometric surface of the lantern body, and the initial material image is a material image that is not superimposed on the trajectory image drawn by the user.

[0045] Here, the copy process is a process form that generates the same image based on the unit effect image. The mirror process is a process form that generates an image that has an axisymmetric structure with respect to a specific axis based on the unit effect image. The number of copy processes and mirror process may be determined depending on the number of geometric surfaces of the three-dimensional geometric structure of the target object.

[0046] In some embodiments, the electronic device may perform mirror processing on the trajectory image to obtain a mirror image corresponding to the trajectory image, and then superimpose the trajectory image and the mirror image corresponding to the trajectory image on different regions of the material image to obtain a unit effect image. Here, the two regions of the material image that are superimposed on the trajectory image and the mirror image corresponding to the trajectory image, respectively, may be in a mirror image state, or the two regions may be understood as being axially symmetrical with respect to a specific axis (e.g., a horizontal axis).

[0047] In some other embodiments, the shapes and sizes of the plurality of geometric surfaces of the body structure of the target element are not all the same. Material images corresponding to the plurality of geometric surfaces of the target element may be pre-stored in the electronic device, and after obtaining the trajectory image, the electronic device may superimpose the trajectory image on the plurality of material images to obtain the plurality of target unit effect images.

[0048] When superimposing a trajectory image on a material image, the size of the trajectory image may not match the size of the material image. In this case, the size of the trajectory image may be adjusted to match the size of the material image so that the trajectory image fits the material image. For example, if the size of the trajectory image is 720 x 1280 and the size of the material image is 1024 x 1024, the size of the trajectory image may be adjusted to 1024 x 1024, and the adjusted trajectory image may then be superimposed on the material image.

[0049] In step S103, a target element is generated based on the multiple target unit effect images, and the body of the target element is a three-dimensional geometric structure including multiple geometric surfaces, and the multiple target unit effect images are pasted onto corresponding geometric surfaces, respectively, to present the multiple target unit effect images on the multiple geometric surfaces.

[0050] In accordance with the description in step S102, the body of the target element is a three-dimensional geometric structure, and the three-dimensional geometric structure may generally include multiple geometric surfaces. The target unit effect image can be regarded as an image presented on multiple geometric surfaces included in the body of the target element.

[0051] The process of generating a target element based on multiple target unit effect images can be seen as performing operations such as rotating, translating, and scaling the target unit effect images so that the target unit effect images fit the position, size, etc. of the corresponding geometric surface in the main structure of the target element, and so that the target unit effect images are displayed pasted onto the geometric surface of a three-dimensional geometric structure.

[0052] The merging process to generate the target element may also be presented in the form of an animation. For example, if the target element is a lantern, a pre-defined lantern model may be used. Specifically, lines are drawn according to the patch frame structure of the lantern model to create a skeleton, and appropriate bind skinning is performed. The lantern model is then split and the lantern model is returned to a single patch state in a reverse direction step by step. The animation is then played backward to present the merging process to generate the target element. Here, each patch in the lantern model corresponds to one target unit effect image.

[0053] In step S104, a target element is presented in the captured video image.

[0054] After generating the target element in the above manner, the target element may be presented in the captured video image using a preset presentation form. The present disclosure is not limited to the preset presentation form. For example, the target element may be presented in one or more forms, such as rotation, vertical movement, left-right swing, or forward-backward swing. A specific presentation form may be set according to actual needs.

[0055] In some embodiments, the preset presentation format may rotate the target element in the captured video image around a rotation axis corresponding to the three-dimensional geometric structure of the target element. During the rotation, the rotation axis may swing left and right. For example, if the target element is a lantern, the lantern may be displayed superimposed on top of the captured video image, and each face of the lantern may be rotated around the central axis of the lantern body. While the lantern is rotating, the lantern may also swing left and right. The display position of the lantern may be set according to needs. For example, the lantern may be displayed in an area near the top of the display screen of the electronic device. This ensures that the lantern does not obscure the user's face in the captured video image while it is being displayed.

[0056] Furthermore, while the target element is being presented in the captured video image, a foreground image that matches the target element may be added to the captured video image, and a filter, text, stamp, etc. that matches the target element may be added to the captured video image. Here, the foreground image, filter, text, stamp, etc. that matches the target element may be arranged in advance.

[0057] Note that the video image for identifying the target identification object and generating a trajectory image and the video image superimposed and presented with the target element are different video images. Alternatively, it may be understood that the electronic device identifies the target identification object in real time based on the captured video images while shooting video clip 1, generates a trajectory image, and further generates a target element based on the trajectory image, and then shoots video clip 2 in real time and presents the target element in the form of a set animation in the video image included in video clip 2. Video clip 1 and video clip 2 may be two video clips shot consecutively, and the time of video clip 1 is earlier than the time of video clip 2.

[0058] In the method according to the above embodiment, captured video images are acquired, a trajectory image is obtained based on the position of a target identification object in each video image, and multiple target unit effect images are obtained by performing a superposition process based on the trajectory image and material images corresponding to the target element. The multiple target unit effect images are merged to obtain a target element. The body of the target element is a three-dimensional geometric structure including multiple geometric surfaces, and the multiple target unit effect images are pasted onto corresponding geometric surfaces to present the multiple target unit effect images on the multiple geometric surfaces of the three-dimensional geometric structure. This aspect draws a trajectory based on a user's movements, and the drawn trajectory is used as one of the materials for generating the target element, allowing the user to participate in the design of the target element and advantageously improving the user's interaction experience. Furthermore, the body of the target element in the effect generated using the above aspect has a three-dimensional structure, which improves the visual expressiveness of the effect.

[0059] Based on the description of the embodiment shown in Fig. 1, the material presentation method according to the present disclosure will be introduced in conjunction with the scenes and drawings shown in Fig. 2A to Fig. 2F, taking as an example a case where the electronic device is a mobile phone and a video editing application (hereinafter abbreviated as Application 1) is installed on the mobile phone. Here, in the embodiment shown in Fig. 2A to Fig. 2F, the target three-dimensional effect is a lantern effect, that is, the target element is a lantern, and the target identification object is the tip of the nose on a human face, and the process of drawing a trajectory image, generating a lantern effect, and presenting the lantern effect will be described.

[0060] Here, when a user triggers an icon corresponding to the lantern effect, the application 1 may illustratively display a user interface 21 as shown in FIG. 2A on the mobile phone and display prompt information on the user interface 21 to prompt the user to start drawing a lantern. The prompt information may be implemented in one or more forms, such as text, animation, or audio, and the present disclosure does not limit the display parameters of the text, such as font, size, or color. In some embodiments, as shown in FIG. 2A , a prompt text content such as "Draw a lantern" may be displayed in an area near the top of the user interface 21, and a lantern animation may be displayed while the prompt text content is being displayed. The duration of the prompt text content may also be preset, for example, 1 second, 2 seconds, or the like.

[0061] When the presentation time length of the prompt text content reaches a preset presentation time length, the application 1 may identify the nose tip portion of the user 1's face and display a paintbrush at the nose tip portion of the user 1. For example, in the embodiment shown in FIG. 2B , the paintbrush is displayed at the nose tip portion of the user 1. The present disclosure is not limited to the style of the paintbrush.

[0062] In this embodiment, the lantern body is spherical, the vertical central axis of the sphere is the axis of rotation, and the arc surfaces surrounding the axis of rotation at 45-degree angles are one geometric surface. When presenting the paintbrush, a lantern patch corresponding to the arc surface may be superimposed on top of the captured video image. The lantern patch is a material image corresponding to the lantern effect.

[0063] In this embodiment, the arc surfaces corresponding to each 45-degree angle are one geometric surface, and the eight geometric surfaces surrounding the central axis of the sphere have the same shape and size, so that the application 1 can display a lantern patch. This allows the user to experience the scene of drawing a lantern patch, improving the user experience.

[0064] In some possible cases, the application 1 may present only a portion of the lantern patch, for example, the lower half of the lantern patch (of course, the upper half of the lantern patch may also be presented). Furthermore, when this form is adopted and implemented, the portion of the lantern patch that is not presented may be masked and hidden, as in the embodiment shown in Fig. 2B. In the embodiment shown in Fig. 2B, the lower half of the lantern patch presented on the user interface 22 may be regarded as the effective drawing area of ​​the trajectory image.

[0065] When this embodiment is adopted and realized, after the user draws a trajectory pattern in the lower half area of ​​the lantern patch, the application 1 may perform a mirror image process on the drawn trajectory image to obtain a mirror image corresponding to the trajectory image, and then superimpose the mirror image corresponding to the trajectory image on the upper half of the lantern patch (or the lower half of the lantern patch). The target lantern patch pattern obtained by adopting this embodiment has a vertically symmetrical effect, which provides the user with a rich visual effect and enhances the expressive power of the lantern effect.

[0066] In some embodiments, application 1 may present a complete lantern patch. A user can draw a trajectory image in an area corresponding to the entire lantern patch. This configuration is simple and can be quickly implemented by electronic devices. In this case, the complete lantern patch presented by application 1 is the valid drawing area of ​​the trajectory image.

[0067] In some embodiments, the application 1 may present a single trajectory drawing area to the user without presenting a lantern patch to the user. The trajectory drawing area may not be related to the shape of the lantern patch.

[0068] In some embodiments, application 1 may present a complete lantern patch. The user may draw in an area corresponding to the entire lantern patch. The electronic device may record only the trajectory the user draws in the lower half (or upper half) of the lantern patch.

[0069] In some cases, the length of time for drawing the trajectory may be limited. To allow the user to understand the remaining time for drawing the trajectory, the remaining time for drawing the trajectory may be displayed in the form of a progress bar in the user interface. For example, still referring to FIG. 2B , the progress bar may be displayed in an area near the top of the user interface 22. To ensure the user's experience of drawing the lantern pattern, the progress bar may obscure as little of the facial area of ​​user 1 and the lower half of the lantern patch in the captured video image as possible.

[0070] User 1 may then move the nose tip region by moving the facial region. Application 1 displays a drawn trajectory image on the mobile phone, following the movement of User 1's nose tip region. In some embodiments, the trajectory image drawn by User 1 is as shown in FIG. 2C.

[0071] In some possible cases, the rendering time length may be preset. The present disclosure does not limit the time length. For example, the rendering time length may be 5 seconds, 6 seconds, 7 seconds, etc., and may be set according to actual circumstances.

[0072] In conjunction with the embodiment shown in FIG. 2B, the lower half of the lantern patch is the active drawing area. While user 1 is moving his or her face, the nose tip may move out of the lower half of the lantern patch, i.e., out of the active drawing area. Application 1 may not record trajectories outside the active drawing area, but may record only drawing trajectories within the active drawing area.

[0073] The following will introduce how to draw a trajectory using the example shown in FIG.

[0074] Referring to the embodiment shown in FIG. 3 , application 1 obtains and identifies a first frame of a captured video image to obtain the position of user 1's nose tip in the first frame of the video image, and then draws a circular element s1 on a pre-constructed canvas. The position of circular element s1 on the canvas may be determined based on the position of user 1's nose tip in the first frame of the video image, and the diameter of circular element s1 may be a preset value. Application 1 obtains and identifies a second frame of a captured video image to obtain the position of user 1's nose tip in the second frame of the video image, and then draws a circular element s2 and a rectangular element r1 connecting circular element s1 and circular element s2 on the canvas. Here, the diameter of circular element s2 may be determined by the length of a line connecting the center of the circle of circular element s1 to the center of the circle of circular element s2. The longer the length of the connecting line, the thinner the thickness of the drawn trajectory. The shorter the length of the connecting line, the thicker the thickness of the drawn trajectory.

[0075] By calculating the time spent at the tip of the nose and controlling the diameter of the pencil brush (i.e., the diameter of the circular material), the thickness of the paintbrush can be changed, achieving a touch effect closer to that of an actual painting.

[0076] Let len ​​represent the distance of the connecting line between the positions of the nose tip of user 1 in the video images of two adjacent frames, Y and basescale represent the first and second parameters related to the size of the circular material, respectively, and spotX represent the size of the circular material. Since the size of the side perpendicular to the drawing direction of the rectangular material is the same as that of the circular material, determining the size of the circular material is equivalent to simultaneously determining the size of the rectangular material.

[0077] In some embodiments, the size of the circular blank may be determined by the following formula:

[0078] Step a: The value of the first parameter Y is calculated by substituting the distance len of the connecting line between the positions of the nose tip of user 1 in the video images of two adjacent frames into the inverse proportional function "Y=A1-len," where A1 is a preset constant, e.g., A1=0.04.

[0079] If Y is greater than the preset value A2, the value of the second parameter basescale is determined in step b. If Y is equal to or less than the preset value A2, the value of the second parameter basescale is determined in step c. For example, A2=0.01.

[0080] Step b: Determine the value of the second parameter basescale by the formula basescale=min(basescale+a1*Y,Y).

[0081] Step c: Determine the value of the second parameter basescale by the formula basescale=max(basescale-a2*Y,Y).

[0082] Here, the aforementioned a1 and a2 are preset constants, for example, a1 = 0.02 and a2 = 0.01. The magnitudes of the values ​​of a1 and a2 mainly affect the change in the thickness of the drawn trajectory, so the magnitudes of the values ​​of a1 and a2 may be set according to the actual situation.

[0083] Step d: The size of spotX is determined by the formula spotX=max(a3, min(basescale, a4)).

[0084] Here, a3 and a4 are the minimum and maximum values ​​of parameters relating to the size of the circular material that have been set in advance, for example, a3=0.003 and a4=0.04.

[0085] After determining the size of spotX, the size of the circular material (for example, diameter or radius) can be obtained by multiplying spotX by the size of the circular material that has been set in advance.

[0086] In the above steps a to d, the parameters a1 to a4 can be regarded as, for example, enlargement / reduction magnifications corresponding to the circular material.

[0087] In conjunction with the formulas shown in steps a to d above, the present disclosure uses a preset value A2 as a judgment condition to prevent the thickness of the drawn path from changing suddenly and achieve the effect of the path thickness changing gradually, softly, and naturally, and when Y exceeds A2, the size of the circular element is increased by a factor of 0.02 (i.e., a1), with the upper limit being Y. When Y is equal to or less than A2, the size of the circular element is decreased by a factor of 0.01 (i.e., a2), with the lower limit being Y.

[0088] As can be seen from the example shown in Figure 1, application 1 can display a trajectory drawn by a specific camera. Therefore, it captures a third frame of video image and draws a circular element s3 and a rectangular element r2 based on the third frame of video image, while the previously drawn circular element s1, circular element s2, and rectangular element r1 remain and are not cleared, allowing the user to see the drawing process.

[0089] Similarly, as application 1 obtains more captured video images, it can obtain drawn trail images by drawing more circular and rectangular elements on the canvas in the above manner.

[0090] To achieve a richer visual effect, for example, the trajectory image may be blurred so that the trajectory in the trajectory image has a luminous effect. When performing the blurring process, application 1 generates a corresponding texture based on the canvas and the circular and rectangular elements drawn on the canvas, and then blurs the entire generated texture to achieve the luminous effect. Note that, to achieve the luminous effect in real time, a new texture must be generated and blurred based on the new texture each time new circular and rectangular elements are drawn on the canvas based on newly obtained video images.

[0091] Note that the present disclosure does not limit the specific form of blurring processing, and examples thereof include, but are not limited to, Gaussian blurring, box blurring, and median blurring.

[0092] In addition, the transparency of the trajectory region in the trajectory image (i.e., all circular material regions and rectangular material regions) may be set to a preset transparency value, for example, 50%, 60%, etc. The present disclosure does not limit the magnitude of the preset transparency value.

[0093] Based on the embodiment shown in FIG. 2C , after a user draws a trajectory image, application 1 performs a mirror image process on the trajectory image to obtain a mirror image corresponding to the trajectory image, superimposes the trajectory image on the lower half of the lantern patch, and superimposes the mirror image corresponding to the trajectory image on the upper half of the lantern patch to obtain one target lantern patch.

[0094] After obtaining the target lantern patch, the application 1 may display the target lantern patch on the mobile phone, i.e., jump from the user interface 23 shown in Fig. 2C to the user interface 24 shown in Fig. 2D. Referring to the embodiment shown in Fig. 2D, the application 1 may display the target lantern patch in the upper region of the user interface 24. Of course, the target lantern patch may be displayed in other positions.

[0095] Here, the application 1 may present the target lantern patch in a preset form, for example, by presenting the target lantern patch in a form in which the target lantern patch gradually becomes smaller.

[0096] In this embodiment, the lantern body is made up of eight arcuate surfaces joined together, so by performing copy and mirror processing on the target lantern patch shown in Figure 2D, eight target lantern patches can be obtained, and the above eight target lantern patches can be arranged and presented in a tiling pattern on the display screen.

[0097] In some embodiments, eight target lantern patches may be arranged horizontally in an area near the top of user interface 25, see the embodiment shown in FIG. 2E.

[0098] In some cases, the number of target lantern patches may be large due to the variety of shapes of the lantern body, and due to limitations on the display screen size of a mobile phone, some of the target lantern patches may be presented in a horizontal array in an area near the top of the user interface 25. For example, five target lantern patches are presented in the user interface 25 shown in FIG. 2E.

[0099] After obtaining the eight target lantern patches, the eight target lantern patches may be merged according to the corresponding merge parameters in accordance with the animation form corresponding to the lantern effect to obtain the lantern effect. For example, the eight target lantern patches in a horizontally tiled state may be moved along a preset path to paste the eight target lantern patches onto eight arc surfaces corresponding to the lantern body, or the eight target lantern patches may be merged together into eight arc surfaces corresponding to the lantern body.

[0100] The merge parameters corresponding to the target lantern patch may include parameters such as corresponding path parameters in the three-dimensional space of the target lantern patch and the scaling size of the target lantern patch. The three-dimensional space may be a three-dimensional coordinate system established based on the lantern body. The corresponding path parameters in the target lantern patch may include coordinate values ​​in the three-dimensional coordinate system for each pixel point in the target lantern patch. The path may be composed of multiple discrete points, and for each point on the path, each pixel point in the target lantern patch may correspond to a set of coordinate values.

[0101] In addition, to generate a vivid lantern effect, other elements included in the lantern, such as the upper and lower rings at both ends of the lantern, and lantern garnishes (e.g., fringe devices connected to the lower ring of the lantern), may be merged with the main structure of the lantern during merging.

[0102] Thereafter, application 1 may switch in a preset transition mode, jump from user interface 25 shown in Fig. 2E to user interface 26 shown in Fig. 2F, and display the lantern effect. Here, when presenting the lantern effect, the images on each arc surface of the lantern may be presented in a rotating manner, with the central axis of the spherical body of the lantern as the center of rotation.

[0103] In addition, to enhance the visual expressiveness of the lantern effect, preset filters, display effects (for example, lighting effects), etc. may be added.

[0104] The time length of the rotation presentation may be preset, for example, 3 seconds, 4 seconds, etc., and the video recording may be terminated when the preset rotation presentation time length is reached, but is not limited to this.

[0105] Furthermore, while the lantern effect is being rotated, the central axis of the lantern body may move, for example, moving parallel to the left and right, swinging left and right, or moving back and forth.

[0106] Additionally, a foreground image, filter, text, stamp, etc. matching the lantern effect may be preset. When jumping from user interface 25 to user interface 26, a foreground image matching the target 3D effect may be added to the captured video image, and a filter, text, stamp, etc. matching the target 3D effect may be added to the captured video image. Taking the foreground image as an example, elements included in the foreground image may be distributed mainly on the four edges of the foreground image so as to minimize the obstruction of the facial area of ​​user 1 in the captured video image and the generated lantern effect. For example, in the embodiment shown in FIG. 2F, a foreground image of a lantern element is superimposed on the top of the captured video image, and the words "encounter good luck" are displayed near the bottom. The foreground image, text, stamp, etc. may have animation effects.

[0107] 2A to 2F, the process of obtaining multiple target lantern patches is a 2D animation process, while the subsequent merging of multiple target lantern patches to generate a lantern effect and the presentation of the lantern effect are 3D animation processes. The connection between the 2D animation process and the 3D animation process is realized by a preset transition form, which makes the screen transition natural and smooth, providing a good user experience.

[0108] 4 is a schematic diagram of a configuration of a material presentation device according to an embodiment of the present disclosure. As shown in FIG. 4, the material presentation device 400 according to this embodiment includes an image acquisition module 401, a trajectory generation module 402, a material generation module 403, and a presentation module 404.

[0109] The image acquisition module 401 is for acquiring the captured video images.

[0110] The trajectory generation module 402 is for obtaining a trajectory image based on the position information of the target identification object in the captured video image.

[0111] The material generation module 403 is for obtaining a plurality of target unit effect images based on the trajectory image, and generating a target element based on the plurality of target unit effect images.

[0112] The presentation module 404 is for presenting the target element in the captured video image.

[0113] The body of the target element is a three-dimensional geometric structure including a plurality of geometric surfaces, and the plurality of target unit effect images are pasted onto corresponding geometric surfaces to present the plurality of target unit effect images on the plurality of geometric surfaces.

[0114] In some embodiments, the material generation module 403 is specifically used to superimpose the trajectory image and a material image corresponding to a target element to obtain a unit effect image, and to perform copy processing and mirror processing on the unit effect image to obtain the multiple target unit effect images.

[0115] In some embodiments, the material generation module 403 is specifically used to perform a mirror image process on the trajectory image to obtain a mirror image corresponding to the trajectory image, and to superimpose each of the trajectory image and the mirror image with different areas of the material image corresponding to the target element to obtain the unit effect image.

[0116] In some embodiments, the trajectory generation module 402 is used to: draw two circular regions on a pre-constructed canvas based on the positions of the target identification object in each of two consecutively captured video images; draw a rectangular region on the canvas based on a connecting line between the positions of the target identification object in each of the two consecutively captured video images; the size of the short side of the rectangular region is the same as the diameter of the circular region, the short side of the rectangular region is a side perpendicular to the connecting line, and the center point of the width of the rectangular region overlaps with the center of the corresponding circular region; and continue drawing based on the consecutively captured video images until the trajectory drawing is completed, thereby obtaining the trajectory image.

[0117] In some embodiments, the trajectory generation module 402 is further used to perform blurring on the trajectory image, so that the trajectory in the trajectory image has a lighting effect.

[0118] In some embodiments, the size of the long side of the rectangular area is proportional to the length of the connecting line, and the long side of the rectangular area is the side parallel to the connecting line.

[0119] In some embodiments, the blurring process is a Gaussian blurring process.

[0120] In some embodiments, the material generation module 403 is specifically used to move the plurality of target unit effect images in a tiling state along corresponding paths, and paste the plurality of target unit effect images onto corresponding geometric surfaces to obtain the target material.

[0121] In some embodiments, the presentation module 404 is specifically used to present the target element in a rotational manner in the captured video image, with the rotation axis corresponding to the three-dimensional geometric structure as the rotation center.

[0122] In some embodiments, the rendering module 404 is specifically adapted to render the target element in a subsequently captured video image.

[0123] In some embodiments, the rendering module 404 is further adapted to add at least one of a foreground image, a filter, text, and a stamp that matches the target element to a subsequently captured video image.

[0124] In some embodiments, the target element is a lantern, and accordingly the effect comprising the target element is a lantern effect.

[0125] The material presentation device according to this embodiment may be used to implement the technical solution of any one of the method embodiments described above, and the implementation principles and technical effects are similar. Reference may be made to the detailed description of the method embodiment described above, and for the sake of brevity, no further description will be given here.

[0126] 5 is a schematic diagram of the configuration of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 5, an electronic device 500 according to this embodiment includes a memory 501 and a processor 502.

[0127] Here, the memory 501 may be an independent physical means, and may be connected to the processor 502 via a bus 503. The memory 501 and the processor 502 may be integrated together and realized by hardware or the like.

[0128] The memory 501 is used to store program instructions that the processor 502 can call to perform the operations of any one of the method embodiments described above.

[0129] Alternatively, when a part or all of the methods of the above embodiments are realized by software, the above electronic device 500 may include only the processor 502. A memory 501 for storing a program is located outside the electronic device 500, and the processor 502 is connected to the memory via circuits / wiring to read and execute the program stored in the memory.

[0130] The processor 502 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and an NP.

[0131] The processor 502 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0132] The memory 501 may include volatile memory, such as random-access memory (RAM). The memory may also include non-volatile memory, such as flash memory, a hard disk drive (HDD), or a solid-state drive (SSD). The memory may also include a combination of the above types of memory.

[0133] An embodiment of the present disclosure further provides a readable storage medium including computer program instructions that, when executed by at least one processor of an electronic device, cause the material presentation method described in any one of the method embodiments above to be implemented.

[0134] An embodiment of the present disclosure further provides a computer program product, the program product including a computer program, the computer program being stored in a readable storage medium, at least one processor of the electronic device being capable of reading the computer program from the readable storage medium, and the at least one processor executing the computer program to cause the electronic device to realize the material presentation method described in any one of the method embodiments above.

[0135] It should be noted that, in this specification, relational terms such as "first" and "second" are used solely to distinguish one entity or operation from another, and do not require or imply the existence of any actual relationship or order between those entities or operations. Furthermore, the terms "comprise," "include," or any other variant thereof, indicate a non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements but also other elements not expressly specified or elements inherent in such process, method, article, or device. Absent further limitations, an element defined by "comprises a..." does not exclude the process, method, article, or device that includes said element from also including other identical elements.

[0136] The foregoing are merely specific embodiments of the present disclosure, and are used to enable those skilled in the art to understand or realize the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to these embodiments herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. Acquire a captured video image, and obtain a trajectory image based on position information of a target identification object in the captured video image, by drawing two circular areas on a pre-constructed canvas based on the position of the target identification object in each of two consecutively captured video images, and drawing a rectangular area on the canvas based on a connecting line between the positions of the target identification object in each of the two consecutively captured video images, wherein the size of the short side of the rectangular area is the same as the diameter of the circular area, and the short side of the rectangular area is a side perpendicular to the connecting line, and the center point of the short side of the rectangular area overlaps with the center of a corresponding circular area, and the circular area and the rectangular area belong to a locus; drawing the locus based on the continuously captured video images until the locus drawing is completed to obtain the locus image, wherein the diameter of the circular area in the first video image is preset, and the length of the diameter of the circular area in the subsequent video image of the two continuously captured video images is determined by the length of the line connecting the circular centers of the circular areas in the two continuously captured video images, and the size of the short side of the rectangular area is the same as the length of the diameter of the circular area in the subsequent video image; obtaining a plurality of target unit effect images based on the trajectory image; A material presentation method comprising: generating a target element based on the plurality of target unit effect images; and presenting the target element in a captured video image, wherein the body of the target element is a three-dimensional geometric structure including a plurality of geometric surfaces; and pasting the plurality of target unit effect images onto the corresponding geometric surfaces to present the plurality of target unit effect images on the plurality of geometric surfaces.

2. The above-mentioned method of obtaining a plurality of target unit effect images based on the trajectory image includes: superimposing the trajectory image and a material image corresponding to the target element to obtain a unit effect image; The material presentation method according to claim 1 , further comprising: performing a copy process and a mirror image process on the unit effect image to obtain the plurality of target unit effect images.

3. The above-mentioned method of superimposing the trajectory image and the material image corresponding to the target element to obtain a unit effect image includes: performing a mirror image process on the trajectory image to obtain a mirror image corresponding to the trajectory image; The material presentation method according to claim 2, further comprising superimposing the trajectory image and the mirror image, respectively, on different areas of the material image corresponding to the target element to obtain the unit effect image.

4. The material presentation method according to claim 1 , further comprising: performing a blurring process on the trajectory image so that the trajectory in the trajectory image has a luminous effect.

5. The material presentation method according to claim 1 , wherein the size of the long side of the rectangular area is proportional to the length of the connecting line, and the long side of the rectangular area is a side parallel to the connecting line.

6. The material presentation method according to claim 4 , wherein the blurring process is a Gaussian blurring process.

7. The aforementioned generating a target element based on the plurality of target unit effect images includes: The material presentation method of claim 1, wherein the plurality of target unit effect images in a tiling state are moved along corresponding paths, and the plurality of target unit effect images are pasted onto corresponding geometric surfaces to obtain the target element.

8. The aforementioned presenting the target element in the captured video image may include: The material presentation method according to claim 1 , further comprising presenting the target element in a rotational manner on a captured video image, the rotational axis corresponding to the three-dimensional geometric structure being the center of rotation.

9. The aforementioned presenting the target element in the captured video image may include:

10. A method of presenting material as claimed in claim 1, including presenting said target element in a subsequently captured video image.

10. The method of claim 1 further comprising adding at least one of a foreground image, a filter, text, and a stamp that matches the target element to a subsequently captured video image.

11. an image acquisition module for acquiring the captured video images; A trajectory generation module for obtaining a trajectory image based on position information of a target identification object in the captured video images, the trajectory generation module including: drawing two circular areas on a pre-constructed canvas based on the position of the target identification object in each of two consecutively captured video images; and drawing a rectangular area on the canvas based on a connecting line between the positions of the target identification object in each of the two consecutively captured video images, wherein the size of the short side of the rectangular area is the same as the diameter of the circular area, and the short side of the rectangular area is a side perpendicular to the connecting line; a trajectory generation module, the center point of a short side of a rectangular area overlaps with the center of a corresponding circular area, and the circular area and the rectangular area belong to a trajectory; and the trajectory generation module is configured to draw the trajectory based on the continuously captured video images until the trajectory drawing is completed, thereby obtaining the trajectory image, wherein the diameter of the circular area in the first video image is preset, and the length of the diameter of the circular area in the second video image among the two continuously captured video images is determined by the length of the line connecting the circular centers of the circular areas in the two continuously captured video images, and the size of the short side of the rectangular area is the same as the length of the diameter of the circular area in the second video image; a material generation module for obtaining a plurality of target unit effect images based on the trajectory image, and generating target elements based on the plurality of target unit effect images; a presentation module for presenting the target element in the captured video image; A material presentation device in which the main body of the target element is a three-dimensional geometric structure including a plurality of geometric surfaces, and the plurality of target unit effect images are pasted onto the corresponding geometric surfaces, and the plurality of target unit effect images are presented on the plurality of geometric surfaces.

12. An electronic device including a memory and a processor, the memory configured to store computer program instructions; The processor is configured to execute the computer program instructions to cause the electronic device to implement the material presentation method according to any one of claims 1 to 10.

13. A readable storage medium containing computer program instructions, comprising: A readable storage medium, the computer program instructions being, when executed by at least one processor of an electronic device, causing the electronic device to implement the material presentation method according to any one of claims 1 to 10.

14. A computer program comprising instructions which, when executed by a processor, cause the computer program to implement the material presentation method according to any one of claims 1 to 10.

Images