Material presentation method, device, electronic device, storage medium, and program product

Abstract

The present disclosure relates to a material presentation method, device, electronic device, storage medium, and program product, in which a captured video image is obtained, a trajectory image is obtained based on the position of a target identification object in the video image, and a superimposition process is performed based on the trajectory image and a material image corresponding to a target element to obtain a plurality of target unit effect images. The plurality of target unit effect images are merged to obtain a target element. The main body of the target three-dimensional effect is a three-dimensional geometric structure including a plurality of surfaces, and the plurality of target unit effect images are pasted onto the corresponding geometric surfaces, respectively, to present the plurality of target unit effect images on the plurality of geometric surfaces of the three-dimensional geometric structure.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application is based on and claims priority from an application having application number CN202210089981.6 and filing date January 25, 2022, the entire disclosure of which is incorporated herein by reference in its entirety.

[0002] The present disclosure relates to the field of Internet technology, and in particular to a material presentation method, device, electronic device, storage medium, and program product. [Background technology]

[0003] With the development of Internet technology, the technology of man-machine interaction between smart terminals and users is becoming more and more mature, and there are many man-machine interaction applications. Meanwhile, man-machine interaction in the entertainment scene can bring more joy to users.

[0004] In existing man-machine interaction scenes, applications can provide effects to users so that the users can use and present them. Summary of the Invention

[0005] According to some embodiments of the present disclosure, Acquiring a captured video image, and obtaining a trajectory image based on position information of a target identification object in the captured video image; obtaining a plurality of target unit effect images based on the trajectory image; The present invention provides a material presentation method, which includes generating a target element based on the plurality of target unit effect images, and presenting the target element in a captured video image, wherein a body of the target element is a three-dimensional geometric structure including a plurality of geometric surfaces, and pasting the plurality of target unit effect images onto corresponding geometric surfaces, thereby presenting the plurality of target unit effect images on the plurality of geometric surfaces.

[0006] In some embodiments, obtaining a plurality of target unit effect images based on the trajectory image as described above includes: superimposing the trajectory image and a material image corresponding to the target element to obtain a unit effect image; and performing a copy process and a mirror image process on the unit effect images to obtain the plurality of target unit effect images.

[0007] In some embodiments, the above-mentioned process of superimposing the trajectory image and a material image corresponding to the target element to obtain a unit effect image includes: performing a mirror image process on the trajectory image to obtain a mirror image corresponding to the trajectory image; The method includes superimposing each of the trajectory image and the mirror image on different areas of a material image corresponding to the target material to obtain the unit effect image.

[0008] In some embodiments, the step of obtaining a trajectory image based on position information of a target object in the captured video image includes: Draw two circular regions on a pre-constructed canvas based on the positions of the target identification object in each of two consecutively captured video images; Draw a rectangular area on the canvas based on a connecting line between the positions of the target identification object in each of the two consecutively captured video images, where the size of the short side of the rectangular area is the same as the diameter of the circular area, the short side of the rectangular area is the side perpendicular to the connecting line, and the center point of the width of the rectangular area overlaps with the center of the corresponding circular area; The method further comprises drawing the trajectory based on the continuously captured video images until the trajectory drawing is completed, thereby obtaining the trajectory image.

[0009] In some embodiments, the method further comprises: The method further includes performing a blurring process on the trajectory image so that the trajectory in the trajectory image has a luminous effect.

[0010] In some embodiments, the size of the long side of the rectangular area is proportional to the length of the connecting line, and the long side of the rectangular area is the side parallel to the connecting line.

[0011] In some embodiments, the blurring process is a Gaussian blurring process.

[0012] In some embodiments, the plurality of target unit effect images in a tiling state are moved along corresponding paths, and the plurality of target unit effect images are sequentially pasted onto corresponding geometric surfaces to obtain the target material.

[0013] In some embodiments, presenting the target element in the captured video image as described above comprises: The method includes presenting the target element in a captured video image by rotating it about an axis of rotation corresponding to the three-dimensional geometric structure.

[0014] In some embodiments, the target element is a paper lantern.

[0015] In some embodiments, presenting the target element in a captured video image as described above includes presenting the target element in a subsequently captured video image.

[0016] In some embodiments, the method further includes adding at least one of a foreground image, a filter, text, and a stamp that matches the target element to a subsequently captured video image.

[0017] According to some further embodiments of the present disclosure, an image acquisition module for acquiring the captured video images; a trajectory generation module for obtaining a trajectory image based on position information of a target identification object in the captured video image; a material generation module for obtaining a plurality of target unit effect images based on the trajectory image, and generating target elements based on the plurality of target unit effect images; a presentation module for presenting the target element in a captured video image; A material presentation device is provided in which the main body of the target element is a three-dimensional geometric structure including a plurality of geometric surfaces, and the plurality of target unit effect images are pasted onto corresponding geometric surfaces, respectively, to present the plurality of target unit effect images on the plurality of geometric surfaces.

[0018] According to yet another embodiment of the present disclosure, there is provided an electronic device including a memory and a processor, the memory is configured to store computer program instructions; The processor provides an electronic device configured to execute the computer program instructions to cause the electronic device to implement the method of presenting material according to any of the embodiments described above.

[0019] According to yet another embodiment of the present disclosure, there is provided a readable storage medium containing computer program instructions, comprising: A readable storage medium is provided in which the computer program instructions, when executed by at least one processor of an electronic device, cause the electronic device to implement the method of presenting material of any of the embodiments described above.

[0020] According to yet another embodiment of the present disclosure, there is provided a computer program product which, when executed by a computer, causes the computer to implement the method for presenting material according to any of the embodiments described above.

[0021] According to further embodiments of the present disclosure, there is provided a computer program comprising instructions which, when executed by a processor, implements the method for presenting material according to any of the embodiments described above. [Brief description of the drawings]

[0022] The drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification serve to explain the principles of the present disclosure.

[0023] In order to more clearly describe the technical solutions in the embodiments or prior art of the present disclosure, the following briefly introduces drawings that are required to be used in the description of the embodiments or prior art. Obviously, a person skilled in the art can obtain other drawings based on these drawings without any creative efforts. [Figure 1] 1 is a schematic diagram showing the flow of a material presentation method according to an embodiment of the present disclosure. [Figure 2A] FIG. 13 is an effect presentation diagram according to an embodiment of the present disclosure. [Figure 2B] FIG. 13 is an effect presentation diagram according to an embodiment of the present disclosure. [Figure 2C] FIG. 13 is an effect presentation diagram according to an embodiment of the present disclosure. [Figure 2D] FIG. 13 is an effect presentation diagram according to an embodiment of the present disclosure. [Figure 2E] FIG. 13 is an effect presentation diagram according to an embodiment of the present disclosure. [Figure 2F]FIG. 13 is an effect presentation diagram according to an embodiment of the present disclosure. [Diagram 3] FIG. 13 is a schematic diagram illustrating the effect of a trajectory pattern according to an embodiment of the present disclosure. [Figure 4] 1 is a schematic diagram illustrating the configuration of a material presentation device according to an embodiment of the present disclosure. [Diagram 5] FIG. 1 is a schematic diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] In order to make the above objects, features and advantages of the present disclosure more clearly understandable, the following further describes the aspects of the present disclosure. It should be noted that, unless contradictory, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0025] In order to fully understand the present disclosure, many specific details are described in the following description, but the present disclosure may be embodied in other forms different from those described herein. Apparently, the embodiments in the specification are only some embodiments of the present disclosure, but not all embodiments.

[0026] The material presentation method according to the present disclosure may be realized by a material presentation device according to the present disclosure, and the material presentation device may be realized in any software and / or hardware form. In some embodiments, the material presentation device may be an electronic device such as a tablet, a mobile phone (e.g., a flip phone, a large screen phone, etc.), a wearable device, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the present disclosure does not limit the specific type of electronic device.

[0027] In the following embodiments, the material presentation method according to the present disclosure will be described in detail with reference to drawings and application scenarios, taking electronic devices as an example.

[0028] 1 is a schematic diagram showing a flow of a material presentation method according to some embodiments of the present disclosure. As shown in FIG 1, the method according to the present embodiment includes steps S101 to S104.

[0029] In step S101, a captured video image is acquired, and a trajectory image is obtained based on position information of a target identification object in the captured video image.

[0030] The captured video image is an image currently captured or taken by the electronic device through an image capture module (e.g., a camera) within a suitable image capture range. The present disclosure does not limit the size, clarity, and other parameters of the captured video image.

[0031] Each captured video image may include a target identification object. The present disclosure is not limited to the target identification object. For example, the target identification object may be a part of a human body, such as a finger, a nose, an eye, a mouth, etc. on a face.

[0032] The electronic device may obtain the captured video images, perform image identification on each video image, determine whether each video image includes a target identification object, and obtain location information of the target identification object in each video image. The present disclosure is not limited to an implementation form in which the electronic device performs image identification. For example, the electronic device may use an image identification model to perform image identification on each captured video image. The image identification model may be, but is not limited to, a neural network model, a convolutional network model, or other types of models.

[0033] Of course, the electronic device may obtain position information of the target identification object in each video image by performing image identification on each captured video image in another manner.

[0034] In the following example, a user's face is located within an image collection range corresponding to an image collection module of an electronic device, the image collection module of the electronic device collects images of the person's face, and the target identification object is the nose on the person's face. This example describes how the electronic device performs drawing based on the position information of the target identification object in each video image to obtain a trajectory image.

[0035] After obtaining the captured video images, the electronic device draws one circular material on the canvas constructed in advance based on the position information of the key point corresponding to the nose of the user's face in each video image, and one rectangular material may connect the newly drawn circular material and the circular material drawn based on the previous video image, and the rectangular material and the circular material may overlap. The video images captured by the electronic device are usually captured continuously. Therefore, as the number of captured video images increases, more circular materials and rectangular materials are drawn in real time to dynamically present a trajectory image generated by following the user's movements.

[0036] Here, the position information of the target identification object in each video image may be represented by the coordinates in each video image of the target identification object. A corresponding coordinate system may be constructed according to actual needs. For example, a two-dimensional coordinate system may be constructed by taking the lower left corner of each captured video image as the origin of the coordinate system, the horizontal direction as the horizontal axis, and the vertical direction as the vertical axis. Of course, the two-dimensional coordinate system may be constructed in other forms.

[0037] Here, the canvas can be regarded as a base on which the trajectory image is placed. The present disclosure does not limit the parameters such as the size and color of the canvas. For example, the size of the canvas may be related to the size of each face of the target object included in the target three-dimensional effect, and the color of the canvas may be any color, such as a preset color, for example, white, gray, black, etc.

[0038] When drawing a circular area (also called a circular material) and a rectangular area (also called a rectangular material) on a canvas, in an initial state, the size (e.g., diameter or radius) of the circular material may be a preset value, and then the size of the circular material may change according to the moving speed of the nose in the person's face. The initial state referred to here means the state of the circular material drawn based on the first video image captured. When drawing the circular material and the rectangular material on the canvas, there is a difference in color between the circular material and the rectangular material and the canvas. This allows the drawn trajectory to be clearly presented to the user. For example, the color of the canvas may be black, and the color of the circular material and the rectangular material may be white.

[0039] In some embodiments, when drawing a trajectory image, a trajectory may be presented by a specific camera. Note that the specific camera mentioned here is not a real real camera, but a virtual camera. Before starting to draw the trajectory image, the ClearType of the camera is set to Amaz.CameraClearType.Done. In this way, the screen of the trajectory image of the previous frame drawn is not cleared, and the effect of drawing the trajectory with a paintbrush is displayed. Here, ClearType represents a clear mode corresponding to the virtual camera, and Amaz.CameraClearType.Done represents setting the clear mode of the virtual camera to not clear, i.e., not to clear the contents of the trajectory image drawn previously. Thereby, the drawn trajectory image can be presented to the user.

[0040] S102: Obtain a plurality of target unit effect images based on the trajectory image.

[0041] In some embodiments, the target element is an element included in the effect to be realized. The body of the target element may be a three-dimensional geometric structure, such as a cone structure, a cylinder structure, a prism structure, a spherical structure including multiple arc surfaces, etc. The three-dimensional geometric structure usually includes multiple geometric surfaces, and the target unit effect image is an image to be presented by pasting it onto each geometric surface included in the body of the target element.

[0042] The size, shape and other parameters of the multiple target unit effect images may be the same or different. The size, shape and other parameters of the multiple target unit effect images are related to the subject structure of the target element and are determined according to the specific situation.

[0043] In some embodiments, the material image and the trajectory image corresponding to the target element may be superimposed to obtain a unit effect image, and then the electronic device may perform a copy process and a mirror process on the unit effect image to obtain multiple target unit effect images.

[0044] The superimposition process referred to here means replacing the background of the trajectory image with a material image corresponding to the target element. If the target element is a lantern, the material image corresponding to the target element is an initial material image to be pasted onto each geometric surface of the lantern body, and the initial material image is a material image that is not superimposed on the trajectory image drawn by the user.

[0045] Here, the copy process is a processing form for generating the same image based on the unit effect image. The mirror image process is a processing form for generating an image having an axisymmetric structure with respect to a specific axis based on the unit effect image. The number of times the copy process and the mirror image process are performed may be determined according to the number of geometric surfaces of the three-dimensional geometric structure of the target object.

[0046] In some embodiments, the electronic device may perform a mirror image process on the trajectory image to obtain a mirror image corresponding to the trajectory image, and then superimpose the trajectory image and the mirror image corresponding to the trajectory image on different regions of the material image to obtain a unit effect image. Here, the two regions superimposed on the trajectory image and the mirror image corresponding to the trajectory image in the material image may be in a mirror image state, or the two regions may be understood as being axially symmetrical with respect to a specific axis (e.g., a horizontal axis).

[0047] In some other embodiments, the shapes and sizes of the plurality of geometric surfaces of the body structure of the target element are not all the same. The material images corresponding to the plurality of geometric surfaces of the target element may be pre-stored in the electronic device, and after obtaining the trajectory image, the electronic device may superimpose the trajectory image on the plurality of material images respectively to obtain a plurality of target unit effect images.

[0048] When the trajectory image is superimposed on the material image, the size of the trajectory image may not match the size of the material image. In that case, the size of the trajectory image may be adjusted to match the size of the material image so that the trajectory image fits the material image. For example, if the size of the trajectory image is 720×1280 and the size of the material image is 1024×1024, the size of the trajectory image may be adjusted to 1024×1024, and the adjusted trajectory image may be superimposed on the material image.

[0049] In step S103, a target element is generated based on the multiple target unit effect images, and the body of the target element is a three-dimensional geometric structure including multiple geometric surfaces, and the multiple target unit effect images are pasted onto corresponding geometric surfaces, respectively, to present the multiple target unit effect images on the multiple geometric surfaces.

[0050] In conjunction with the description in step S102, the body of the target element is a three-dimensional geometric structure, and the three-dimensional geometric structure may generally include multiple geometric surfaces. The target unit effect image can be regarded as an image presented on multiple geometric surfaces included in the body of the target element.

[0051] The process of generating a target element based on multiple target unit effect images can be viewed as rotating, translating, scaling, and other operations on the target unit effect images so that the target unit effect images fit the position, size, etc. of the corresponding geometric surface in the main body structure of the target element, and the target unit effect images are displayed pasted onto the geometric surface of a three-dimensional geometric structure.

[0052] The process of merging to generate the target element may also be presented in the form of animation. For example, if the target element is a lantern, a pre-defined lantern model may be adopted to achieve this. Specifically, lines are drawn according to the patch frame structure of the lantern model, a skeleton is created, and corresponding bind skin is performed, and the lantern model is divided to return the lantern model to a single patch state in the reverse direction step by step, and then the animation is played in the reverse direction to present the process of merging to generate the target element. Here, each patch in the lantern model corresponds to one target unit effect image.

[0053] In step S104, a target element is presented in the captured video image.

[0054] After generating the target element in the above manner, a preset presentation form may be adopted to present the target element in the captured video image. The present disclosure is not limited to the preset presentation form. For example, the presentation form may be one or more forms such as rotation presentation, up and down movement, left and right swing, front and back swing, etc., and a specific presentation form may be set according to actual needs.

[0055] In some embodiments, the preset presentation form may present the target element in the captured video image in a rotational manner, with a rotation axis corresponding to the three-dimensional geometric structure of the target element as the rotation center. When presenting in a rotational manner, the rotation axis may swing left and right. For example, if the target element is a lantern, the lantern may be displayed superimposed on the top of the captured video image, and each face of the lantern may be presented in a rotational manner with the central axis of the lantern body as the rotation axis. In addition, while the lantern is rotating, the lantern may further swing left and right. In addition, the display position of the lantern may be set according to needs. For example, the lantern may be presented in an area close to the top of the display screen of the electronic device. This can ensure that the lantern does not block the user's face in the captured video image while the lantern is presented.

[0056] Also, while the target element is being presented in the captured video image, a foreground image that matches the target element may be added to the captured video image, and a filter, character, stamp, etc. that matches the target element may be added to the captured video image. Here, the foreground image, filter, character, stamp, etc. that matches the target element may be pre-arranged.

[0057] Note that the video image for identifying the target identification object and generating the trajectory image and the video image presented superimposed on the target element are different video images. Alternatively, it may be understood that the electronic device identifies the target identification object in real time based on the captured video image while shooting the video clip 1, generates a trajectory image, and further generates the target element based on the trajectory image, and then shoots the video clip 2 in real time and presents the target element in the form of a set animation in the video image included in the video clip 2. The video clip 1 and the video clip 2 may be two video clips shot consecutively, and the time of the video clip 1 is earlier than the time of the video clip 2.

[0058] In the method according to the above embodiment, a captured video image is acquired, a trajectory image is obtained based on the position of the target identification object in each video image, and a superimposition process is performed based on the trajectory image and a material image corresponding to the target element to obtain a plurality of target unit effect images. The plurality of target unit effect images are merged to obtain a target element. The main body of the target element is a three-dimensional geometric structure including a plurality of geometric surfaces, and the plurality of target unit effect images are pasted onto the corresponding geometric surfaces, respectively, to present the plurality of target unit effect images on the plurality of geometric surfaces of the three-dimensional geometric structure. This aspect draws a trajectory based on the user's action, and uses the drawn trajectory as one of the materials for generating the target element, thereby realizing the user's participation in the design of the target element, which is advantageous for improving the user's interaction experience. In addition, the main body of the target element in the effect generated by adopting the above aspect is a three-dimensional structure, which can improve the visual expressiveness of the effect.

[0059] Based on the description of the embodiment shown in Fig. 1, a material presentation method according to the present disclosure will be introduced in conjunction with the scenes and drawings shown in Fig. 2A to Fig. 2F, taking as an example a case where the electronic device is a mobile phone and a video editing application (hereinafter, abbreviated as Application 1) is installed on the mobile phone. Here, in the embodiment shown in Fig. 2A to Fig. 2F, a process of drawing a trajectory image, generating a lantern effect, and presenting the lantern effect will be described, taking as an example a case where the target three-dimensional effect is a lantern effect, that is, the target element is a lantern, and the target identification object is the tip of the nose on a person's face.

[0060] Here, when a user triggers an icon corresponding to the lantern effect, the application 1 may exemplarily display a user interface 21 as shown in FIG. 2A on a mobile phone, and display prompt information on the user interface 21 to prompt the user to start drawing a lantern. The prompt information may be realized in one or more forms, such as text, animation, voice, etc., and the present disclosure does not limit any display parameters, such as font, size, color, etc. of the text. In some embodiments, as shown in FIG. 2A, a prompt text content of "draw a lantern" may be presented in an area near the top of the user interface 21, and a lantern animation may be presented while the prompt text content is presented. In addition, the presentation time length of the prompt text content may be preset, for example, 1 second, 2 seconds, etc.

[0061] When the presentation time length of the prompt text content reaches a preset presentation time length, the application 1 may identify a nose tip portion of the face of the user 1, and display a paintbrush at the nose tip portion of the user 1. For example, in the embodiment shown in FIG. 2B, the paintbrush is displayed at the nose tip portion of the user 1. The present disclosure is not limited to the style of the paintbrush.

[0062] In this embodiment, the body of the lantern is spherical, the vertical central axis of the sphere is the axis of rotation, and the circular arc surfaces surrounding the axis of rotation at 45 degrees each are one geometric surface. When presenting the paintbrush, a lantern patch corresponding to the circular arc surface may be presented superimposed on top of the captured video image. The lantern patch is a material image corresponding to the lantern effect.

[0063] In this embodiment, the arc surfaces corresponding to 45 degrees each are one geometric surface, and the eight geometric surfaces surrounding the central axis of the sphere have the same shape and size, so that the application 1 can display the lantern patch. This allows the user to experience the scene of drawing the lantern patch, improving the user experience.

[0064] In some possible cases, the application 1 may present a part of the lantern patch, for example, the lower half of the lantern patch (of course, the upper half of the lantern patch may be presented). When this form is adopted and realized, the part of the lantern patch that is not presented may be masked and hidden, as in the embodiment shown in Fig. 2B. In the embodiment shown in Fig. 2B, the lower half of the lantern patch presented on the user interface 22 may be regarded as the effective drawing area of ​​the trajectory image.

[0065] When this embodiment is adopted, after the user draws a trajectory pattern in the lower half area of ​​the lantern patch, the application 1 may perform a mirror image process on the drawn trajectory image to obtain a mirror image corresponding to the trajectory image, and superimpose the mirror image corresponding to the trajectory image on the upper half of the lantern patch (or the lower half of the lantern patch). The target lantern patch pattern obtained by adopting this embodiment has a top-bottom symmetric effect, which can provide the user with rich visual effects and enhance the expressiveness of the lantern effect.

[0066] In some embodiments, the application 1 may present a complete lantern patch. The user can draw a trajectory image in an area corresponding to the entire lantern patch. This form is simple and can be quickly realized by an electronic device. In this case, the complete lantern patch presented by the application 1 is the effective drawing area of ​​the trajectory image.

[0067] In some embodiments, the application 1 may present a single trajectory drawing area to the user without presenting a lantern patch to the user. The trajectory drawing area may not be related to the shape of the lantern patch.

[0068] In some embodiments, application 1 may present a complete lantern patch. The user may draw in an area corresponding to the entire lantern patch. The electronic device may record only the trajectory that the user draws in the lower half (or upper half) of the lantern patch.

[0069] In some cases, the time length for drawing the trajectory may be limited. To allow the user to understand the remaining time length for drawing the trajectory, the remaining time length for drawing the trajectory may be presented in the form of a progress bar in the user interface. For example, still referring to FIG. 2B, the progress bar may be presented in an area near the top of the user interface 22. To ensure the user's experience of drawing the lantern pattern, the progress bar may not obstruct the face area of ​​the user 1 and the lower half area of ​​the lantern patch in the captured video image as much as possible.

[0070] Then, the user 1 may move the nose tip area by moving the face region. The application 1 displays the drawn trajectory image on the mobile phone, following the movement of the nose tip area of ​​the user 1. In some embodiments, the trajectory image drawn by the user 1 is as shown in FIG. 2C.

[0071] In some possible cases, the drawing time length may be preset. The present disclosure is not limited to the time length. For example, the drawing time length may be 5 seconds, 6 seconds, 7 seconds, etc., and may be set according to the actual situation.

[0072] In conjunction with the embodiment shown in FIG. 2B, the lower half of the lantern patch is the effective drawing area. While the user 1 is moving the face, the nose tip may move out of the lower half of the lantern patch, i.e., out of the effective drawing area. The application 1 may not record the trajectory outside the effective drawing area, but may record only the drawing trajectory within the effective drawing area.

[0073] The following describes how to draw a trajectory using the example shown in FIG.

[0074] With reference to the embodiment shown in FIG. 3, when the application 1 obtains the first frame of the captured video image, identifies and obtains the position of the nose tip of the user 1 in the first frame of the video image, the application 1 draws a circular material s1 on a canvas constructed in advance. The position of the circular material s1 on the canvas may be determined based on the position of the nose tip of the user 1 in the first frame of the video image, and the diameter of the circular material s1 may be a preset value. When the application 1 obtains the second frame of the captured video image, identifies and obtains the position of the nose tip of the user 1 in the second frame of the video image, the application 1 draws a circular material s2 and a rectangular material r1 connecting the circular material s1 and the circular material s2 on the canvas. Here, the diameter of the circular material s2 may be determined by the length of the line connecting the center of the circle of the circular material s1 and the center of the circle of the circular material s2. The longer the length of the connecting line, the thinner the thickness of the drawn trajectory, and the shorter the length of the connecting line, the thicker the thickness of the drawn trajectory.

[0075] By calculating the dwell time at the tip of the nose and controlling the diameter of the pencil brush (i.e., the diameter of the circular material), the thickness of the paintbrush can be changed, achieving a touch effect that is closer to that of an actual painting.

[0076] Let len ​​represent the distance of the connecting line between the positions of the nose tip of user 1 in the video images of two adjacent frames, Y and basescale represent the first and second parameters related to the size of the circular material, respectively, and spotX represent the size of the circular material. Since the size of the side perpendicular to the drawing direction of the rectangular material is the same as that of the circular material, determining the size of the circular material is equivalent to simultaneously determining the size of the rectangular material.

[0077] In some embodiments, the size of the circular blank may be determined according to the following scheme.

[0078] Step a: The distance len of the connecting line between the positions of the nose tip of the user 1 in the video images of two adjacent frames is substituted into the inverse proportional function "Y=A1-len" to obtain the value of the first parameter Y. Here, A1 is a preset constant, for example, A1=0.04.

[0079] If Y is greater than the preset value A2, the value of the second parameter basescale is determined in step b. If Y is equal to or less than the preset value A2, the value of the second parameter basescale is determined in step c. For example, A2=0.01.

[0080] Step b: Determine the value of the second parameter basescale by the formula basescale=min(basescale+a1*Y,Y).

[0081] Step c: Determine the value of the second parameter basescale by the formula basescale=max(basescale-a2*Y,Y).

[0082] Here, the above-mentioned a1 and a2 are preset constants, for example, a1 = 0.02, a2 ​​= 0.01. The magnitudes of the values ​​of a1 and a2 mainly affect the change in the thickness of the drawn trajectory, so the magnitudes of the values ​​of a1 and a2 may be set according to the actual situation.

[0083] Step d: The size of spotX is determined by the formula spotX=max(a3,min(basescale,a4)).

[0084] Here, a3 and a4 are the minimum and maximum values ​​of parameters relating to the size of the circular material that are set in advance, for example, a3=0.003, a4=0.04.

[0085] After determining the size of spotX, the size of the circular material (for example, diameter or radius) can be obtained by multiplying spotX by a preset size of the circular material.

[0086] In the above steps a to d, the parameters a1 to a4 can be regarded as, for example, enlargement / reduction ratios corresponding to the circular material.

[0087] In conjunction with the formulas shown in steps a to d above, the present disclosure prevents the thickness of the drawn path from changing suddenly, and achieves the effect of the path thickness changing gradually, softly, and naturally, by using a preset value A2 as a judgment condition, and when Y exceeds A2, the size of the circular material is increased by a factor of 0.02 (i.e., a1), and the upper limit is Y. When Y is equal to or less than A2, the size of the circular material is decreased by a factor of 0.01 (i.e., a2), and the lower limit is Y.

[0088] As is clear from the introduction of the embodiment shown in Fig. 1, the application 1 can present a trajectory drawn by a specific camera. Therefore, the application 1 captures a third frame of video image, and draws a circular material s3 and a rectangular material r2 based on the third frame of video image, and the previously drawn circular material s1, circular material s2, and rectangular material r1 are all left behind and are not cleared, so that the application 1 can present the drawing process to the user.

[0089] Similarly, as the application 1 obtains more captured video images, it can obtain drawn trail images by drawing more circular and rectangular elements onto the canvas in the above manner.

[0090] In order to obtain a richer visual effect, for example, the trajectory image may be blurred so that the trajectory in the trajectory image has a luminous effect. When performing the blurring process, the application 1 generates a suitable texture based on the canvas and the circular and rectangular materials drawn on the canvas, and performs a blurring process on the entire generated texture, thereby realizing the luminous effect. Note that, in order to realize the luminous effect in real time, it is necessary to generate a new texture and perform a blurring process based on the new texture every time a new circular and rectangular material is drawn on the canvas based on a newly obtained video image.

[0091] Note that the present disclosure does not limit the specific form of the blurring process, and may be, for example, Gaussian blurring, box blurring, median blurring, or the like, but is not limited to these.

[0092] Also, the transparency of the trajectory region (i.e., all circular material regions and rectangular material regions) in the trajectory image may be set to a preset transparency value, for example, 50%, 60%, etc. The present disclosure does not limit the magnitude of the preset transparency value.

[0093] Based on the embodiment shown in FIG. 2C , after a user draws a trajectory image, application 1 performs a mirror image process on the trajectory image to obtain a mirror image corresponding to the trajectory image, superimposes the trajectory image on the lower half of the lantern patch, and superimposes the mirror image corresponding to the trajectory image on the upper half of the lantern patch to obtain one target lantern patch.

[0094] After obtaining the target lantern patch, the application 1 may display the target lantern patch on the mobile phone, i.e., jump from the user interface 23 shown in Fig. 2C to the user interface 24 shown in Fig. 2D. Referring to the embodiment shown in Fig. 2D, the application 1 may display the target lantern patch in the upper region of the user interface 24. Of course, the target lantern patch may be displayed in other positions.

[0095] Here, the application 1 may present the target lantern patch in a preset form, for example, the target lantern patch may be presented in a form in which the target lantern patch becomes gradually smaller.

[0096] In this embodiment, the lantern body is made up of eight arcuate surfaces connected together, so by performing copy and mirror processes on the target lantern patch presented in Figure 2D, eight target lantern patches can be obtained, and the above eight target lantern patches can be arranged and presented in a tiling pattern on the display screen.

[0097] In some embodiments, eight target lantern patches may be arranged horizontally in an area near the top of user interface 25, see the embodiment shown in FIG. 2E.

[0098] In some cases, the number of target lantern patches may be large due to the variety of shapes of the lantern body, and due to the constraints of the display screen size of a mobile phone, some of the target lantern patches may be presented in a horizontal array in an area near the top of the user interface 25. For example, five target lantern patches are presented in the user interface 25 shown in FIG. 2E.

[0099] After obtaining the eight target lantern patches, the eight target lantern patches may be merged according to the corresponding merge parameters according to the animation form corresponding to the lantern effect to obtain the lantern effect. For example, the eight target lantern patches in the horizontally tiled state may be moved along a preset path to paste the eight target lantern patches onto the eight arc surfaces corresponding to the lantern body, or the eight target lantern patches may be merged together into the eight arc surfaces corresponding to the lantern body.

[0100] The merge parameters corresponding to the target lantern patch may include parameters such as corresponding path parameters in the 3D space of the target lantern patch, the scaled size of the target lantern patch, etc. The 3D space may be a 3D coordinate system constructed based on the lantern body. The corresponding path parameters in the target lantern patch may include coordinate values ​​in the 3D coordinate system of each pixel point in the target lantern patch. The path may be composed of multiple discrete points, and for each point on the path, each pixel point in the target lantern patch may correspond to a set of coordinate values.

[0101] In addition, in order to generate a vivid lantern effect, during merging, other elements contained in the lantern, such as the upper and lower wheels at both ends of the lantern, and the lantern's garnish (e.g., a fringe device connected to the lower wheel of the lantern), may also be merged with the main structure of the lantern.

[0102] Thereafter, the application 1 may switch in a preset transition form, jump from the user interface 25 shown in Fig. 2E to the user interface 26 shown in Fig. 2F, and display the lantern effect. Here, when presenting the lantern effect, the images on each circular arc surface of the lantern may be presented in a rotating manner, with the central axis of the spherical body of the lantern as the center of rotation.

[0103] In addition, in order to enhance the visual expressiveness of the lantern effect, preset filters, display effects (for example, lighting effects), etc. may be added.

[0104] In addition, the time length of the rotation presentation may be preset. For example, the time length of the rotation presentation may be set to 3 seconds, 4 seconds, etc., and when the preset rotation presentation time length is reached, the video recording may be set to end, but is not limited to this.

[0105] In addition, while the lantern effect is being rotated and presented, the central axis of the lantern body may move, for example, moving parallel to the left and right, swinging from side to side, or moving back and forth.

[0106] Also, a foreground image, a filter, a character, a stamp, etc., matching the lantern effect may be set in advance. When jumping from the user interface 25 to the user interface 26, a foreground image matching the target three-dimensional effect may be added to the captured video image, and a filter, a character, a stamp, etc., matching the target three-dimensional effect may be added to the captured video image. Taking the foreground image as an example, the elements included in the foreground image may be distributed mainly at the four peripheral positions of the foreground image so as not to obstruct the face area of ​​the user 1 in the captured video image as much as possible and not to obstruct the generated lantern effect as much as possible. For example, in the embodiment shown in FIG. 2F, a foreground image of a lantern element is superimposed on the top of the captured video image, and the character "encounter good luck" is displayed in the area near the bottom, and the foreground image, the character, the stamp, etc. may have an animation effect.

[0107] As can be seen from the above-mentioned embodiment shown in Figures 2A to 2F, the process of obtaining multiple target lantern patches is a two-dimensional animation process, while later merging multiple target lantern patches to generate a lantern effect and presenting the lantern effect is a three-dimensional animation process. The connection between the 2D animation process and the 3D animation process is realized by a preset transition form, so that the screen switching is natural and smooth, and can bring a good experience to the user.

[0108] 4 is a schematic diagram of a configuration of a material presentation device according to an embodiment of the present disclosure. As shown in FIG. 4, the material presentation device 400 according to the embodiment includes an image acquisition module 401, a trajectory generation module 402, a material generation module 403, and a presentation module 404.

[0109] The image acquisition module 401 is for acquiring the captured video images.

[0110] The trajectory generation module 402 is for obtaining a trajectory image based on the position information of the target identification object in the captured video image.

[0111] The material generation module 403 is for obtaining a plurality of target unit effect images based on the trajectory image, and generating a target element based on the plurality of target unit effect images.

[0112] The presentation module 404 is for presenting the target element in the captured video image.

[0113] The body of the target element is a three-dimensional geometric structure including a plurality of geometric surfaces, and the plurality of target unit effect images are pasted onto corresponding geometric surfaces to present the plurality of target unit effect images on the plurality of geometric surfaces.

[0114] In some embodiments, the material generation module 403 is specifically used for superimposing the trajectory image and a material image corresponding to a target element to obtain a unit effect image, and for performing copy processing and mirror processing on the unit effect image to obtain the multiple target unit effect images.

[0115] In some embodiments, the material generation module 403 is specifically used for performing a mirror image process on the trajectory image to obtain a mirror image corresponding to the trajectory image, and for superimposing each of the trajectory image and the mirror image with different areas of the material image corresponding to the target element to obtain the unit effect image.

[0116] In some embodiments, the trajectory generation module 402 is used to draw two circular areas on a pre-constructed canvas based on the positions of the target identification object in each of two consecutively taken video images, and draw a rectangular area on the canvas based on a connecting line between the positions of the target identification object in each of two consecutively taken video images, where the size of the short side of the rectangular area is the same as the diameter of the circular area, the short side of the rectangular area is a side perpendicular to the connecting line, and the center point of the width of the rectangular area overlaps with the center of the corresponding circular area, and continue drawing based on the consecutively taken video images until the trajectory drawing is completed, to obtain the trajectory image.

[0117] In some embodiments, the trajectory generation module 402 is further used to perform a blurring process on the trajectory image, so that the trajectory in the trajectory image has a lighting effect.

[0118] In some embodiments, the size of the long side of the rectangular area is proportional to the length of the connecting line, and the long side of the rectangular area is the side parallel to the connecting line.

[0119] In some embodiments, the blurring process is a Gaussian blurring process.

[0120] In some embodiments, the material generation module 403 is specifically used for moving the multiple target unit effect images in a tiling state along corresponding paths, and pasting the multiple target unit effect images onto corresponding geometric surfaces to obtain the target material.

[0121] In some embodiments, the presentation module 404 is specifically used to present the target element in a rotational manner in the captured video image around a rotation axis corresponding to the three-dimensional geometric structure.

[0122] In some embodiments, the rendering module 404 is specifically adapted to render the target element in a subsequently captured video image.

[0123] In some embodiments, the rendering module 404 is further adapted to add at least one of a foreground image, a filter, text, and a stamp that matches the target element to a subsequently captured video image.

[0124] In some embodiments, the target element is a lantern and accordingly the effect comprising the target element is a lantern effect.

[0125] The material presentation device according to this embodiment may be used to implement the technical solution of any one of the method embodiments described above, and the implementation principles and technical effects are similar, so reference may be made to the detailed description of the method embodiments described above, and for the sake of brevity, no further description will be given here.

[0126] 5 is a schematic diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 5, an electronic device 500 according to this embodiment includes a memory 501 and a processor 502.

[0127] Here, the memory 501 may be an independent physical means, and may be connected to the processor 502 via a bus 503. The memory 501 and the processor 502 may be integrated together and realized by hardware or the like.

[0128] The memory 501 is used to store program instructions that the processor 502 can call to perform the operations of any one of the method embodiments described above.

[0129] Alternatively, when a part or all of the methods in the above embodiments are realized by software, the above electronic device 500 may include only the processor 502. A memory 501 for storing a program is external to the electronic device 500, and the processor 502 is connected to the memory via a circuit / wiring to read and execute the program stored in the memory.

[0130] The processor 502 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and a NP.

[0131] The processor 502 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0132] The memory 501 may include volatile memory, such as random-access memory (RAM). The memory may also include non-volatile memory, such as flash memory, a hard disk drive (HDD) or a solid-state drive (SSD). The memory may also include a combination of the above types of memory.

[0133] An embodiment of the present disclosure further provides a readable storage medium including computer program instructions which, when executed by at least one processor of an electronic device, cause the material presentation method described in any one of the method embodiments above to be implemented.

[0134] An embodiment of the present disclosure further provides a computer program product, the program product including a computer program, the computer program being stored in a readable storage medium, at least one processor of the electronic device being capable of reading the computer program from the readable storage medium, and the at least one processor executing the computer program to cause the electronic device to realize the material presentation method described in any one of the method embodiments above.

[0135] It should be noted that, in this specification, relational terms such as "first" and "second" are used solely for the purpose of distinguishing one entity or operation from another entity or operation, and do not require or imply that such an actual relationship or sequence exists between the entities or operations. Furthermore, the terms "comprise", "include", or any other variation thereof, are intended to indicate a non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements, but also other elements not expressly stated or that are inherent in such process, method, article, or device. Absent more limitations, an element limited by "comprises a..." does not exclude the inclusion of other identical elements in the process, method, article, or device that includes the element.

[0136] The above are merely specific embodiments of the present disclosure, and are used to enable those skilled in the art to understand or realize the present disclosure. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not limited to these embodiments herein, but has the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. Acquiring a captured video image, and obtaining a trajectory image based on position information of a target identification object in the captured video image; obtaining a plurality of target unit effect images based on the trajectory image; A material presentation method comprising: generating a target element based on the plurality of target unit effect images; and presenting the target element in a captured video image, wherein a body of the target element is a three-dimensional geometric structure including a plurality of geometric surfaces; and pasting the plurality of target unit effect images onto corresponding geometric surfaces to present the plurality of target unit effect images on the plurality of geometric surfaces.

2. The above-mentioned method of obtaining a plurality of target unit effect images based on the trajectory image includes: superimposing the trajectory image and a material image corresponding to the target element to obtain a unit effect image; The material presentation method according to claim 1 , further comprising: performing a copy process and a mirror image process on the unit effect image to obtain the plurality of target unit effect images.

3. The above-mentioned trajectory image and the material image corresponding to the target element are superimposed to obtain a unit effect image, performing a mirror image process on the trajectory image to obtain a mirror image corresponding to the trajectory image; The material presentation method according to claim 2 , further comprising superimposing each of the trajectory image and the mirror image on different areas of the material image corresponding to the target element to obtain the unit effect image.

4. The above-mentioned method of obtaining a trajectory image based on the position information of the target object in the captured video image includes the steps of: Draw two circular regions on a pre-constructed canvas based on the positions of the target identification object in each of two consecutively captured video images; Draw a rectangular area on the canvas based on a connecting line between the positions of the target identification object in each of the two consecutively captured video images, where the size of the short side of the rectangular area is the same as the diameter of the circular area, the short side of the rectangular area is the side perpendicular to the connecting line, and the center point of the width of the rectangular area overlaps with the center of the corresponding circular area; 2. The material presentation method according to claim 1, further comprising: drawing the path based on continuously captured video images until the path drawing is completed, thereby obtaining the path image.

5. The material presentation method according to claim 4 , further comprising performing a blurring process on the trajectory image so that the trajectory in the trajectory image has a luminous effect.

6. The material presentation method according to claim 4 , wherein a size of a long side of the rectangular area is proportional to a length of the connecting line, and the long side of the rectangular area is a side parallel to the connecting line.

7. The material presentation method according to claim 5 , wherein the blurring process is a Gaussian blurring process.

8. The above-mentioned generating a target element based on the plurality of target unit effect images includes: The material presentation method according to claim 1 , further comprising the steps of: moving the plurality of target unit effect images in a tiling state along corresponding paths; pasting the plurality of target unit effect images onto corresponding geometric surfaces to obtain the target material.

9. The step of presenting the target element in the captured video image as described above further comprises: The method of presenting material according to claim 1 , further comprising presenting the target element in a rotated manner on a captured video image, the rotation axis corresponding to the three-dimensional geometric structure being the center of rotation.

10. The step of presenting the target element in the captured video image as described above further comprises:

10. A method of presenting material as claimed in claim 1, further comprising presenting said target element in a subsequently captured video image.

11. 2. The method of claim 1, further comprising adding at least one of a foreground image, a filter, text, and a stamp that matches the target element to a subsequently captured video image.

12. an image acquisition module for acquiring the captured video images; a trajectory generation module for obtaining a trajectory image based on position information of a target identification object in the captured video image; a material generation module for obtaining a plurality of target unit effect images based on the trajectory image, and generating target elements based on the plurality of target unit effect images; a presentation module for presenting the target element in a captured video image; A material presentation device in which the main body of the target element is a three-dimensional geometric structure including a plurality of geometric surfaces, and the plurality of target unit effect images are pasted onto corresponding geometric surfaces, thereby presenting the plurality of target unit effect images on the plurality of geometric surfaces.

13. An electronic device including a memory and a processor, the memory is configured to store computer program instructions; The processor is configured to execute the computer program instructions to cause the electronic device to implement the material presentation method according to any one of claims 1 to 11.

14. A readable storage medium containing computer program instructions, comprising: A readable storage medium, the computer program instructions being executed by at least one processor of an electronic device, causing the electronic device to implement the material presentation method according to any one of claims 1 to 11.

15. A computer program product which, when executed by a computer, causes the computer to implement the material presentation method according to any one of claims 1 to 11.

16. A computer program comprising instructions which, when executed by a processor, causes the implementation of the method for presenting material according to any one of claims 1 to 11.

Images