A method and device for implementing a real-time image breaking special effect and a storage medium

Abstract

The application relates to the technical field of image processing, and discloses a real-time image breaking special effect implementation method and device and a storage medium, which comprises the following steps: calculating vertex positions of image breaking according to different requirements, then determining vertex indexes of each piece of fragment based on the vertex positions and a fragment thickness, calculating a motion state of each piece of fragment by exerting an external force, and finally updating the state of the fragment and real-time rendering the result, so that the breaking special effect of the image can be real-time rendered, the modern programmable pipeline of a graphic processing unit (GPU) is used to accelerate the calculation and rendering of the image, the processing efficiency of the image breaking special effect is greatly improved, the real-time processing can be realized in a mobile device, the shape and quantity of the fragment can be selected, the content of the breaking effect is more abundant, the breaking special effect generation process is fast and smooth, the effect is real, and the physical feeling is strong.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus and storage medium for implementing real-time image fragmentation effects. Background Technology

[0002] Thanks to the rapid development of the short video industry, users are paying more and more attention to the diversity and realism of video editing and image effects. Image shattering effect is a type of image effect that makes the input image appear to be broken into pieces, which is a simulation of physical shattering.

[0003] Currently, the implementation of image fragmentation effects is relatively simple, with only one fragment shape supported. However, the real-time performance and realism of special effects rendering are important aspects of video image processing. Existing special effects have numerous parameters that are difficult to adjust, and the rendering is slow and unusable. They are also relatively monotonous in terms of content, lacking a variety of variable fragment shapes to choose from, have poor physical simulation, weak realism, and a poor overall experience. Summary of the Invention

[0004] The purpose of this application is to overcome the shortcomings of the prior art and provide a method, apparatus and storage medium for implementing real-time image shattering effects.

[0005] Firstly, a method for implementing real-time image fragmentation effects is provided, including:

[0006] S101. Obtain the original image, the fragment shape, fragment quantity, and fragment thickness required by the user;

[0007] S102. Calculate the vertex position of the fragment based on its shape and number;

[0008] S103. Calculate the rendering vertex index of the fragment based on the vertex position and fragment thickness;

[0009] S104. Calculate the motion state of the fragment's vertex after it is subjected to force based on the applied external force.

[0010] S105. Obtain parameter adjustment data, update the fragment vertices according to the adjusted parameters, and render in real time;

[0011] S106. Generate and output image or video with image shattering effects.

[0012] Furthermore, the shape of the fragments includes, but is not limited to, rectangles, rhombuses, triangles, or random polygons.

[0013] Furthermore, when calculating the vertex positions of the fragments, the length and width of the image are normalized to [0,1], and the image is divided into M*N small fragments. The number of vertices in each fragment is determined, and the normalized vertex coordinates of each fragment are calculated. The vertices of the random polygons are the vertices of the convex polygons calculated using the Voronoi diagram.

[0014] Furthermore, when calculating the rendering vertex indices of the fragment, the rendering indices of the surface triangles for the front, back, and multiple sides of the fragment's outer surface are as follows:

[0015] Front triangle: {i,i+1,i+2}, {i,i+2,i+3}…{i,i+L-3,i+L-2}, {i,i+L-2,i+L-1}

[0016] Back triangle: {i+n,i+n+1,i+n+2}, {i+n,i+n+2,i+n+3}…

[0017] {i+n,i+n+L-3,i+n+L-2}, {i+n,i+n+L-2,i+n+L-1}

[0018] Lateral triangle: {i,i+n+1,i+n},{i,i+n+1,i+1};

[0019] …

[0020] {i+t,i+t+n+1,i+t+n}, {i+t,i+t+n+1,i+t+1,};

[0021] …

[0022] {i+L-1,i+n,i+L-1+n},{i+L-1,i+n,i}

[0023] Where i represents the starting vertex of the current fragment, n represents the number of fragments, L represents the number of edges of the polygon, and t represents the edge number of the fragment.

[0024] Furthermore, the motion state of the apex of the fragment after being subjected to force includes the velocity and rotation angle of the apex.

[0025] Furthermore, the parameter adjustment data includes, but is not limited to, fragment shape, fragment quantity, fragment thickness, force magnitude, and fragment movement duration.

[0026] Furthermore, parameter adjustment data is acquired, and the fragment vertices are updated and rendered in real time based on the adjusted parameters, including:

[0027] Adjust the parameters and determine whether the position of the fragment vertex changes and whether the applied external force changes;

[0028] If the vertex position changes, return to step S102;

[0029] If the vertex position remains unchanged but the applied external force changes, return to step S104.

[0030] If the vertex position remains unchanged and the applied external force remains unchanged, the vertex position, vertex index, and adjustment parameters are input to the vertex shader. The fragment vertex position of the current frame is calculated based on the initial state and adjustment parameters and in accordance with Newton's second law. The fragment is then rendered by the fragment shader, and step S106 is executed.

[0031] Secondly, a device for implementing real-time image fragmentation effects is provided, comprising:

[0032] The acquisition module is used to acquire the original image, the fragment shape, number of fragments, and fragment thickness required by the user;

[0033] The vertex position calculation module is used to calculate the vertex position of the fragments based on their shape and number.

[0034] The index calculation module is used to calculate the rendering vertex index of the fragment based on the vertex position and fragment thickness.

[0035] The motion state technology module is used to calculate the motion state of the fragment's apex after it is subjected to external force.

[0036] The update and rendering module is used to acquire parameter adjustment data and update and render the fragment vertices based on the adjusted parameters.

[0037] The output module is used to generate and output images or videos with image shattering effects.

[0038] Thirdly, a computer-readable storage medium is provided that stores program code for execution by a device, the program code including steps for performing a method as described in any implementation of the first aspect.

[0039] Fourthly, an electronic device is provided, the electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the method as in any of the implementations of the first aspect.

[0040] This application has the following beneficial effects: the method can render image shattering effects in real time, accelerate image calculation and rendering in the GPU using the programmable pipeline of modern graphics processors, greatly improve the processing efficiency of image shattering effects, can process in real time on mobile devices, and can also select the shape and number of fragments, thereby making the content of the shattering effect richer, and the shattering effect generation process is fast and smooth, with realistic effects and strong physical sense. Attached Figure Description

[0041] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an undue limitation of this application.

[0042] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart of the method for implementing the real-time image fragmentation effect in Embodiment 1 of this application;

[0044] Figure 2 This is a schematic diagram of the surface triangle vertex index in the method for implementing the real-time image shattering effect in Embodiment 1 of this application, taking a rectangle as an example;

[0045] Figure 3 This is a front view of the force simulation of fragments in the method for implementing real-time image fragmentation effects according to Embodiment 1 of this application;

[0046] Figure 4 This is a top view of the stress simulation of fragments in the method for implementing the real-time image fragmentation effect in Embodiment 1 of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Example 1

[0049] The method for implementing a real-time image shattering effect according to Embodiment 1 of this application includes: S101, acquiring the original image, the desired fragment shape, number of fragments, and fragment thickness; S102, calculating the vertex positions of the fragments based on the fragment shape and number of fragments; S103, calculating the rendering vertex index of the fragments based on the vertex positions and fragment thickness; S104, calculating the motion state of the vertices of the fragments after being subjected to force based on the applied external force; S105, acquiring parameter adjustment data, updating the fragment vertices based on the adjusted parameters, and rendering in real time; S106, generating and outputting an image or video with the image shattering effect. This method can render the image shattering effect in real time, utilizes the programmable pipeline of modern graphics processors to accelerate the calculation and rendering of images in the GPU, greatly improving the processing efficiency of the image shattering effect. It can be processed in real time on mobile devices, and the shape and number of fragments can be selected, thus making the content presented by the shattering effect richer. Moreover, the shattering effect generation process is fast and smooth, the effect is realistic, and the physical sense is strong.

[0050] Specifically, Figure 1 The flowchart illustrates the implementation method of the real-time image shattering effect in Embodiment 1 of the application. This implementation method relies on rendering engines such as OpenGL and uses them for parallel computation. First, the vertex positions of the image shattering are calculated according to different requirements. Then, based on the vertex positions and fragment thickness, the vertex index of each fragment is determined. Next, the motion speed and rotation angle of each fragment are calculated through the applied external force. Finally, the state of the fragments is updated and the result is rendered in real time. Specifically, it includes:

[0051] S101. Obtain the original image, the fragment shape, fragment quantity, and fragment thickness required by the user;

[0052] The original image is uploaded by the user, or the user can select an image from a specified image database as the original image. The shape of the fragments includes, but is not limited to, rectangles, rhombuses, triangles, or random polygons. It should be noted that in a single shattering effect, only fragments of the same shape can be formed, that is, all the broken fragments have the same shape. For example, all the fragments are rectangles. In a single shattering effect, fragments of multiple shapes can also be formed, that is, the broken fragments contain at least two shapes. For example, there are three shapes of fragments in a single shattering effect: triangles, rectangles, and rhombuses. Of course, there can also be only two shapes of fragments: triangles and rectangles.

[0053] S102. Calculate the vertex position of the fragment based on its shape and number;

[0054] Specifically, based on the user-selected fragment shape and number of fragments, the vertex positions of the broken fragments are calculated. Fragment shapes include, but are not limited to, rectangles, rhombuses, triangles, and random polygons. Different shapes have different vertex position calculation formulas. Taking a rectangular fragment shape as an example, first, the image's length and width are normalized to [0,1]. Then, the image is divided into M*N small rectangles. Each fragment has 4 vertices. The vertex positions of the fragment in the i-th column and j-th row are as follows:

[0055] P (left,top) =(j / N,j / M)

[0056] P (left,bottom) =(i / N,j+1 / M)

[0057] P (right,top) = (i+1 / N, j / M)

[0058] P (right,bottom) = (i+1 / N, j+1 / M)

[0059] Where P represents the normalized coordinates of the four vertices of the current fragment;

[0060] It should be noted that the same calculation method is used when the fragment shape is other, such as triangles, rhombuses, or random polygons. Among them, the vertices of random polygons are the vertices of convex polygons calculated using Voronoi diagrams. Voronoi diagrams are currently widely used, and the specific process will not be elaborated here. The calculation of the vertex coordinates of rectangles using Voronoi diagrams is based on existing calculation methods, and this is only one type of fragment. The triangles and rhombuses mentioned here are also calculated in the same way. Of course, there can be many types of fragments in a fragmentation effect, and all fragment shapes can be completely random. Alternatively, the user can determine the position of the fragment vertices through a mask, or the user can input some vertices to determine the position of the fragment vertices.

[0061] S103. Calculate the rendering vertex index of the fragment based on the vertex position and fragment thickness;

[0062] Specifically, to render a realistic fragment effect, the front, back, and multiple sides of the fragment need to be rendered. Therefore, the rendering indices for the triangles on the fragment surface are as follows:

[0063] Front triangle: {i,i+1,i+2}, {i,i+2,i+3}…{i,i+L-3,i+L-2}, {i,i+L-2,i+L-1}

[0064] Back triangle: {i+n,i+n+1,i+n+2}, {i+n,i+n+2,i+n+3}…

[0065] {i+n,i+n+L-3,i+n+L-2}, {i+n,i+n+L-2,i+n+L-1}

[0066] Lateral triangle: {i,i+n+1,i+n},{i,i+n+1,i+1};

[0067] …

[0068] {i+t,i+t+n+1,i+t+n}, {i+t,i+t+n+1,i+t+1,};

[0069] …

[0070] {i+L-1,i+n,i+L-1+n},{i+L-1,i+n,i}

[0071] Where i represents the starting vertex of the current fragment, n represents the number of fragments (i.e., there are a total of n fragments), L represents the number of edges of the polygon, and t represents the number of the fragment edge (i.e., t represents the t-th edge of the polygon).

[0072] Please see Figure 2 For example, taking a rectangular fragment as an example, the vertices of the triangles to be rendered in each face are determined according to the surface triangle rendering index calculation formula above. Then, the triangles of each face are rendered through depth testing. All fragments can be rendered in the same way. Here, a triangle is the smallest unit of a rendering face. A rectangle is composed of two triangles, and a polygon is composed of multiple triangles. Each rendering is completed by rendering triangles to complete the rendering of the entire plane. The three vertices of the rendered triangle need to be determined, and the other parts are completed by automatic interpolation by the engine.

[0073] S104. Calculate the motion state of the fragment's vertex after it is subjected to force based on the applied external force.

[0074] Specifically, the motion state of the apex of the fragment after being subjected to force includes the velocity and rotation angle of the apex. Please refer to [link / reference]. Figure 3 and Figure 4 The external force on the fragment is generated by the surface of a spherical region (i.e., Figure 3 and Figure 4 The orange dashed sphere in the image is used for simulation. Within the spherical region, the external force acting on the fragment is represented by a vector from the center of the sphere to the fragment's center of gravity. The coordinates of the center of the sphere are (cx, cy, -r), and the coordinates of the fragment's center of gravity are (x, y, 0). Then, the direction of the force and the velocity of the fragment are:

[0075] delta=max((d-sr) / (r-sr),0.0f)

[0076] V (x,y,z)= (x-cx, y-cy, r)*(1.0-delta)

[0077] R (x,y,z) =(-Vy,Vx,0.0)

[0078] Where d represents the distance from the center of gravity of a stressed fragment to the center of the circle on the image plane, sr represents the radius of the stressed circular region, r represents the radius of the entire stressed sphere, delta represents the ratio of the distances within the annular region between the stressed region and the unstressed region (within the radius of the sphere), the ratio of points within the smaller circle is 0, and the ratio of points on the larger circle is 1, V (x,y,z) R represents the external force acting on the fragment with center of gravity (x,y,0). (x,y,z) This indicates the rotation angle of the fragment.

[0079] S105. Obtain parameter adjustment data, update the fragment vertices according to the adjusted parameters, and render in real time;

[0080] Specifically, parameter adjustment data is acquired, including but not limited to fragment shape, fragment quantity, fragment thickness, force magnitude, and fragment movement duration. Then, the fragment vertices are updated and rendered in real-time based on the adjusted parameters, including:

[0081] Adjust the parameters and determine whether the position of the fragment vertex changes and whether the applied external force changes;

[0082] If the vertex position changes, return to step S102 to recalculate the fragment vertex position;

[0083] If the vertex position remains unchanged but the applied external force changes, return to step S104 to recalculate the motion state of the vertex after the fragment is subjected to force.

[0084] If the vertex position remains unchanged and the applied external force remains unchanged, the vertex position, vertex index, and adjustment parameters are input to the vertex shader. The fragment vertex position of the current frame is calculated based on the initial state and adjustment parameters and in accordance with Newton's second law. The fragment is then rendered by the fragment shader, and step S106 is executed.

[0085] S106. Generate and output image or video with image shattering effects.

[0086] Example 2

[0087] The apparatus for implementing a real-time image fragmentation effect according to Embodiment 2 of this application includes:

[0088] The acquisition module is used to acquire the original image, the fragment shape, number of fragments, and fragment thickness required by the user;

[0089] The vertex position calculation module is used to calculate the vertex position of the fragments based on their shape and number.

[0090] The index calculation module is used to calculate the rendering vertex index of the fragment based on the vertex position and fragment thickness.

[0091] The motion state technology module is used to calculate the motion state of the fragment's apex after it is subjected to external force.

[0092] The update and rendering module is used to acquire parameter adjustment data and update and render the fragment vertices based on the adjusted parameters;

[0093] The output module is used to generate and output image or video fragmentation effects.

[0094] Example 3

[0095] The present application discloses a computer-readable storage medium that stores program code for execution by a device, the program code including steps for performing the method in any implementation of the present application, as described in the first embodiment of the present application.

[0096] The computer-readable storage medium may be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM); the computer-readable storage medium may store program code, and when the program stored in the computer-readable storage medium is executed by a processor, the processor is used to perform the steps of the method in any of the implementations of Embodiment 1 of this application.

[0097] Example 4

[0098] An electronic device according to Embodiment 4 of this application includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the method in any of the implementations in Embodiment 1 of this application.

[0099] The processor can be a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), graphics processing unit (GPU), or one or more integrated circuits, used to execute relevant programs to implement the method in any of the implementations of Embodiment 1 of this application.

[0100] The processor can also be an integrated circuit electronic device with signal processing capabilities. In implementation, each step of the method in any of the implementations of Embodiment 1 of this application can be completed by the integrated logic circuitry in the processor's hardware or by software instructions.

[0101] The aforementioned processor can also be a general-purpose processor, digital signal processor, application-specific integrated circuit (ASIC), off-the-shelf programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the functions required by the units included in the data processing apparatus of the embodiments of this application, or executes the methods in any implementation of Embodiment 1 of this application.

[0102] The above are merely preferred embodiments of this application; however, the scope of protection of this application is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in this application, based on the technical solution and its improved concept, should be covered within the scope of protection of this application.

Claims

1. A method for implementing a real-time image fragmentation effect, characterized in that, include: S101. Obtain the original image, the fragment shape, fragment quantity, and fragment thickness required by the user; S102. Calculate the vertex position of the fragment based on its shape and number; S103. Calculate the rendering vertex index of the fragment based on the vertex position and fragment thickness; S104. Calculate the motion state of the fragment's vertex after it is subjected to force based on the applied external force. S105. Obtain parameter adjustment data, update the fragment vertices according to the adjusted parameters, and render in real time; S106. Generate and output image or video with image shattering effects; Specifically, when calculating the rendering vertex indices of the fragment, the rendering indices of the front, back, and multiple sides of the outer surface of the fragment are as follows: Front triangle: {i,i+1,i+2}, {i,i+2,i+3}…{i,i+L-3,i+L-2}, {i,i+L-2,i+L-1} Back triangle: {i+n,i+n +1,i+n +2}, {i+n,i+n +2,i+n +3}… {i+n,i+n +L-3,i+n +L-2}, {i+n,i+n +L-2,i+n +L-1} Lateral triangle: {i,i+n+1,i+n},{i,i+n+1,i+1}; … {i+t,i+t+n+1,i+t+n}, {i+t, i+t+n+1, i+t+1}; … {i+L-1,i+n,i+L-1+n}, {i+L-1,i+n,i} Where i represents the starting vertex of the current fragment, n represents the number of fragments, L represents the number of polygon edges, and t represents the fragment edge number.

2. The method for implementing real-time image fragmentation effects according to claim 1, characterized in that, The shape of the fragments includes rectangles, rhombuses, triangles, or random polygons.

3. The method for implementing real-time image fragmentation effects according to claim 2, characterized in that, When calculating the vertex positions of the fragments, the length and width of the image are normalized to [0,1]. The image is then divided into M*N small fragments. The number of vertices in each fragment is determined, and the normalized vertex coordinates of each fragment are calculated. The vertices of the random polygons are the vertices of the convex polygons calculated using the Voronoi diagram.

4. The method for implementing real-time image fragmentation effects according to claim 1, characterized in that, The motion state of the apex of the fragment after being subjected to force includes the velocity of the apex and the rotation angle.

5. The method for implementing real-time image fragmentation effects according to claim 1, characterized in that, The parameter adjustment data includes fragment shape, fragment quantity, fragment thickness, force magnitude, and fragment movement duration.

6. The method for implementing real-time image fragmentation effects according to claim 5, characterized in that, Acquire parameter adjustment data, update fragment vertices based on the adjusted parameters, and render in real time, including: Adjust the parameters and determine whether the position of the fragment vertex changes and whether the applied external force changes; If the vertex position changes, return to step S102; If the vertex position remains unchanged but the applied external force changes, return to step S104. If the vertex position remains unchanged and the applied external force remains unchanged, the vertex position, vertex index, and adjustment parameters are input to the vertex shader. The fragment vertex position of the current frame is calculated based on the initial state and adjustment parameters and in accordance with Newton's second law. The fragment is then rendered by the fragment shader, and step S106 is executed.

7. A device for implementing real-time image fragmentation effects, characterized in that, include: The acquisition module is used to acquire the original image, the fragment shape, number of fragments, and fragment thickness required by the user; The vertex position calculation module is used to calculate the vertex position of the fragments based on their shape and number. The index calculation module is used to calculate the rendering vertex index of the fragment based on the vertex position and fragment thickness. The motion state technology module is used to calculate the motion state of the fragment's apex after it is subjected to external force. The update and rendering module is used to acquire parameter adjustment data and update and render the fragment vertices based on the adjusted parameters. The output module is used to generate and output images or videos with image shattering effects. Specifically, when calculating the rendering vertex indices of the fragment, the rendering indices of the front, back, and multiple sides of the outer surface of the fragment are as follows: Front triangle: {i,i+1,i+2}, {i,i+2,i+3}…{i,i+L-3,i+L-2}, {i,i+L-2,i+L-1} Back triangle: {i+n,i+n +1,i+n +2}, {i+n,i+n +2,i+n +3}… {i+n,i+n +L-3,i+n +L-2}, {i+n,i+n +L-2,i+n +L-1} Lateral triangle: {i,i+n+1,i+n},{i,i+n+1,i+1}; … {i+t,i+t+n+1,i+t+n}, {i+t, i+t+n+1, i+t+1}; … {i+L-1,i+n,i+L-1+n}, {i+L-1,i+n,i} Where i represents the starting vertex of the current fragment, n represents the number of fragments, L represents the number of polygon edges, and t represents the fragment edge number.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium is for program code executed by the device, the program code including steps for performing the method as described in any one of claims 1-6.

9. An electronic device, characterized in that, The electronic device includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the method as described in any one of claims 1-6.

Images