Video data processing method, device, equipment and computer readable storage medium

Abstract

The present application provides a video data processing method, device, equipment and computer-readable storage medium; the method includes: obtaining a first image corresponding to the last video frame of the first video currently being played, and a second image corresponding to the first video frame of the second video to be played; displaying the first image after the first video is played, and performing particle processing on the first image to obtain a plurality of image particles; based on the second image, performing diffusion and aggregation processing on the plurality of image particles, wherein the first color values ​​of the plurality of image particles are mixed with the second color values ​​of the corresponding positions in the second image during the diffusion and aggregation process, and after the diffusion and aggregation processing, the particleized second image is displayed; and playing the second video. Through the present application, a smooth transition can be achieved when switching between videos to avoid screen jitter.

Description

Technical Field

[0001] The present application relates to image processing technology, and in particular to a video data method, apparatus, device and computer-readable storage medium. Background Art

[0002] With the development of Internet technology and terminal devices, people's daily life, study and work cannot be separated from the assistance of terminal devices. Currently, watching short videos, playing games, listening to music, etc. in leisure time have become popular entertainment methods. When watching short videos or in the introduction video of game activities, there may be multiple videos playing continuously. In related technologies, when switching between two videos, it is generally a direct hard switch, which easily causes picture jitter and reduces the video playback quality when switching videos. Summary of the invention

[0003] The embodiments of the present application provide a video data processing method, device and computer-readable storage medium, which can achieve smooth transition during video switching and avoid image jitter.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides a method for processing video data, the method comprising:

[0006] Obtain a first image corresponding to the last video frame of a first video currently being played, and a second image corresponding to the first video frame of a second video to be played;

[0007] When the first video is played, the first image is displayed, and the first image is particle-processed to obtain a plurality of image particles;

[0008] Based on the second image, performing diffusion and aggregation processing on the plurality of image particles, wherein in the diffusion and aggregation process, first color values ​​of the plurality of image particles are mixed with second color values ​​of corresponding positions in the second image, and after the diffusion and aggregation processing, a particle-based second image is displayed;

[0009] Play the second video.

[0010] The present application provides a video data processing device, including:

[0011] An image acquisition module, used to acquire a first image corresponding to the last video frame of a first video currently being played, and a second image corresponding to the first video frame of a second video to be played;

[0012] A particle processing module, used for displaying the first image after the first video is played, and performing particle processing on the first image to obtain a plurality of image particles;

[0013] a diffusion aggregation module, configured to perform diffusion aggregation processing on the plurality of image particles based on the second image, wherein in the diffusion aggregation process, first color values ​​of the plurality of image particles are mixed with second color values ​​of corresponding positions in the second image, and after the diffusion aggregation processing, the particle-based second image is displayed;

[0014] A video playing module is used to play the second video.

[0015] An embodiment of the present application provides an electronic device, the electronic device comprising:

[0016] A memory for storing computer executable instructions;

[0017] The processor is used to implement the video data processing method provided in the embodiment of the present application when executing the computer executable instructions stored in the memory.

[0018] An embodiment of the present application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the video data processing method provided in the embodiment of the present application when executed by a processor.

[0019] An embodiment of the present application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, the video data processing method provided by the embodiment of the present application is implemented.

[0020] The embodiments of the present application have the following beneficial effects:

[0021] When video switching is required, the first image corresponding to the last video frame of the first video before switching and the second image corresponding to the first video frame of the second video to be played after switching are obtained. After the first video is played, the first image is displayed and the first video is hidden. Then, the first image is particle-processed to obtain a plurality of image particles. Then, the plurality of image particles are dispersed and aggregated. During the dispersion and aggregation process, the first color value of the image particle is mixed with the second color value of the corresponding position in the second image. After diffusion and aggregation, the transition from the first image to the second image is completed, so that the second image is displayed, and finally the second video is played. Since the first image corresponding to the last frame of the first video before switching is particle-processed, and after the particle diffusion and aggregation process, the second image corresponding to the first frame of the second video to be played after switching is displayed, the switching from the first video to the second video is visually completed through the transition between the first image and the second image. Not only is the processing process simple, but also the amount of calculation is small, so that computing resources can be saved and computing efficiency can be improved. In addition, the screen jitter can be avoided to achieve smooth switching and transition of the video. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a schematic diagram of the network architecture of the video playback system 100 provided in an embodiment of the present application;

[0023] Figure 2 is a schematic diagram of the structure of the terminal 400 provided in an embodiment of the present application;

[0024] Figure 3A is a flowchart of a video data processing method provided in an embodiment of the present application;

[0025] Figure 3B is a schematic diagram of an implementation flow of particle processing of a first image provided by an embodiment of the present application;

[0026] Figure 3C It is a schematic diagram of the implementation process of diffusion and aggregation processing of each image particle provided in an embodiment of the present application;

[0027] Figure 4 It is a schematic diagram of the implementation process of diffusion processing of image particles provided in an embodiment of the present application;

[0028] Figure 5A This is a schematic diagram of an implementation process of color mixing of image particles provided in an embodiment of the present application;

[0029] Figure 5B This is another schematic diagram of a process for implementing color mixing of image particles provided in an embodiment of the present application;

[0030] Figure 6 It is a schematic diagram of the implementation process of color mixing of image particles during the polymerization process provided by an embodiment of the present application;

[0031] Figure 7 This is another implementation flow diagram of the video data processing method provided in the embodiment of the present application;

[0032] Fig. 8A is a schematic diagram of the last frame of video a and the first frame of video b provided in an embodiment of the present application;

[0033] Figure 8B is a schematic diagram of an image of a certain frame when playing video a provided in an embodiment of the present application;

[0034] Fig.9A This is a schematic diagram of an image when the perspective camera is close to the image;

[0035] Fig. 9B This is a schematic diagram of an image when the perspective camera is far away from the image;

[0036] Fig. 10Ais a schematic diagram of particles when parameter w changes during particle diffusion provided by an embodiment of the present application;

[0037] Fig. 10B This is a schematic diagram showing bStart.jpg after particle aggregation is completed provided in an embodiment of the present application;

[0038] Fig.11A is a schematic diagram of particle diffusion provided in an embodiment of the present application;

[0039] Fig. 11B It is a schematic diagram showing particles during the polymerization process provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0041] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0042] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0043] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0044] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0045] 1) A vector field is a function of one vector corresponding to another vector, that is, a vector function. The common application of a vector field is that in three-dimensional space, each point has a vector of direction and intensity. Vector fields are used to describe some physical processes and data, such as the characteristics of airflow, electric field, magnetic field, light field, etc.

[0046] 2) Perspective Camera: It is a projection mode used to simulate what the human eye sees. It is the most commonly used projection mode in rendering 3D scenes.

[0047] 3) A shader is a piece of program code used to perform advanced image processing and can be widely used in computer graphics. It is a program that performs high-speed parallel computing on the GPU. It is mainly used in graphics rendering, simulation, and computing, and can produce realistic and high-quality graphics effects. In computer graphics, shaders are generally divided into two types: vertex shaders and fragment shaders. Vertex shaders process vertices in graphics, such as position, texture coordinates, normal vectors, etc., while fragment shaders are responsible for actual color processing, which is used to calculate and display the color, brightness, transparency, etc. of pixels.

[0048] 4) Pages: Information is organized in pages, and information pages are implemented by language. Hypertext links are established between each information page for easy browsing.

[0049] 5) OpenGL Shading Language (GLSL) functions are shader programs executed on the graphics processing unit (GPU). GLSL functions are basic components for implementing various shader special effects and graphic effects, and can efficiently perform mathematical calculations, matrix operations, vector operations, color processing, etc. GLSL functions can be divided into many categories, such as arithmetic functions, geometric functions, vector functions, matrix functions, color functions, texture functions, etc. Each function is closely related to the parts involved in the performance and effect, and they can be combined and spliced ​​to achieve more advanced special effects and effects.

[0050] 6) Curl Noise is a concept in the field of computer graphics used to generate realistic natural scenery and fluid effects. It is a noise-based technique used to generate random, natural motion, and is often used to simulate complex natural phenomena such as fluids, flames, and clouds. curlNoise can be used to simulate the vortex motion of viscous fluids, and its effect is more natural and realistic than traditional noise techniques. In curlNoise, each noise sample is regarded as a vortex with rotational energy. The motion vector of each point is calculated based on the vector field of the vortex, and the vortex is expanded into a continuous vector field to simulate the vortex motion. In computer graphics and computer games, curlNoise is often used to create realistic fluid animations and natural scenes. It is also one of the important applications in fields such as optical fiber and image processing.

[0051] 7) Color value. The color value is represented by hexadecimal system, which is red (Red, R), green (Green, G), and blue (B, Blue). The lowest value of each color is 0 (hexadecimal 00), and the highest value is 255 (hexadecimal FF). The hexadecimal value is written as a # followed by three or six hexadecimal characters. The three-digit representation is: #RGB, which is converted to a six-digit representation: #RRGGBB.

[0052] The embodiments of the present application provide a video data processing method, apparatus, device, computer-readable storage medium, and computer program product, which can solve the problem of image jitter when switching videos in the related art. The exemplary application of the electronic device provided by the embodiments of the present application is described below. The electronic device provided by the embodiments of the present application can be implemented as various types of user terminals such as laptops, tablet computers, desktop computers, set-top boxes, mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), smart phones, smart speakers, smart watches, smart TVs, and vehicle-mounted terminals, and can also be implemented as servers. Below, an exemplary application when the device is implemented as a terminal will be described.

[0053] See also Figure 1 , Figure 1 is a schematic diagram of the network architecture of the video playback system 100 provided in the embodiment of the present application, such as Figure 1 As shown, the video playback system 100 includes: a server 200, a network 300 and a terminal 400. The terminal 400 is connected to the server 200 via the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two. In the video playback system 100, a database 500 is also provided to provide data support to the server 200. The database 500 can be independent of the server 200, or it can be provided in the server 200. Figure 1 The illustrated case is that the database 500 is independent of the server 200 .

[0054] The terminal 400 is used to display video frame images on the graphical interface 410 , and the server 200 is used to provide the terminal 400 with video data to be played. In an embodiment of the present application, the continuous playback of short videos in the terminal 400 is taken as an example for explanation. After starting the short video function or starting the short video application, the terminal 400 can send a video acquisition request to the server 200, and the server 200 sends the video data to the terminal 400. The terminal 400 displays the playback data through the graphical interface 410, and the terminal 400 obtains the first image corresponding to the last video frame of the first video currently being played, and the second image corresponding to the first video frame of the second video to be played; when the first video is played, the first image is displayed, and the first image is particle-processed to obtain a plurality of image particles; then, based on the second image, the plurality of image particles are diffused and aggregated, wherein the first color values ​​of the plurality of image particles are mixed with the second color values ​​of the corresponding positions in the second image during the diffusion and aggregation process, and after the diffusion and aggregation process, the particle-ized second image is displayed, and then the second video is played, so that after the first video is played, a smooth transition from the last frame of the first video to the first frame of the second video can be achieved, thereby avoiding the screen jitter caused by video switching.

[0055] In some embodiments, the server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal 400 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a car terminal, etc., but is not limited thereto. The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0056] See also Figure 2 , Figure 2 is a schematic diagram of the structure of the terminal 400 provided in an embodiment of the present application, Figure 2 The terminal 400 shown includes: at least one processor 410, a memory 450, at least one network interface 420 and a user interface 430. The various components in the terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not shown in FIG. Figure 2 Various buses are labeled as bus system 440 .

[0057] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0058] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0059] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0060] The memory 450 includes a volatile memory or a nonvolatile memory, and may also include both volatile and nonvolatile memories. The nonvolatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0061] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0062] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0063] A network communication module 452, used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB), etc.;

[0064] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., display screen, speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripherals and displaying content and information);

[0065] The input processing module 454 is used to detect one or more user inputs or interactions from one of the one or more input devices 432 and translate the detected inputs or interactions.

[0066] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2 The video data processing device 455 stored in the memory 450 is shown, which can be software in the form of a program and a plug-in, etc., including the following software modules: an image acquisition module 4551, a particle processing module 4552, a diffusion aggregation module 4553 and a video playback module 4554. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be explained below.

[0067] In other embodiments, the device provided in the embodiments of the present application can be implemented in hardware. As an example, the device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the video data processing method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application specific integrated circuits (Application Specific Integrated Circuit, ASIC), digital signal processors (Digital Signal Processor, DSP), programmable logic devices (Programmable Logic Device, PLD), complex programmable logic devices (Complex Programmable Logic Device, CPLD), field programmable gate arrays (Field-Progr ammable Gate Array, FPGA) or other electronic components.

[0068] In some embodiments, the terminal or server can implement the video data processing method provided in the embodiment of the present application by running various computer executable instructions or computer programs. For example, computer executable instructions can be commands, machine instructions or software instructions at the microprogram level. The computer program can be a native program or software module in the operating system; it can be a local (Native) application (APPlicatio n, APP), that is, a program that needs to be installed in the operating system to run, such as a live broadcast APP or an instant messaging APP; it can also be a small program, that is, a program that can be run only by downloading it to a browser environment; it can also be a small program that can be embedded in any APP. In short, the above-mentioned computer executable instructions can be instructions in any form, and the above-mentioned computer program can be an application, module or plug-in in any form.

[0069] The video data processing method provided in the embodiment of the present application will be described in conjunction with the exemplary application and implementation of the terminal provided in the embodiment of the present application.

[0070] The following describes the video data processing method provided by the embodiment of the present application. As mentioned above, the electronic device implementing the video data processing method of the embodiment of the present application can be a terminal, a server, or a combination of the two. Therefore, the execution subject of each step will not be repeatedly described below.

[0071] It should be noted that the video data processing example below is described using continuously played short videos as an example. Those skilled in the art, based on their understanding of the following text, can apply the video data processing method provided in the embodiment of the present application to other scenarios of continuously playing videos.

[0072] See also Figure 3A , Figure 3A is a flow chart of the video data processing method provided in the embodiment of the present application, which will be combined with Figure 3A The steps shown are explained, and the main body of the steps in Figure 3 is the terminal.

[0073] In step 101, a first image corresponding to the last video frame of a first video currently being played and a second image corresponding to the first video frame of a second video to be played are obtained.

[0074] In some embodiments, each video played continuously can be parsed, and the first video frame image and the last video frame image of each video can be stored. It should be noted that the first video frame image and the last video frame image in a video are stored as separate images. That is, in the subsequent steps, a single image is particle-ized and dispersed and aggregated, which can reduce the complexity of video data processing and improve the efficiency of video data processing. When video switching is required, the first image corresponding to the last video frame of the first video currently being played before the switching is obtained, and the second image corresponding to the first video frame of the second video to be played after the switching is obtained.

[0075] In some embodiments, after acquiring the first image and the second image, the first original size information of the first image can be acquired, the second original size information of the second image can be acquired, and it can be determined whether the first original size information is the same as the second original size information, wherein the first original size information includes a first original width and a first original height, and the second original size information includes a second original width and a second original height. When the first original width is the same as the second original width, and the first original height is the same as the second original height, it is determined that the first original size information is the same as the second original size information. When at least one of the first original height and the second original height, and the first original width and the second original width is different, it is determined that the first original size information is different from the second original size information. When the first original size information is different from the second original size information, the second image is adjusted based on the first original size information and the second original size information to obtain an adjusted second image, and the size information of the adjusted second image is the first original size information, that is, the second image is adjusted to the same size as the first image.

[0076] When adjusting the second image based on the first original size information and the second original size information, first, the first original size information and the second original size information are used to determine the scaling ratio for the second image, and then the second image is scaled based on the scaling ratio to obtain the adjusted second image. Exemplarily, if the first original height is different from the second original height, but the first original width is the same as the second original width, the ratio of the first original height to the second original height can be determined as the scaling ratio, and then the second original height is multiplied by the scaling ratio to obtain the adjusted second height, and the adjusted second height and the second original width are determined as the size information of the adjusted second image, so that it can be ensured that when the diffusion aggregation processing is performed, the size information of the first image and the second image is the same, thereby simplifying the diffusion aggregation processing process and improving the efficiency of video data processing.

[0077] In step 102, after the first video is played, the first image is displayed, and the first image is particle-processed to obtain a plurality of image particles.

[0078] It should be noted that when the first video is finished playing, the first image is displayed, and the first video is hidden at this time. Since the first image is the same as the image corresponding to the last video frame of the first video, it will give people a visual feeling that the first video is still being played. In some embodiments, the "processing the first image into particles to obtain a plurality of image particles" in step 102 can be performed by Figure 3B Steps 1021 to 1024 are shown to be implemented and are described in detail below.

[0079] In step 1021, a texture object of the first image is created, and a material object of the first image is created.

[0080] In some embodiments, the first image can be created by a loader in a JavaScript 3D image engine library. For example, a texture object of the first image can be created using a texture loading function, wherein the texture loading function is a loader in the JavaScript 3D image engine library Three.js library, which is used to load and create a texture object. A texture object is a picture or graphic resource applied to the surface of a 3D rendering object, which can help improve the graphic effect and increase the sense of reality and details.

[0081] In some embodiments, a material object of the first image can be created through a shader material constructor. The material object has three main attributes: vertexShader, fragmentShader, and uniforms, where vertexShader represents a vertex shader, through which a program written in GLSL language can be defined, and fragmentShader represents a fragment shader. Like a vertex shader, a GLSL program can also be customized. The uniforms attribute is a channel between a JavaScript program and a shader program, and uniforms variables are passed to a vertex shader and a fragment shader through the uniforms attribute.

[0082] In some embodiments, after the material object of the first image is created, two shader programs can be passed to the material object: a vertex shader program (VertexShader) and a fragment shader program (FragmentShade). The vertex shader program is usually used to process the vertex information of the rendered object, calculate the vertex position, normal vector, UV coordinates, color and other attributes, and pass it to the fragment shader for subsequent processing. The fragment shader program is mainly used to process the information of a single rendered pixel (or fragment), calculate its color, transparency and other attributes on the screen, and pass it to the rendering pipeline for final display.

[0083] In step 1022, a texture attribute is added to the material object, and the texture object is determined as the attribute value of the texture attribute.

[0084] In some embodiments, a texture attribute may be added to the uniforms attribute of the material object, and the texture object created in step 1021 may be determined as the attribute value of the texture attribute, so that the texture resource of the first image may be used through this texture attribute in subsequent shader programs.

[0085] In step 1023, a plane vertex network and a point cloud object are created, and the plane vertex network and the material object are determined as parameter information of the point cloud object.

[0086] In some embodiments, when creating a plane vertex network, the aspect ratio of the first image, the width and height of the plane vertex network are first obtained, and based on the aspect ratio, the width and the height, the number of width sampling points and the number of height sampling points of the plane vertex network are determined. Finally, based on the width, the height, the number of width sampling points and the number of height sampling points, the plane vertex network is created.

[0087] In some embodiments, the width and height of the plane vertex network may be pre-set, for example, the width of the plane vertex network may be 2, and the height may be 2. The aspect ratio of the first image may be 1:2, then, when determining the number of width sampling points and the number of height sampling points of the plane vertex network, it is necessary to satisfy that the ratio of the number of width sampling points to the number of height sampling points satisfies the aspect ratio of the first image, illustratively, the number of width sampling points may be 480, and the number of height sampling points may be 960, and finally, based on the width, height, the number of width sampling points, and the number of height sampling points, the plane vertex network is created by calling a library function for creating plane vertices.

[0088] In some embodiments, a point cloud object is created by calling a library function for creating a point cloud object. The point cloud object consists of a series of points, each of which contains its position, color, size and other properties in 3D space. Point cloud rendering is often used to simulate various scenes in the real world, such as natural phenomena, particle effects, and gas clouds. After the point cloud object is created, the plane vertex network and material object are determined as parameter information of the point cloud object.

[0089] In step 1024, based on the material object including the texture object, the plane vertex network is shaded, and the shaded plane vertex network is converted into a point cloud to obtain a plurality of image particles.

[0090] In some embodiments, the renderer will shade the plane vertex network according to the logic in the vertex shader program and the fragment shader program, thereby converting the shaded plane vertex network into a point cloud, completing the particle processing of the first image, obtaining multiple image particles, and displaying the particleized first image.

[0091] In the above steps 1021 to 1024, a plane vertex mesh is created in a cross-browser script using WebGL technology, and a texture object and a material object corresponding to the first image are created. The texture object corresponding to the first image is determined as the attribute value of the texture attribute in the material object, so that the texture resources of the first image can be used in the subsequent rendering process. After that, a point cloud object is created, and the material object and the plane vertex network are passed as parameters to the point cloud object, and two shader programs are used for rendering to complete the particleization of the first image, thereby providing a data basis for the subsequent diffusion and aggregation processing of the first image according to the granularity of the image particles.

[0092] In step 103, based on the second image, a diffusion and aggregation process is performed on the plurality of image particles.

[0093] In which, during the diffusion polymerization process, the first color values ​​of multiple image particles are mixed with the second color values ​​of corresponding positions in the second image, so that each image particle transitions from the first color value to the second color value, thereby displaying the particle-based second image after the diffusion polymerization process.

[0094] In some embodiments, see Figure 3C , step 103 can be implemented through the following steps 1031 to 1034, which are described in detail below.

[0095] In step 1031, a second color value of each image particle in the second image is determined.

[0096] In some embodiments, the number of width sampling points and the number of height sampling points of the plane vertex network are first obtained; then, the second image is segmented according to the number of width sampling points and the number of height sampling points to obtain image blocks corresponding to each image particle in the second image; then, the color value of each image block is determined, and the color value of each image block is determined as the second color value of each corresponding image particle.

[0097] In some embodiments, the second image is segmented according to the number of width sampling points and the number of height sampling points, which can be understood as dividing the second image into image blocks of the number of width sampling points*the number of height sampling points, wherein each row has image blocks of the number of width sampling points and each column has image blocks of the number of height sampling points. Exemplarily, the number of width sampling points is 480 and the number of height sampling points is 960, then the second image is segmented to obtain 480*960 image blocks.

[0098] When determining the color value of an image block, the color value of at least one pixel included in the image block is obtained, and one of the following is determined as the color value of the image block: the color mean of all pixel values ​​in the image block, the color value of a pixel at a preset position in the image block, and the color mean of pixel points at at least two preset positions in the image block.

[0099] In some embodiments, when there is only one pixel in an image block, the color value of the image block is the color value of the pixel. When there are at least two pixels in an image block, it can be the color mean of all pixels, or the color value of a pixel at a preset position in the image block, exemplarily, it can be the color value of a pixel at the upper left corner of the image block, or it can be the color mean of at least two pixels at preset positions in the image block, exemplarily, it can be the color mean of two pixels at the upper left corner and the lower right corner.

[0100] In step 1032, the vortex noise function and the initial position of each image particle in the first image are called to determine the scattering target position of each image particle.

[0101] In some embodiments, a swirl noise function is used to modulate the particle positions to create a curling effect. Figure 4 Step 1032 can be implemented by the following steps 321 to 323, as shown below. Figure 4 Provide explanation.

[0102] In step 321, a preset distortion degree parameter and current scene time are obtained.

[0103] The distortion degree parameter distortion can be an interpolation variable created in the uniforms attribute of the material object, which is used to control the degree of distortion of the image particles. The current scene time time is a uniform variable passed to the shader by the Java Script code.

[0104] In step 322, based on the distortion degree parameter, the current scene time, and the initial position of each image particle in the first image, the offset information of each image particle is determined.

[0105] In some embodiments, when step 322 is implemented, a first position vector is first determined based on the initial positions of the individual image particles in the first image; then a second position vector is determined based on the initial positions of the individual image particles in the first image and the current scene time, and the second position vector is determined as the input of the vortex noise function to obtain an output vector, and finally the product of the first position vector, the output vector and the distortion degree parameter is determined as the offset information.

[0106] The initial position position of each image particle in the first image is a built-in variable in the vertex shader program, representing the initial position vector of the current vertex (i.e., the image particle). The first position vector is determined based on the initial position of each image particle in the first image. The initial x coordinate value of the image particle is multiplied by the first weight to obtain the x component of the first position vector, the initial y coordinate value of the image particle is multiplied by the second weight to obtain the y component of the first position vector, and the first preset value is determined as the z component of the first position vector, that is, the first position vector is a 3D vector. Assuming that the first preset value is 1, the first weight is 2, and the second weight is 1, the first position vector can be expressed as (position.x*2, position.y*1,1). The first weight and the second weight can be pre-selected and set.

[0107] The second position vector is obtained by performing a certain calculation on the x component and y component of the initial position of the image particle and the current scene time. The x component of the second position vector can be obtained by weighted summing the x component of the initial position and the current scene time, the y component of the second position vector can be obtained by weighted summing the y component of the initial position and the current scene time, and the z component of the second position component can be obtained by weighted summing the x component of the initial position and the y component of the initial position. It should be noted that the weight value used in the above weighted summation is pre-set according to the experimental data, so as to ensure that a beautiful distortion effect is achieved during the diffusion process. The second position vector is determined as the input of the vortex noise function to obtain an output vector, which is also a three-dimensional vector, and then the first position vector and the output vector are multiplied bit by bit to obtain an intermediate vector, and the product of the intermediate vector and the distortion degree parameter is determined as the offset information of the image particle.

[0108] In step 323, the initial positions of the image particles are superimposed with the corresponding offset information to obtain the scattered target positions of the image particles.

[0109] In some embodiments, the x component of the initial position of the image particle is superimposed with the x component in the offset information to obtain the x component of the scattered target position of the image particle. Similarly, the y component of the initial position of the image particle is superimposed with the y component in the offset information to obtain the y component of the scattered target position of the image particle. The z component of the initial position of the image particle is superimposed with the z component in the offset information to obtain the z component of the scattered target position of the image particle.

[0110] Continue to refer to Figure 3C , proceed to step 1032 for explanation.

[0111] In step 1033, diffusion processing is performed on each image particle to move each image particle to its corresponding scattered target position.

[0112] In some embodiments, after determining the scattering target position of each image particle, each image particle can be moved to the scattering target position according to a preset appearance effect, and when each image particle is moved to the scattering target position, before step 1034, Figure 5A The color mixing of image particles is performed in the manner shown below. Figure 5A Provide explanation.

[0113] In step 201A, when each image particle reaches its corresponding scattering target position, a first mixing weight is obtained.

[0114] In some embodiments, the first mixing weight represents the proportion of the second color value corresponding to the image particle in the mixed color value. For example, if the first mixing weight is 0, it means that the proportion of the second color value corresponding to the image particle in the mixed color value is 0, that is, the mixed color value of the image particle is the first color value. If the first mixing weight is 0.5, it means that the proportion of the second color value corresponding to the image particle in the mixed color value is 0.5, that is, the mixed color value of the image particle is the average of the first color value and the second color value.

[0115] In step 202A, the first color value and the second color value of each image particle are mixed using a first mixing weight to obtain a first mixed color value of each image particle.

[0116] In some embodiments, the first mixed color value of the image particles can be determined according to formula (1):

[0117] C mix1 =(1-ω 1 )*C 1 +ω 1 *C 2 (1);

[0118] Among them, ω 1 is the first mixing weight, C 1 is the first color value of the image particle, C 2 is the second color value of the image particle, C mix1 is the first mixed color value.

[0119] In the above steps 201A to 202A, when the image particles reach their respective corresponding scattering target positions, the first color value and the second color value of the image particles are mixed, which can reduce the amount of calculation for color value mixing and improve the efficiency of video data processing.

[0120] In some embodiments, during the dispersion of the image particles, Figure 5B Steps 201B to 202B shown perform color mixing of image particles, and the following combination Figure 5B Provide explanation.

[0121] In step 201B, when each image particle moves to its corresponding scattering target position, a second mixing weight is determined based on the first moving distance of each image particle.

[0122] In some embodiments, the first moving distance of the image particles is the moving distance relative to the initial position of the image particles, and the second mixing weight is positively correlated with the first moving distance, that is, the larger the first moving distance between the image particles and the initial position during the dispersion process, the higher the second mixing weight. A first conversion function between the first moving distance of the image particles and the second mixing weight can be preset, and the first conversion function can be a continuous function or a piecewise function. When the first conversion function is a continuous function, it means that the second mixing weight changes in real time with the first moving distance of the image particles. When the first conversion function is a piecewise function, the second mixing weight can be set to change when the first moving distance reaches a preset distance. In some embodiments, in order to reduce the amount of calculation, the second mixing weight can be determined based on the first moving distance when the first moving distance of the image particles reaches one-quarter, one-half, three-quarters, and the maximum offset distance. Exemplarily, assuming that the maximum offset distance is 8, the second mixing weight can be set to 0 when the first moving distance of the image particles is between 0-2, and when the first moving distance reaches 2, the second mixing weight is determined to be 0.125, and when the first moving distance is between 2 and 4, the second mixing weight remains unchanged at 0.125, and when the first moving distance reaches 4, the second mixing weight is determined to be 0.25, and when the first moving distance is between 4 and 6, the second mixing weight remains unchanged at 0.25, and when the first moving distance reaches 6, the second mixing weight is determined to be 0.375, and when the first moving distance is between 6 and 8, the second mixing weight remains unchanged at 0.375, and when the first moving distance reaches the maximum offset distance of 8, the second mixing weight reaches 0.5.

[0123] In step 202B, the first color value and the second color value of each image particle are mixed using the second mixing weight to obtain a second mixed color of each image particle.

[0124] In some embodiments, ω in formula (1) can be 1 Updated to ω 2 , thereby using the second mixing weight to obtain the second mixed color of the image particles.

[0125] In the above steps 201B to 202B, during the diffusion process of the image particles, the second mixing weight is dynamically adjusted as the first moving distance of the image particles changes, so that the mixed color value can also change dynamically during the movement of the image particles, thereby improving the smoothness of the color mixing.

[0126] Continue to refer to Figure 3C , proceed to step 1033 for explanation.

[0127] In step 1034, the image particles moved to the scattered target position are aggregated to restore each image particle to an initial position, thereby obtaining a particle-based second image.

[0128] In some embodiments, see Figure 6 When implementing step 1034, it can be implemented through the following steps 341 to 343, which are described in detail below.

[0129] In step 341 , in the process of each image particle returning from its corresponding scattered target position to its corresponding initial position, a third mixing weight is determined based on a second moving distance of each image particle relative to the scattered target position.

[0130] In some embodiments, the third mixing weight is positively correlated with the second moving distance. When implementing step 341, similar to step 201B, a second conversion function between the second moving distance of the image particle and the third mixing weight can be preset, and the second conversion function can be a continuous function or a piecewise function. When the second conversion function is a continuous function, it means that the third mixing weight changes in real time with the second moving distance of the image particle. When the second conversion function is a piecewise function, the third mixing weight can be set to change when the second moving distance reaches a preset distance.

[0131] In step 342, the first color value and the second color value of each image particle are mixed using the third mixing weight to obtain a third mixed color value of each image particle.

[0132] In some embodiments, ω in formula (1) can be 1 Updated to the third mixing weight ω 3 , and then refer to formula (1) and use the third mixing weight to obtain the third mixed color value of the image particle.

[0133] In step 343, when each image particle is restored from its corresponding scattered target position to its corresponding initial position, the mixed color value of each image particle is the second color value, and a particle-based second image is obtained.

[0134] In some embodiments, each image particle can follow a trajectory of diffusion from the initial position to the scattered target position, and return from the scattered target position to the corresponding initial position, or can follow different trajectories to return from the scattered target position to the corresponding initial position. When each image particle returns to the initial position, the third mixing weight is 1, that is, the mixed color value of each image particle is the second color value, and the second image is obtained.

[0135] Through the above steps 341 to 343, the first color value and the second color value of the image particles can be mixed through the third mixing weight in the process of the image particles aggregating from the scattered target position to the initial position, and when the image particles are restored to the initial position, the third mixed color value of the image particles is the second color value, that is, at this time the image particles complete the transition from the first color value to the second color value, thereby displaying the second image.

[0136] In some embodiments, as shown in steps 341 to 343, during the process of image particles recovering from a scattered target position to an initial position, the mixed color value of the image particles changes dynamically with a second moving distance of the image particles relative to the scattered target position. Alternatively, the image particles may directly change to the second color value after recovering to the initial position. This can reduce the amount of data calculation and improve video processing efficiency.

[0137] Continue to refer to Figure 3A , proceed to step 103 for explanation.

[0138] In step 104, the second video is played.

[0139] In some embodiments, after completing the transition from the first image to the second image, the second image can be hidden and the second video can be played. Since the second image is the image corresponding to the first video frame of the second video, hiding the second image to playing the second video is visually imperceptible. Therefore, the switching from the first video to the second video can be completed by smoothly switching from the first image to the second image, which not only avoids picture jitter, but also improves the fun of video switching through the process of diffusion and aggregation.

[0140] In the video data processing method provided in the embodiment of the present application, when video switching is required, the first image corresponding to the last video frame of the first video before switching and the second image corresponding to the first video frame of the second video to be played after switching are obtained, and after the first video is played, the first image is displayed and the first video is hidden, and then the first image is particle-processed to obtain a plurality of image particles, and then the plurality of image particles are dispersed and aggregated, and during the dispersion and aggregation process, the first color value of the image particle is mixed with the second color value of the corresponding position in the second image, and after the diffusion and aggregation process, the transition from the first image to the second image is completed, thereby displaying the second image, and finally playing the second video. Since the first image corresponding to the last frame of the first video before switching is particle-processed, and after the particle diffusion and aggregation process, the second image corresponding to the first frame of the second video to be played after switching is displayed, the switching from the first video to the second video is visually completed through the transition between the first image and the second image, not only the processing process is simple, but also the screen jitter can be avoided, and the smooth switching and transition of the video can be achieved.

[0141] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0142] The video data processing method provided in the embodiment of the present application can be applied to a scenario where multiple videos need to be switched when played continuously. For example, it can be applied to a short video playback scenario or a game recommendation activity video playback scenario. Assume that video a and video b need to be switched, that is, after video a is played, video b is played.

[0143] The video data processing method provided in the embodiment of the present application uses WebGL technology to achieve the switching and connection transition of video a and video b. The Web Graphics Library (WebGL) is a JavaScript application programming interface (Application Programming Interface, API) based on the OpenGL for Embedded Systems (OpenGL ES), which can present 3D and 2D graphics on a web browser in a hardware-accelerated manner. WebGL realizes 3D graphics rendering in the browser through the collaboration of JavaScript code and GPU hardware, and can display high-quality images on web pages, including complex graphics, graphics with changing transparency, and advanced light and shadow effects. WebGL provides native support for 3D graphics on the Web, and it also has good cross-platform features and can run smoothly on most modern browsers.

[0144] Figure 7This is another implementation flow diagram of the video data processing method provided in the embodiment of the present application. Figure 7 Provide explanation.

[0145] In step 301, the last frame of video a (aEnd.jpg) and the first frame of video b (bStart.jpg) are saved.

[0146] Fig. 8A is a schematic diagram of the last frame of video a and the first frame of video b provided in an embodiment of the present application, wherein: Fig. 8A 801 is aEnd.jpg corresponding to the last frame of video a, and 802 is bStart.jpg corresponding to the first frame of video b.

[0147] In step 302, video a is played.

[0148] here, Figure 8B An image of a certain frame when playing video a is shown.

[0149] In step 303, when the video a ends playing, aEnd.jpg is displayed.

[0150] In some embodiments, when video a ends, aEnd.jpg corresponding to the last video frame of video a is displayed, covering video a, that is, video a is hidden. At this time, a separate image is displayed instead of the last frame of video a, which facilitates the subsequent particleization of the image.

[0151] In step 304, aEnd.jpg is particle-ized.

[0152] In some embodiments, based on the 3D JavaScript library, aEnd.jpg is used as a texture, split into several particles, the first color value of each particle in aEnd.jpg is collected, and the first color value is assigned to the corresponding particle. When implementing, it can be achieved by the following steps:

[0153] Step a, first create a texture object of the aEnd.jpg image by calling the texture loader function.

[0154] Step b, creating a material object for the aEnd.jpg image by calling a library function of the material object, and passing two shader programs into the material object, the two shader programs including a vertex shader program (VertexShader) and a fragment shader program (FragmentShader).

[0155] Step c, set a texture attribute in the uniforms property of the material object, and set the texture object created in step a as the attribute value of the texture attribute. In this way, the image texture resource can be used through this `texture` attribute in the subsequent shader program.

[0156] Step d, by calling the function of creating a plane geometry, a plane geometry is created, the width of the geometry can be 2, the height can be 2, and 481 and 821 points are sampled in the horizontal and vertical directions respectively.

[0157] Step e, creating a point cloud object by calling a function to create a point cloud object, and using the plane geometry and material object created in step d as parameters of the point cloud object.

[0158] Step f, add this point cloud object to the scene.

[0159] In some embodiments, the renderer shades the plane geometry according to the logic in the shader program, converts it into a point cloud, and displays it in the scene, thereby completing the particleization of aEnd.jpg.

[0160] After aEnd.jpg is particle-processed, Fig.9A As shown, when the perspective camera is close, clear points can be seen, such as Fig. 9B As shown, when the perspective camera is zoomed out, the individual point cloud particles that make up the picture will not be seen, but a normal picture will be seen.

[0161] In step 305, the second color value of each particle on bStart.jpg is obtained.

[0162] In step 306, the positions of the particles are changed to form an animation effect of the image spreading out. During the spreading movement, the first color value and the second color value of each particle are mixed using the mix function.

[0163] In some embodiments, a swirl noise function is used to change the position of each particle to form a spreading effect. In the dynamic process of spreading, the first color value and the second color value of each particle are mixed through the GLSL built-in mixing function mix(aEnd.jpg, bStart.jpg, w), where w represents the proportion of the second color value in the mixed color value, and the initial value of w is 0, that is, Fig. 10A As shown in 1001 in the figure, each particle is initially displayed based on the first color value in aEnd.jpg. When w is 0.25, the mixed color value of each particle is as follows Fig. 10AAs shown in 1002; when w is 0.5, as Fig. 10A As shown in 1003 in FIG. 1 , the mixed color value is the average of the first color value and the second color value of each particle. When w is 1, Fig. 10B As shown, each particle is displayed based on the second color value of bStart.jpg. In the process of dispersion and aggregation of each particle, the color value of each particle is made to transition from aEnd.jpg to bStart.jpg by adjusting the w parameter of the mix function.

[0164] In step 307, after each particle is dispersed to the target position, it starts to aggregate and each particle re-aggregates to the initial position. During the aggregation process, the color proportion of the second color value is increased. When the particles aggregate to the initial position, the color proportion of the second color value is 1.

[0165] In some embodiments, the offset of each particle to the farthest target position is determined based on the position information of each particle, the current scene time `time and the distortion degree parameter. Among them, position is a built-in variable in VertexShader, representing the position vector of the current vertex, and time is a uniform variable passed to Shader by JavaScript code, indicating the time of the current scene. The distortion degree parameter (distortion) is an interpolation variable created from the uniforms attribute of the material object, which controls the degree of color distortion. Exemplarily, the distortion degree parameter can be a real number between 0 and 1.

[0166] When calculating the offset, first use the position of each particle to determine a new first position vector (exemplarily, the first position vector can be position.x*2.0, position.y, 1.0). Then, based on the position and time of each particle, determine the second position vector (exemplarily, the second position vector can be position.x*0.002+time*0.1, position.y*0.008+time*0.1, (position.x+position.y)*0.02) and pass the second position vector into the curlNoise function to obtain a 3D output vector. The curlNoise function here is a custom function used to calculate the vector field value of a specific point in the 3D noise field. Finally, multiply the first position vector and the output vector to obtain an offset, which represents the offset between the initial position and the target position in the noise field, and produces a distortion effect to a certain extent through the distortion degree parameter, such as Fig.11AAs shown, creating an interesting and dynamic visual effect.

[0167] The offset is added to the initial position vector to get the target position vector of each particle. It is used in the output of the final vertex shader to return the calculated vertex position to the Three.js rendering pipeline, thereby rendering a pattern with a distorted effect on the screen.

[0168] In the embodiment of the present application, after each particle is dispersed to the target position, the w parameter in the mixing function may be 0.5, that is, at this time, each particle is displayed based on the color average of the first color value and the second color value. In addition, during the process of each particle re-aggregating from the target position to the initial position, the w parameter of the mix function gradually increases, and when the particles return to the initial position, w reaches a maximum value of 1, that is, at this time, each particle is displayed based on the second color value.

[0169] Fig. 11B is a schematic diagram showing particles in the polymerization process provided in the embodiment of the present application. When the w parameter value in the mixing function is 0.75, the mixed color value of each particle is as follows: Fig. 11B As shown in 1101, when the w parameter value in the mixing function is 1, the mixed color value of each particle is as follows Fig. 11B As shown in 1102, the bStart.jpg image is displayed at this time.

[0170] In step 308 , the particle aggregation is completed, that is, the switching between aEnd.jpg and bStart.jpg is completed.

[0171] In step 309, bStart.jpg is hidden and video b starts playing.

[0172] Since the first frame of video b is displayed after the aggregation is completed, and then video b starts to be played, a smooth transition of video switching can be achieved.

[0173] In an embodiment of the present application, an optimized solution for switching and connecting two videos is provided. By using WebGL technology, a first image corresponding to the last video frame and a second image corresponding to the first video frame of two adjacently played video frames are cut into multiple image particles, and the first color value of each image particle in the first image and the second color value in the second image are saved. A mathematical formula is used to change the particle-based video frame image from an initial arrangement state to a dispersed or other shape. During the dispersed aggregation process, the color values ​​of the image particles are mixed. When the aggregation is completed, the color values ​​of the image particles transition to the second color value and return to the initial arrangement state to present the second image. Then, the second image is hidden to play the next video, thereby avoiding screen jitter and achieving smooth connection of video switching.

[0174] It can be understood that in the embodiments of the present application, when data related to user information, first video, second video, etc. is applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0175] The following is a description of an exemplary structure of the video data processing device 455 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software modules stored in the video data processing device 455 of the memory 450 may include:

[0176] The image acquisition module 4551 is used to acquire a first image corresponding to the last video frame of the first video currently being played, and a second image corresponding to the first video frame of the second video to be played;

[0177] A particle processing module 4552 is used to display the first image after the first video is played, and to perform particle processing on the first image to obtain a plurality of image particles;

[0178] A diffusion aggregation module 4553 is used to perform diffusion aggregation processing on the plurality of image particles based on the second image, wherein in the diffusion aggregation process, the first color values ​​of the plurality of image particles are mixed with the second color values ​​of the corresponding positions in the second image, and after the diffusion aggregation processing, the particle-based second image is displayed;

[0179] The video playing module 4554 is used to play the second video.

[0180] In some embodiments, the particle processing module 4552 is further configured to:

[0181] Creating a texture object of the first image, and creating a material object of the first image;

[0182] Adding a texture attribute to the material object, and determining the texture object as the attribute value of the texture attribute;

[0183] Creating a plane vertex network and a point cloud object, and determining the plane vertex network and the material object as parameter information of the point cloud object;

[0184] Based on the material object including the texture object, the plane vertex network is shaded, and the shaded plane vertex network is converted into a point cloud to obtain a plurality of image particles.

[0185] In some embodiments, the particle processing module 4552 is further configured to:

[0186] Obtaining the aspect ratio of the first image, and the width and height of the plane vertex network;

[0187] Determine the number of width sampling points and the number of height sampling points of the plane vertex network based on the aspect ratio, the width and the height;

[0188] The plane vertex network is created based on the width, the height, the number of width sampling points, and the number of height sampling points.

[0189] In some embodiments, the apparatus further comprises:

[0190] A first acquisition module, used to acquire first original size information of the first image and second original size information of the second image;

[0191] A size adjustment module is used to adjust the second image based on the first original size information and the second original size information when the first original size information is different from the second original size information to obtain an adjusted second image, wherein the size information of the adjusted second image is the first original size information.

[0192] In some embodiments, the diffusion aggregation module 4553 is further used to:

[0193] Determine a second color value of each image particle in the second image;

[0194] Calling the vortex noise function and the initial position of each image particle in the first image to determine the scattered target position of each image particle;

[0195] Performing diffusion processing on each of the image particles to move each of the image particles to a corresponding scattered target position;

[0196] The image particles moved to the scattered target position are aggregated to restore the image particles to the initial position, thereby obtaining a particle-based second image.

[0197] In some embodiments, the diffusion aggregation module 4553 is further used to:

[0198] Obtain the number of width sampling points and the number of height sampling points of the plane vertex network;

[0199] Segmenting the second image according to the number of width sampling points and the number of height sampling points to obtain image blocks corresponding to each image particle in the second image;

[0200] The color value of each image block is determined, and the color value of each image block is determined as the second color value of each corresponding image particle.

[0201] In some embodiments, the diffusion aggregation module 4553 is further used to:

[0202] For each of the image blocks, obtain the color value of at least one pixel included in the image block; determine one of the following as the color value of the image block: the color mean of all pixel values ​​in the image block, the color value of a pixel at a preset position in the image block, or the color mean of a pixel at at least two preset positions in the image block.

[0203] In some embodiments, the diffusion aggregation module 4553 is further used to:

[0204] Get the preset distortion degree parameters and current scene time;

[0205] Determining offset information of each image particle based on the distortion degree parameter, the current scene time, and an initial position of each image particle in the first image;

[0206] The initial positions of the image particles are superimposed with the corresponding offset information to obtain the scattered target positions of the image particles.

[0207] In some embodiments, the diffusion aggregation module 4553 is further used to:

[0208] Determining a first position vector based on an initial position of each image particle in the first image;

[0209] Determining a second position vector based on an initial position of each image particle in the first image and a current scene time;

[0210] Determine the second position vector as the input of the vortex noise function, and determine the output vector;

[0211] The product of the first position vector, the output vector and the distortion degree parameter is determined as offset information.

[0212] In some embodiments, the apparatus further comprises:

[0213] A second acquisition module, used for acquiring a first mixing weight when each of the image particles reaches a corresponding scattering target position;

[0214] The first mixing module is used to mix the first color value and the second color value of each image particle by using the first mixing weight to obtain a first mixed color value of each image particle.

[0215] In some embodiments, the apparatus further comprises:

[0216] A first determining module, configured to determine a second mixing weight based on a first moving distance of each image particle in a process in which each image particle moves to a corresponding scattered target position;

[0217] The second mixing module is used to mix the first color value and the second color value of each image particle by using the second mixing weight to obtain a second mixed color value of each image particle.

[0218] In some embodiments, the diffusion aggregation module 4553 is further used to:

[0219] Restoring each of the image particles from the corresponding scattered target positions to the corresponding initial positions;

[0220] In the process of each image particle returning from the corresponding scattered target position to the corresponding initial position, determining a third mixing weight based on a second moving distance of each image particle relative to the scattered target position;

[0221] The first color value and the second color value of each image particle are mixed using the third mixing weight to obtain a third mixed color value of each image particle, wherein when each image particle is restored to the initial position, the mixed color value of each image particle is the second color value to obtain a particle-based second image.

[0222] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the electronic device reads the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device executes the video data processing method described in the embodiment of the present application.

[0223] The present application embodiment provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions or computer programs are stored. When the computer-executable instructions or computer programs are executed by a processor, the processor will be caused to execute the video data processing method provided by the present application embodiment, for example, Figure 3A , Figure 7 The video data processing method is shown.

[0224] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0225] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0226] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0227] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0228] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A video data processing method, characterized in that: The method comprises: Obtain a first image corresponding to the last video frame of a first video currently being played, and a second image corresponding to the first video frame of a second video to be played; When the first video is played, the first image is displayed, and the first image is particle-processed to obtain a plurality of image particles; Based on the second image, performing diffusion and aggregation processing on the plurality of image particles, wherein in the diffusion and aggregation process, first color values ​​of the plurality of image particles are mixed with second color values ​​of corresponding positions in the second image, and after the diffusion and aggregation processing, a particle-based second image is displayed; Play the second video.

2. The method according to claim 1, characterized in that The step of performing particle processing on the first image to obtain a plurality of image particles includes: Creating a texture object of the first image, and creating a material object of the first image; Adding a texture attribute to the material object, and determining the texture object as the attribute value of the texture attribute; Creating a plane vertex network and a point cloud object, and determining the plane vertex network and the material object as parameter information of the point cloud object; Based on the material object including the texture object, the plane vertex network is shaded, and the shaded plane vertex network is converted into a point cloud to obtain a plurality of image particles.

3. The method according to claim 2, characterized in that The step of creating a planar vertex network comprises: Obtaining the aspect ratio of the first image, and the width and height of the plane vertex network; Determine the number of width sampling points and the number of height sampling points of the plane vertex network based on the aspect ratio, the width and the height; The plane vertex network is created based on the width, the height, the number of width sampling points, and the number of height sampling points.

4. The method according to claim 3, characterized in that The method further comprises: Acquire first original size information of the first image, and acquire second original size information of the second image; When the first original size information is different from the second original size information, the second image is adjusted based on the first original size information and the second original size information to obtain an adjusted second image, and the size information of the adjusted second image is the first original size information.

5. The method according to claim 3, characterized in that The step of performing diffusion and aggregation processing on the plurality of image particles based on the second image includes: Determine a second color value of each image particle in the second image; Calling the vortex noise function and the initial position of each image particle in the first image to determine the scattered target position of each image particle; Performing diffusion processing on each of the image particles to move each of the image particles to a corresponding scattered target position; The image particles moved to the scattered target position are aggregated to restore the image particles to the initial position, thereby obtaining a particle-based second image.

6. The method according to claim 5, characterized in that The determining of the second color value of each image particle in the second image includes: Obtain the number of width sampling points and the number of height sampling points of the plane vertex network; Segmenting the second image according to the number of width sampling points and the number of height sampling points to obtain image blocks corresponding to each image particle in the second image; The color value of each image block is determined, and the color value of each image block is determined as the second color value of each corresponding image particle.

7. The method according to claim 6, characterized in that Determining the color value of each image block includes: For each of the image blocks, obtaining a color value of at least one pixel included in the image block; One of the following is determined as the color value of the image block: a color mean of all pixel values ​​in the image block, a color value of a pixel point at a preset position in the image block, or a color mean of pixel points at at least two preset positions in the image block.

8. The method according to claim 5, characterized in that The step of determining the scattered target position of each image particle based on the vortex noise function and the initial position of each image particle in the first image comprises: Get the preset distortion degree parameters and current scene time; Determining offset information of each image particle based on the distortion degree parameter, the current scene time, and an initial position of each image particle in the first image; The initial positions of the image particles are superimposed with the corresponding offset information to obtain the scattered target positions of the image particles.

9. The method according to claim 8, characterized in that The step of determining the offset information of each image particle based on the distortion degree parameter, the current scene time, and the initial position of each image particle in the first image includes: Determining a first position vector based on an initial position of each image particle in the first image; Determining a second position vector based on an initial position of each image particle in the first image and a current scene time; Determine the second position vector as the input of the vortex noise function, and determine the output vector; The product of the first position vector, the output vector and the distortion degree parameter is determined as offset information.

10. The method according to any one of claims 5 to 9, characterized in that: The method further comprises: When each of the image particles reaches the corresponding scattered target position, obtaining a first mixing weight; The first color value and the second color value of each image particle are mixed by using the first mixing weight to obtain a first mixed color value of each image particle.

11. The method according to any one of claims 5 to 9, characterized in that: The method further comprises: In the process of each image particle moving to the corresponding scattered target position, determining a second mixing weight based on the first moving distance of each image particle; The first color value and the second color value of each image particle are mixed by using the second mixing weight to obtain a second mixed color value of each image particle.

12. The method according to any one of claims 5 to 9, characterized in that: The step of performing aggregating processing on the image particles moved to the scattered target position to restore the image particles to the initial position to obtain a particle-based second image includes: In the process of each image particle returning from the corresponding scattered target position to the corresponding initial position, determining a third mixing weight based on a second moving distance of each image particle relative to the scattered target position; Using the third mixing weight, the first color value and the second color value of each image particle are mixed to obtain a third mixed color value of each image particle; When the image particles are restored to the initial position, the mixed color value of the image particles is the second color value, and a particle-based second image is obtained.

13. A video data processing device, characterized in that: The device comprises: An image acquisition module, used to acquire a first image corresponding to the last video frame of a first video currently being played, and a second image corresponding to the first video frame of a second video to be played; A particle processing module, used for displaying the first image after the first video is played, and performing particle processing on the first image to obtain a plurality of image particles; a diffusion aggregation module, configured to perform diffusion aggregation processing on the plurality of image particles based on the second image, wherein in the diffusion aggregation process, first color values ​​of the plurality of image particles are mixed with second color values ​​of corresponding positions in the second image, and after the diffusion aggregation processing, the particle-based second image is displayed; A video playing module is used to play the second video.

14. An electronic device, characterized in that: The electronic device comprises: A memory for storing computer executable instructions; A processor, configured to implement the method according to any one of claims 1 to 12 when executing the computer executable instructions stored in the memory.

15. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the method according to any one of claims 1 to 12 is implemented.

16. A computer program product comprising computer executable instructions or a computer program, characterized in that When the computer executable instructions or computer program are executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images