Video processing method, device, storage medium, equipment and computer program product

Abstract

A video processing method, apparatus, storage medium, device, and computer program product can be applied to various fields such as cloud technology, artificial intelligence, blockchain, vehicle networking, intelligent transportation, and smart home. The method includes: obtaining a plurality of image frames included in a video to be processed, each image frame including a first image area, and the first image area containing target text; preprocessing the first image area of each image frame to obtain a second image area of each image frame; determining a target image corresponding to the second image area of each image frame according to the state information of moving particles at different time points, where the state information includes color information and first position information; performing a filling process on the second image area of each image frame according to the target image corresponding to the second image area of each image frame to obtain a target video. Through the method of the present application, a video with dynamic text special effects can be generated, enriching the diversity of the display methods of text in the video.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a video processing method, a video processing device, a computer-readable storage medium, a computer device, and a computer program product. Background Art

[0002] Currently, the text in videos remains at the static special effect display. For example, designers define the font and gradient color effect of the text in the video, and add the defined text to the corresponding positions in each image frame of the video. When the video is played, the text usually shows a static display effect. It can be seen that this display method is not flexible enough and the display method is relatively single. Summary of the Invention

[0003] Embodiments of this application provide a video processing method, device, storage medium, device, and computer program product, which can generate a video with dynamic text special effects and enrich the diversity of the display methods of text in the video.

[0004] On the one hand, embodiments of this application provide a video processing method, and the method includes:

[0005] Obtain a plurality of image frames included in the video to be processed, each of the plurality of image frames includes a first image area, and the first image area contains target text;

[0006] Preprocess the first image area of each of the plurality of image frames to obtain a second image area of each of the image frames;

[0007] According to the state information of the moving particles at different time points, determine the target image corresponding to the second image area of each of the plurality of image frames, where the state information includes color information and first position information, and the first position information is the position information of the moving particles when moving according to the fluid motion rule, and the moving particles are moving objects with predetermined attribute characteristics;

[0008] Fill the second image area of each of the image frames according to the target image corresponding to the second image area of each of the image frames to obtain a target video.

[0009] On the one hand, embodiments of this application provide a video processing device, and the device includes:

[0010] An obtaining unit, configured to obtain a plurality of image frames included in the video to be processed, each of the plurality of image frames includes a first image area, and the first image area contains target text;

[0011] A processing unit for preprocessing a first image region of each of the plurality of image frames to obtain a second image region of each of the image frames;

[0012] The processing unit is further configured to determine a target image corresponding to the second image region of each of the plurality of image frames according to the state information of the moving particles at different time points, where the state information includes color information and first position information, and the first position information is the position information of the moving particles when moving according to the fluid motion rules, and the moving particles are moving objects with predetermined attribute characteristics;

[0013] The processing unit is further configured to perform a filling process on the second image region of each of the image frames according to the target image corresponding to the second image region of each of the image frames to obtain a target video.

[0014] On the one hand, an embodiment of the present application provides a computer device, which includes a processor, a communication interface, and a memory. The processor, the communication interface, and the memory are interconnected. Among them, the memory stores a computer program, and the processor is configured to call the computer program to execute the video processing method of any of the above possible implementation manners.

[0015] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the video processing method of any of the above possible implementation manners is implemented.

[0016] On the one hand, an embodiment of the present application further provides a computer program product, where the computer program product includes a computer program or computer instructions, and when the computer program or computer instructions are executed by a processor, the steps of the video processing method provided by the embodiment of the present application are implemented.

[0017] On the one hand, an embodiment of the present application further provides a computer program, where the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium, and a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to enable the computer device to execute the video processing method provided by the embodiment of the present application.

[0018] In the embodiments of the present application, multiple image frames included in the video to be processed can be obtained. Each image frame includes a first image area, and the first image area contains the target text. Then, the first image area of each image frame among the multiple image frames is preprocessed to obtain a second image area of each image frame. According to the state information of the moving particles at different time points, the target image corresponding to the second image area of each image frame is determined. The state information includes color information and first position information, and the first position information is the position information of the moving particles when moving according to the fluid motion rules. The moving particles are moving objects with predetermined attribute characteristics. Finally, according to the target image corresponding to the second image area of each image frame, the second image area of each image frame is filled and processed, and the target video can be obtained. Through the method of the present application, a video with dynamic text special effects can be generated, enriching the diversity of the display methods of the text in the video. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 Schematic diagram of the system architecture of a video processing system provided by an embodiment of the present application;

[0021] Figure 2 Schematic flow chart of a video processing method provided by an embodiment of the present application Figure 1 ;

[0022] Figure 3 Schematic flow chart of a video processing method provided by an embodiment of the present application Figure 2 ;

[0023] Figure 4 Schematic example diagram of a second image area provided by an embodiment of the present application;

[0024] Figure 5 Schematic diagram of a background grid provided by an embodiment of the present application;

[0025] Figure 6 Schematic diagram of solving the velocity information by interpolation involved in an embodiment of the present application;

[0026] Figure 7 Schematic diagram of a dynamic text special effect provided by an embodiment of the present application;

[0027] Figure 8 Schematic flow chart of a video processing method provided by an embodiment of the present application Figure 3 ;

[0028] Figure 9 It is a schematic structural diagram of a video processing device provided by an embodiment of the present application;

[0029] Figure 10 It is a schematic structural diagram of a computer device provided by an embodiment of the present application. Specific embodiments

[0030] Next, the technical methods in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present application.

[0031] The present application proposes a video processing method, which can generate videos with dynamic text special effects, enrich the diversity of text display methods in videos, and can be applied to various fields or scenarios such as cloud technology, artificial intelligence, blockchain, Internet of Vehicles, intelligent transportation, and smart home. In one embodiment, the video processing method can be implemented based on computer vision technology in artificial intelligence technology. Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, including both hardware-level technologies and software-level technologies. Artificial intelligence basic technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, large video processing technology, operation / interaction systems, and mechatronics. Artificial intelligence software technologies mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, machine learning / deep learning, autonomous driving, and intelligent transportation. Computer vision technology is a science that studies how to make machines "see". Further, it refers to using cameras and computers to replace the human eye to perform machine vision such as target recognition, tracking, and measurement on targets, and further perform graphic processing to make the computer process into an image that is more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes technologies such as image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and also includes common biometric recognition technologies such as face recognition and fingerprint recognition.

[0032] Please refer to Figure 1 , Figure 1 It is a schematic system architecture diagram of a video processing system provided by an embodiment of the present application; Figure 1The system architecture shown can be used to implement the video processing method proposed in the embodiments of the present application. As Figure 1 shown, the system architecture includes: a server 10 and multiple terminal devices 11 (3 are shown as examples in the figure).

[0033] Among them, the server 10 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. The terminal device 11 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart voice interaction device, a smart home appliance, a vehicle-mounted terminal, etc., but is not limited thereto. Figure 1 The terminal device 11 shown is connected to the server 10 through a network.

[0034] Figure 1 The system architecture shown can implement the video processing method provided in the embodiments of the present application. Taking the server 10 or the terminal device 11 executing this method as an example, its implementation process generally includes: the server 10 or the terminal device 11 can obtain multiple image frames included in the video to be processed, each image frame in the multiple image frames includes a first image area, the first image area contains target text, and then preprocess the first image area of each image frame in the multiple image frames to obtain a second image area of each image frame. According to the state information of the moving particles at different time points, determine the target image corresponding to the second image area of each image frame in the multiple image frames. The state information includes color information and first position information. The first position information is the position information of the moving particles when moving according to the fluid motion rule. The moving particles are moving objects with predetermined attribute characteristics. Finally, according to the target image corresponding to the second image area of each image frame, perform a filling process on the second image area of each image frame, and a target video can be obtained.

[0035] In one embodiment, the video processing method can be jointly executed by the server 10 and the terminal device 11. Specifically, the server 10 receives multiple image frames included in the video to be processed sent by the terminal device 11, then processes the multiple image frames included in the video to be processed by using the video processing method of the present application to obtain a target video, and returns the target video to the terminal device 11.

[0036] In the embodiments of the present application, since the moving particles are constantly moving according to the fluid motion rules, different target images can be generated based on the state information at different time points, so that the target texts in each image frame filled with different target images can present different special effects. Therefore, when the target video is played, the text can present dynamic special effects. Therefore, the present application can generate appropriate dynamic special effects for the text in the video based on the particle motion characteristics of the moving particles, enriching the diversity of the display methods of the text in the video.

[0037] It can be understood that the schematic diagram of the system architecture described in the embodiments of the present application is to more clearly illustrate the technical solutions of the embodiments of the present application, and does not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0038] The following elaborates in detail on the specific implementation manners of the video processing method.

[0039] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of a video processing method provided by an embodiment of the present application Figure 1 . This method can be applied to the server 10 and / or the terminal device 11 in the above video processing system. The method includes:

[0040] S201. Obtain a plurality of image frames included in the video to be processed. Each image frame in the plurality of image frames includes a first image area, and the first image area contains target text.

[0041] In the embodiments of the present application, the video to be processed can be any video that needs to add dynamic text special effects. The video to be processed includes a plurality of image frames, and each image frame in the plurality of image frames includes a first image area. The first image area can be of any shape and can be a partial image area or the entire image area in each image frame. The first image area contains target text, and the target text can be included in each image frame itself or added to each image frame. The target text can include characters and text symbols. The characters can be Chinese characters, English characters, etc., and the text symbols can be letter symbols, punctuation marks, formula symbols, symbol patterns, etc.

[0042] S202. Preprocess the first image area of each image frame in the plurality of image frames to obtain a second image area of each image frame.

[0043] In the embodiments of the present application, preprocessing can be performed on the image region corresponding to the target text in the first image region of each image frame. The image region corresponding to the target text refers to the image region where the target text drawn in the first image region is located. The image region corresponding to the target text in the second image region obtained after preprocessing is hollowed out, that is, there is no color filling in the image region corresponding to the target text.

[0044] S203. Determine the target image corresponding to the second image region of each image frame in multiple image frames according to the state information of the moving particles at different time points. The state information includes color information and first position information. The first position information is the position information of the moving particles when moving according to the fluid motion rules. The moving particles are moving objects with predetermined attribute characteristics.

[0045] The moving particles can be moving objects with predetermined attribute characteristics set artificially. The predetermined attribute characteristics can be set by imitating the inherent attributes of real particles and can involve multiple dimensions such as particle radius, particle mass, particle density, and particle volume.

[0046] In one embodiment, the color information of the moving particles can be defined artificially in advance. For example, 1000 moving particles are set to be yellow and 1000 moving particles are set to be white; or the pixel values of each pixel point in the image region corresponding to the target text can also be determined as the color information of the moving particles.

[0047] The moving particles can move continuously according to the fluid motion rules in a two-dimensional space. The position information of the moving particles at a certain time point can be used as the first position information of the moving particles. When generating a target image using the state information of the moving particles at a certain time point, the color rendering range can be determined using the first position information and particle radius of the moving particles at this time point, and the color to be rendered in this color rendering range can be determined using the color information of the moving particles, so as to obtain the target image.

[0048] It should be noted that since the video to be processed includes multiple image frames, it is necessary to obtain the first position information of the moving particles at different time points to generate the target image corresponding to each image frame.

[0049] S204. Perform a filling process on the second image region of each image frame according to the target image corresponding to the second image region of each image frame to obtain the target video.

[0050] Since the image region corresponding to the target text in the second image region is hollowed out, the target image can be used to fill the hollowed-out part of the second image region. For example, the target image is overlapped with the second image region to determine which part of the target image can fill the hollowed-out part, and the image region of the target image that can fill the hollowed-out content is used as the supplement of the missing content in the second image region, so as to obtain each filled image frame. Each filled image frame can constitute the target video. Since the moving particles are always moving and different target images are generated, when the target video is played, the target text in the target video will present a dynamic special effect.

[0051] In the embodiment of the present application, multiple image frames included in the video to be processed can be obtained. Each image frame includes a first image region, and the first image region contains the target text. Then, the first image region of each image frame among the multiple image frames is preprocessed to obtain the second image region of each image frame. According to the state information of the moving particles at different time points, the target image corresponding to the second image region of each image frame is determined. The state information includes color information and the first position information. The first position information is the position information of the moving particles when moving according to the fluid motion rule. The moving particles are moving objects with predetermined attribute characteristics. Finally, according to the target image corresponding to the second image region of each image frame, the second image region of each image frame is filled, and the target video can be obtained. Through the method of the present application, different target images can be used to fill the target text in each image frame, so that when the target video is played, the target text can present different special effects. Therefore, the present application can generate appropriate dynamic special effects for the text in the video based on the particle motion characteristics of the moving particles, enriching the diversity of the display methods of the text in the video.

[0052] Please refer to Figure 3 , Figure 3 which is a schematic flow chart of a video processing method provided by an embodiment of the present application. Figure 2 This method can be applied to the server 10 and / or the terminal device 11 in the above video processing system. The method includes but is not limited to the following steps:

[0053] S301. Obtain multiple image frames included in the video to be processed. Each image frame among the multiple image frames includes a first image region, and the first image region contains the target text.

[0054] In one implementation, the text image region of each image frame can be recognized. The text image region contains text, which can be specifically implemented by Optical Character Recognition (OCR) technology. OCR technology refers to the recognition of text in an image, and in addition, OCR technology can also return the text in the image in text form. When there are multiple text image regions, the largest text image region or the text image region larger than a preset area among the multiple text image regions can be used as the first image region, and the text in the largest text image region or the text image region larger than the preset area is the target text. That is, the first image region can be one or more, and this application does not limit this.

[0055] S302. Preprocess the first image region of each image frame among multiple image frames to obtain the second image region of each image frame.

[0056] In one embodiment, the contour information of the target text in the first image region of each image frame can be obtained. For example, the contour information of the target text is extracted through the built-in function of OPENCV, where OPENCV is a cross-platform computer vision and machine learning software library distributed under the BSD license (open source). Further, according to the contour information of the target text, the image region corresponding to the target text in the first image region of each image frame is hollowed out to obtain the second image region of each image frame. For example, Figure 4 the left figure in is the first image region, Figure 4 the right figure in is the second image region. It can be seen that the internal image region of the target text "streaming light" has been hollowed out.

[0057] S303. Determine the first position information of each moving particle when each moving particle moves for N time steps according to the fluid motion rule, where N is an integer greater than or equal to 0.

[0058] In the embodiment of this application, the time step can be any time duration unit, or it can be determined by the frame rate of the video to be processed. For example, if the frame rate of the video to be processed is 50 frames per second and each image frame is played for 0.02 seconds, then the time step can be 0.02 seconds, that is, the time step is the playing duration of the image frames in the video to be processed.

[0059] In one implementation, the background grid can be determined according to the first image region. Specifically, the shapes and sizes of the first image regions of each image frame determined are the same. For any first image region of an image frame, the pixel points within the first image region of this image frame are represented by 1, and the pixel points outside the first image region of this image frame are represented by 0. Then a grid unit is a pixel point marked with 1, and all the pixel points marked with 1 constitute the background grid.

[0060] Further, when N is equal to 0, that is, when each moving particle moves for 0 time steps, the initial position information of each moving particle in the background grid can be determined. This initial position information can be predefined artificially, and the initial position information of each moving particle is determined as the first position information when each moving particle moves for N time steps. When N is greater than 0, according to the fluid motion rules, the second position information of each moving particle in the background grid when moving for N time steps is determined, and the second position information of each moving particle is determined as the first position information when each moving particle moves for N time steps.

[0061] By adopting the embodiment of the present application, the motion boundary of the moving particles can be restricted by the background grid. Since the hollowed-out part is located in the second image area, when the motion range of the moving particles exceeds the background grid, that is, exceeds the second image area, it is meaningless to fill the hollowed-out part.

[0062] In the embodiment of the present application, each moving particle can be made to simulate the motion of a fluid in the background grid according to the fluid motion rules, and the second position information of each moving particle in the background grid when moving for N time steps is used as the first position information when each moving particle moves for N time steps.

[0063] Specifically, the transmission control rule, the momentum control rule, and the force control rule (used to describe the fluid motion rules) can be determined based on the fluid simulation model and the configuration parameters. In one embodiment, the fluid simulation model used in the present application is established based on an incompressible inviscid fluid. The incompressible inviscid fluid has the following characteristics: the fluid viscosity is 0 and the fluid density is constant. Then the configuration parameters include the fluid viscosity (0) and the fluid density (constant), which can ensure that the transmission control rule, the momentum control rule, and the force control rule are determined for the incompressible inviscid fluid. The present application specifically uses the Euler equation (a fluid simulation model) to simulate the incompressible inviscid fluid. The Euler equation includes a system of equations, and each equation in the system of equations represents mass conservation (continuity), momentum conservation, and energy conservation respectively. The present application mainly uses the momentum control rule and the mass control rule in the system of equations, as shown in the following formulas (1) and (2), which are respectively the momentum control rule and the mass control rule applicable to the incompressible inviscid fluid. The momentum control rule is determined according to the law of conservation of momentum, which means that the increment of the total momentum of the fluid in the control volume = the momentum of the fluid flowing into the control volume + the increment of the momentum generated by the external force. The mass control rule is obtained according to the law of conservation of mass, which means that the increment of the total mass of the fluid in the control volume = the mass of the fluid flowing into the control volume.

[0064]

[0065]

[0066] Among them, represents the acceleration due to gravity, represents the velocity information of the moving particle, t represents time, ρ represents the fluid density, p represents the fluid pressure exerted on the moving particle, and D represents the material derivative.

[0067] The following equation (3) is the derivation formula of the material derivative.

[0068]

[0069] Among them, represents the position information of the moving particle, q represents a general physical quantity, such as the velocity information, temperature information, and position information of the moving particle.

[0070] In addition, from the Lagrangian perspective, the rate of change of q with respect to time is 0, and based on this, the transport control rule is defined.

[0071]

[0072] When this application uses the transport control rule, q mainly represents the position information of the moving particle. Therefore, the transport control rule is used to indicate that the rate of change of the position information of the moving particle with respect to time is 0.

[0073] At the same time, the configuration parameters also include the type of acting force, such as gravity, wind force, etc. This application assumes that the type of acting force exerted on the moving particle is gravity, then the force control rule can be determined by the following equation (5). The force control rule is used to indicate the motion control of the applied acting force on the moving particle. The following equation (5) reveals that when only gravity is applied, the change in velocity is affected by the acceleration due to gravity.

[0074]

[0075] In addition, equations (1) can be discretized into the following equation (6) through equations (3) and (4). Equation (6) is still the momentum control rule.

[0076]

[0077] It should be noted that other types of fluids can be simulated according to actual needs, such as incompressible viscous fluids, etc. This application does not make any limitations in this regard. In addition, the configuration parameters can also include the initial position information of the above-mentioned moving particle.

[0078] After obtaining the momentum control rule, the transmission control rule, and the force control rule, the velocity information and position information of a moving particle when moving in accordance with the fluid motion rule can be solved based on this. It is understandable that the fluid motion of the moving particle needs to follow the momentum control rule, the transmission control rule, and the force control rule.

[0079] Refer to Figure 5 , Figure 5 FIG. is a schematic diagram of a background grid. Each small cell represents a grid unit. It can be seen that the background grid can include multiple grid units, and auxiliary velocity information is stored at the four grid points (vertices) of each grid unit. The auxiliary velocity information has the magnitude and direction of the velocity.

[0080] In one embodiment, the third position information of each moving particle when moving for N - 1 time steps can be obtained. According to the third position information of each moving particle, the target grid unit where each moving particle is located in the background grid can be determined, and the auxiliary velocity information stored at the grid points of the target grid unit can be obtained. The auxiliary velocity information is determined according to the first velocity information of each moving particle located in the target grid unit. The first velocity information is the velocity information of each moving particle when moving in accordance with the fluid motion rule. Then, according to the third position information of each moving particle, the time step, and the auxiliary velocity information stored at the grid points of the target grid unit, the second position information of each moving particle when moving for N time steps can be determined.

[0081] Taking moving particle 1 as an example for illustration, assume that the third position information of moving particle 1 when moving for N - 1 time steps is What needs to be determined now is the second position information of moving particle 1 when moving for N time steps, that is, As Figure 6 shown, the auxiliary velocity information of the four grid points of the target grid unit where moving particle 1 is located is Then, through the following equations (7) - (9), the third velocity information of moving particle 1 when moving for N - 1 time steps can be interpolated using the auxiliary velocity information of the four grid points of the target grid unit where a, b, c, and d represent the distances between moving particle 1 and the boundaries of the target grid unit.

[0082]

[0083]

[0084]

[0085] Furthermore, the third position information of moving particle 1 time step Δt and the third velocity information Substituting into the following formula (10), the second position information of the moving particle 1 is obtained

[0086]

[0087] If more accurate second position information is needed, as shown in the following formulas (11) and (12), the second position information of the moving particle 1 can also be determined using the second-order Runge-Kutta format

[0088]

[0089]

[0090] The transmission control rule shown in formula (4) can be discretized to determine in formula (12). The specific calculation formula is as shown in the following formula (13).

[0091]

[0092] Among them, assuming that is (x2, y2), which can be obtained through formula (11), is (x1, y1), then Δx1 is (x2 - x1), and Δy1 is (y2 - y1).

[0093] If the second position information of the moving particle is outside the background grid, the center position of the grid cell closest to the second position information can be taken as the second position information of the moving particle. Finally, taking the second position information of the moving particle as the first position information of the moving particle can ensure that the moving particle is within the background grid.

[0094] The processing method of this application for generating the third velocity information of the moving particle 1 using the auxiliary velocity information of the four grid points of the target grid cell can avoid wasting computing resources and storage resources to memorize the velocity information of a large number of moving particles at each time point. However, every other time step Δt, the velocity information stored at the four grid points of the target grid cell needs to be updated according to the first velocity information of each moving particle located in the target grid cell. The following takes each grid cell as an example for detailed description.

[0095] The target moving particle is any moving particle currently located in each grid cell. In one embodiment, according to the transmission control rule, and the first position information of the target moving particle when it moves N time steps, and the third position information when it moves N - 1 time steps the second velocity information of the target moving particle is determined as shown in the following formula (14).

[0096]

[0097] Among them, assume that is (x3, y3), is (x1, y1), then Δx2 is (x3 - x1), and Δy2 is (y3 - y1).

[0098] Furthermore, the second velocity information of the target moving particle is updated according to the momentum control rule and the force control rule to obtain the first velocity information of the target moving particle when moving according to the fluid motion rule

[0099]

[0100]

[0101] Among them, Equation (15) is an update implemented based on the discretized force control rule, which applies the influence of gravity to the moving particle; Equation (16) is an update based on the discretized momentum control rule on the basis of Equation (15), which projects the velocity onto a divergence-free field, that is, the velocity field satisfies a divergence of 0, and is used to ensure an incompressible and inviscid fluid.

[0102] After determining the first velocity information of each moving particle currently located in each grid cell when moving according to the fluid motion rule, the influence factor corresponding to each moving particle can be determined according to the particle mass of each moving particle. The influence factor is equal to the particle mass of moving particle 1 / the sum of the particle masses of all moving particles in the grid cell where moving particle 1 is located. The influence factor is used to indicate the degree of influence of the moving particle on the grid cell where it is located. Then, according to the influence factor corresponding to each moving particle and the first velocity information of each moving particle, the target velocity information is determined. For example, the sum of the products of the influence factors of each moving particle and the corresponding first velocity information is used as the target velocity information. Finally, the auxiliary velocity information stored at the grid points of each grid cell can be updated according to the target velocity information. Specifically, in Equation (9) can be replaced with the target velocity information. a, b, c, and d can be determined according to the boundary distance between the first position information of the target moving particle and the grid cell where the target moving particle is located. Then, the auxiliary velocity information of the four grid points of the grid cell is interpolated backward according to Equations (7) - (9), and the auxiliary velocity information stored at the four grid points of the grid cell is updated using the interpolated auxiliary velocity information.

[0103] S304. Obtain the color information of each moving particle.

[0104] In one embodiment, the theme color information can be determined according to the color information included in each image frame. For example, the most common color information included in each image frame is obtained. If they are similar colors, they are classified into the same type of color information, and then the most common color information is used as the theme color information. Further, one or more reference color information having a color matching relationship with the theme color information are determined. The color matching relationship can be a contrast color matching relationship, a similar color matching relationship, etc. For example, if the theme color information is golden yellow, the reference color relationship can be light yellow (similar relationship), purple (contrast relationship). Preferably, in order to make the text present a dynamic special effect, the reference color information can include multiple. Finally, according to one or more reference color information, the color information of each moving particle is determined. For example, 1000 moving particles are yellow and 1000 moving particles are light yellow.

[0105] S305. According to the first position information and color information of each moving particle, determine the target image corresponding to the second image area of the (N + 1)-th image frame in multiple image frames.

[0106] The multiple image frames can be arranged in chronological order. In a feasible embodiment, after obtaining the first position information and color information of each moving particle when moving N time steps, the target image corresponding to the second image area of the (N + 1)-th image frame in multiple image frames can be generated.

[0107] S306. According to the target image corresponding to the second image area of each image frame, perform a filling process on the second image area of each image frame to obtain a target video.

[0108] In one embodiment, the target image corresponding to the second image area of each image frame can be set as the background layer of the second image area of each image frame to obtain each filled image frame, and then the target video is generated according to each filled image frame. As Figure 7 shown, for the dynamic change special effect of the target text "flowing light" in the target video, it can be seen that as the moving particles of different grayscales move, the target text "flowing light" can present different display effects.

[0109] In a feasible embodiment, as Figure 8As shown in the figure, after extracting multiple text image regions by using the OCR technology, the largest text image region can be used as the first image region, and the text contour of the target text in the first image region is extracted to obtain the contour information of the target text. Then, according to the contour information of the target text, the image region corresponding to the target text is determined, and the image region corresponding to the target text is filled to determine the background grid. Specifically, each pixel point in the image region corresponding to the target text is marked with 1, so as to obtain the background grid. Then, based on the fluid motion rules, the fluid motion of the moving particles is simulated in the background grid to generate the target video. The background grid determined in this way can make the generated target image just fill the hollow part in the second image region.

[0110] It should be noted that the video processing method provided in this application can also be used to generate dynamic text special effects in real time. For example, the time step is the playing time of each image frame. The second image region in the currently playing image frame can be filled with the determined target image every other time step. When the next image frame is played, the second image region in the next image frame is filled with the newly determined target image.

[0111] In the embodiments of this application, by simulating the fluid of the moving particles according to the fluid motion rules, the existing text special effects can be dynamicized, appropriate text dynamic special effects can be generated in the video, the diversity of the display methods of the text in the video can be enriched, and at the same time, the realism can be improved and the rendering speed can be optimized; in addition, since the color information of the moving particles can be specified manually or specified by the theme color, customized special effect tones can be generated.

[0112] It can be understood that in the specific implementation manners of this application, relevant data such as the video to be processed are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0113] The above has elaborated in detail the method of the embodiments of this application. To facilitate the better implementation of the above method of the embodiments of this application, correspondingly, the device of the embodiments of this application is provided below. Please refer to Figure 9 , Figure 9 which is a schematic structural diagram of a video processing device provided by the embodiments of this application. The video processing device 90 may include:

[0114] An obtaining unit 901, configured to obtain multiple image frames included in the video to be processed, where each image frame in the multiple image frames includes a first image region, and the first image region contains target text;

[0115] A processing unit 902, configured to preprocess a first image region of each of the multiple image frames to obtain a second image region of each of the image frames;

[0116] The processing unit 902 is further configured to determine a target image corresponding to the second image region of each of the multiple image frames according to state information of a moving particle at different time points, where the state information includes color information and first position information, and the first position information is the position information of the moving particle when moving according to fluid motion rules, and the moving particle is a moving object with predetermined attribute characteristics;

[0117] The processing unit 902 is further configured to perform a filling process on the second image region of each of the image frames according to the target image corresponding to the second image region of each of the image frames to obtain a target video.

[0118] In an embodiment, the processing unit 902 is specifically configured to: determine contour information of the target text in the first image region of each of the image frames; perform a hollowing process on an image region corresponding to the target text in the first image region of each of the image frames according to the contour information of the target text to obtain the second image region of each of the image frames.

[0119] In an embodiment, the processing unit 902 is specifically configured to: set the target image corresponding to the second image region of each of the image frames as a background layer of the second image region of each of the image frames to obtain each filled image frame; generate a target video according to each of the filled image frames.

[0120] In an embodiment, the obtaining unit 901 is specifically configured to: determine first position information of each of the multiple moving particles when moving N time steps according to fluid motion rules, where N is an integer greater than or equal to 0;

[0121] The processing unit 902 is specifically configured to: determine a target image corresponding to the second image region of the (N + 1)-th image frame among the multiple image frames according to the first position information and color information of each of the moving particles.

[0122] In an embodiment, the processing unit 902 is specifically configured to: determine theme color information according to color information included in each of the image frames; determine one or more reference color information having a color matching relationship with the theme color information; determine the color information of each of the moving particles according to the one or more reference color information.

[0123] In one embodiment, the processing unit 902 is specifically configured to: determine a background grid according to the first image region; if N is equal to 0, determine the initial position information of each moving particle in the background grid, and determine the initial position information of each moving particle as the first position information of each moving particle; if N is greater than 0, determine the second position information of each moving particle when moving N time steps in the background grid according to the fluid motion rule, and determine the second position information of each moving particle as the first position information of each moving particle.

[0124] In one embodiment, the background grid includes a plurality of grid cells, and auxiliary velocity information is stored at the grid points of each grid cell; the obtaining unit 901 is specifically configured to: obtain the third position information of each moving particle when moving N-1 time steps.

[0125] The processing unit 902 is specifically configured to: determine the target grid cell where each moving particle is located in the background grid according to the third position information of each moving particle; obtain the auxiliary velocity information stored at the grid point of the target grid cell, where the auxiliary velocity information is determined according to the first velocity information of each moving particle located in the target grid cell, and the first velocity information is the velocity information when each moving particle moves according to the fluid motion rule; determine the second position information of each moving particle according to the third position information of each moving particle, the time step, and the auxiliary velocity information stored at the grid point of the target grid cell.

[0126] In one embodiment, the obtaining unit 901 is specifically configured to: obtain the first velocity information of each moving particle located in each grid cell when moving according to the fluid motion rule.

[0127] The processing unit 902 is specifically configured to: determine the influence factor corresponding to each moving particle according to the particle mass of each moving particle; determine the target velocity information according to the influence factor corresponding to each moving particle and the first velocity information of each moving particle; update the auxiliary velocity information stored at the grid point of each grid cell according to the target velocity information.

[0128] In one embodiment, the processing unit 902 is specifically configured to: determine a transmission control rule, a momentum control rule, and a force control rule based on a fluid simulation model and configuration parameters, where the transmission control rule is used to indicate the rate of change of the position information of moving particles over time, the momentum control rule is determined based on the law of conservation of momentum, and the force control rule is used to indicate the motion control of the applied force on the moving particles, and the configuration parameters include one or more of the type of applied force, initial position information, fluid viscosity, and fluid density; determine the second velocity information of the target moving particle according to the transmission control rule, and the first position information and the third position information of the target moving particle, where the target moving particle is any moving particle currently located in each grid cell; update the second velocity information of the target moving particle according to the momentum control rule and the force control rule to obtain the first velocity information of the target moving particle when moving according to the fluid motion rule.

[0129] It can be understood that the functions of the functional units of the video processing device described in the embodiments of the present application can be specifically implemented according to the methods in the above method embodiments, and the specific implementation process can refer to the relevant descriptions of the above method embodiments, which will not be elaborated here.

[0130] In the embodiments of the present application, multiple image frames included in the video to be processed can be obtained. Each image frame includes a first image area, and the first image area contains target text. Then, the first image area of each image frame among the multiple image frames is preprocessed to obtain a second image area of each image frame. According to the state information of the moving particles at different time points, the target image corresponding to the second image area of each image frame is determined. The state information includes color information and first position information, and the first position information is the position information of the moving particles when moving according to the fluid motion rule. The moving particles are moving objects with predetermined attribute characteristics. Finally, according to the target image corresponding to the second image area of each image frame, the second image area of each image frame is filled to obtain the target video. Through the method of the present application, a video with dynamic text special effects can be generated, enriching the diversity of the display methods of text in the video.

[0131] As Figure 10 shown, Figure 10 is a schematic structural diagram of a computer device provided by an embodiment of the present application. The internal structure of the computer device 100 is as Figure 10 shown, including: one or more processors 1001, a memory 1002, and a communication interface 1003. The above processors 1001, memory 1002, and communication interface 1003 can be connected through a bus 1004 or other means. In the embodiments of the present application, the connection through the bus 1004 is taken as an example.

[0132] Among them, the processor 1001 (or CPU (Central Processing Unit)) is the computing core and control core of the computer device 100. It can parse various instructions within the computer device 100 and process various data of the computer device 100. For example, the CPU can be used to parse the power-on and power-off instructions sent by the user to the computer device 100 and control the computer device 100 to perform power-on and power-off operations. Another example is that the CPU can transmit various interaction data between the internal structures of the computer device 100, and so on. The communication interface 1003 may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi, mobile communication interfaces, etc.), and is controlled by the processor 1001 to send and receive data. The memory 1002 (Memory) is the memory device in the computer device 100 and is used to store computer programs and data. It can be understood that the memory 1002 here can include both the built-in memory of the computer device 100 and, of course, the extended memory supported by the computer device 100. The memory 1002 provides a storage space, and this storage space stores the operating system of the computer device 100, which may include but is not limited to: Windows system, Linux system, Android system, iOS system, and so on. This application does not make any limitations in this regard. The processor 1001 performs the following operations by running the computer program stored in the memory 1002:

[0133] Obtain a plurality of image frames included in the video to be processed, each image frame in the plurality of image frames includes a first image area, and the first image area contains target text;

[0134] Preprocess the first image area of each image frame in the plurality of image frames to obtain a second image area of each image frame;

[0135] According to the state information of the moving particles at different time points, determine the target image corresponding to the second image area of each image frame in the plurality of image frames. The state information includes color information and first position information, and the first position information is the position information of the moving particles when moving according to the fluid motion rules. The moving particles are moving objects with predetermined attribute characteristics;

[0136] Perform a filling process on the second image area of each image frame according to the target image corresponding to the second image area of each image frame to obtain a target video.

[0137] In one embodiment, the processor 1001 is specifically configured to: determine the contour information of the target text in the first image area of each image frame; perform a hollowing process on the image area corresponding to the target text in the first image area of each image frame according to the contour information of the target text, so as to obtain the second image area of each image frame.

[0138] In one embodiment, the processor 1001 is specifically configured to: set the target image corresponding to the second image area of each image frame as the background layer of the second image area of each image frame, so as to obtain each filled image frame; generate a target video according to each filled image frame.

[0139] In one embodiment, the processor 1001 is specifically configured to: determine the first position information of each moving particle among a plurality of moving particles when moving N time steps according to the fluid motion rule, where N is an integer greater than or equal to 0; determine the target image corresponding to the second image area of the (N + 1)-th image frame among the plurality of image frames according to the first position information and color information of each moving particle.

[0140] In one embodiment, the processor 1001 is specifically configured to: determine the theme color information according to the color information included in each image frame; determine one or more reference color information having a color matching relationship with the theme color information; determine the color information of each moving particle according to the one or more reference color information.

[0141] In one embodiment, the processor 1001 is specifically configured to: determine a background grid according to the first image area; if N is equal to 0, determine the initial position information of each moving particle among the plurality of moving particles in the background grid, and determine the initial position information of each moving particle as the first position information of each moving particle; if N is greater than 0, determine the second position information of each moving particle when moving N time steps in the background grid according to the fluid motion rule, and determine the second position information of each moving particle as the first position information of each moving particle.

[0142] In one embodiment, the background grid includes a plurality of grid cells, and auxiliary velocity information is stored at the grid points of each grid cell; specifically, the processor 1001 is configured to: obtain third position information of each moving particle at N-1 time steps; specifically, the processor 1001 is configured to: determine a target grid cell in the background grid where each moving particle is located according to the third position information of each moving particle; obtain the auxiliary velocity information stored at the grid point of the target grid cell, where the auxiliary velocity information is determined according to the first velocity information of each moving particle located in the target grid cell, and the first velocity information is the velocity information of each moving particle when moving according to the fluid motion rules; determine the second position information of each moving particle according to the third position information of each moving particle, the time step, and the auxiliary velocity information stored at the grid point of the target grid cell.

[0143] In one embodiment, the processor 1001 is specifically configured to: obtain the first velocity information of each moving particle located in each grid cell when moving according to the fluid motion rules; determine an influence factor corresponding to each moving particle according to the particle mass of each moving particle; determine target velocity information according to the influence factor corresponding to each moving particle and the first velocity information of each moving particle; update the auxiliary velocity information stored at the grid point of each grid cell according to the target velocity information.

[0144] In one embodiment, the processor 1001 is specifically configured to: determine a transmission control rule, a momentum control rule, and a force control rule based on a fluid simulation model and configuration parameters, where the transmission control rule is used to indicate the change rate of the position information of the moving particle over time, the momentum control rule is determined based on the law of conservation of momentum, and the force control rule is used to indicate the motion control of the applied force on the moving particle, and the configuration parameters include one or more of the type of applied force, initial position information, fluid viscosity, and fluid density; determine the second velocity information of the target moving particle according to the transmission control rule, and the first position information and the third position information of the target moving particle, where the target moving particle is any moving particle currently located in each grid cell; update the second velocity information of the target moving particle according to the momentum control rule and the force control rule to obtain the first velocity information of the target moving particle when moving according to the fluid motion rules.

[0145] Specifically, in implementation, the processor 1001, the memory 1002, and the communication interface 1003 described in the embodiments of the present application may execute the implementation manners described in a video processing method provided by the embodiments of the present application, or may execute the implementation manners described in a video processing device provided by the embodiments of the present application, which will not be elaborated herein.

[0146] In an embodiment of the present application, multiple image frames included in a video to be processed can be obtained. Each image frame includes a first image area that contains target text. Then, preprocessing is performed on the first image area of each image frame among the multiple image frames to obtain a second image area of each image frame. According to the state information of moving particles at different time points, a target image corresponding to the second image area of each image frame is determined. The state information includes color information and first position information. The first position information is the position information of the moving particles when they move according to the rules of fluid motion. The moving particles are moving objects with predetermined attribute characteristics. Finally, filling processing is performed on the second image area of each image frame according to the target image corresponding to the second image area of each image frame, and a target video can be obtained. Through the method of the present application, a video with dynamic text special effects can be generated, enriching the diversity of the display methods of text in the video.

[0147] An embodiment of the present application also provides a computer-readable storage medium. A computer program is stored in the computer-readable storage medium. When it runs on a computer device, it causes the computer device to execute the video processing method in any of the above possible implementation manners. Its specific implementation manner can refer to the foregoing description and will not be elaborated here.

[0148] An embodiment of the present application also provides a computer program product. The computer program product includes a computer program or computer instructions. When the computer program or computer instructions are executed by a processor, the steps of the video processing method provided by the embodiment of the present application are implemented. Its specific implementation manner can refer to the foregoing description and will not be elaborated here.

[0149] An embodiment of the present application also provides a computer program. The computer program includes computer instructions. The computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to execute the video processing method provided by the embodiment of the present application. Its specific implementation manner can refer to the foregoing description and will not be elaborated here.

[0150] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, some steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0151] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The storage medium can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, etc.

[0152] The foregoing disclosure is only part of the embodiments of the present application. Of course, it cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application still fall within the scope covered by the present application.

Claims

1. A video processing method, characterized in that, The method includes: Obtaining a plurality of image frames included in the video to be processed, each image frame of the plurality of image frames including a first image region that contains target text; Preprocessing the first image region of each image frame of the plurality of image frames to obtain a second image region of each image frame; Determining first position information of each of a plurality of moving particles when each moving particle moves for N time steps according to fluid motion rules, where N is an integer greater than or equal to 0; wherein, if N is equal to 0, then determining the initial position information of each moving particle in the background grid as the first position information of each moving particle; if N is greater than 0, then determining second position information of each moving particle in the background grid when each moving particle moves for N time steps according to fluid motion rules, and determining the second position information of each moving particle as the first position information of each moving particle; the background grid is determined according to the first image region; Obtaining color information of each of the moving particles; Determining a target image corresponding to the second image region of the (N + 1)-th image frame among the plurality of image frames according to the first position information and color information of each of the moving particles; the moving particles are moving objects with predetermined attribute characteristics; Performing a filling process on the second image region of each image frame according to the target image corresponding to the second image region of each image frame to obtain a target video.

2. The method according to claim 1, wherein The preprocessing the first image region of each image frame of the plurality of image frames to obtain a second image region of each image frame includes: Determining contour information of the target text in the first image region of each image frame; Performing a hollowing process on the image region corresponding to the target text in the first image region of each image frame according to the contour information of the target text to obtain a second image region of each image frame.

3. The method according to claim 1 or 2, characterized in that, The performing a filling process on the second image region of each image frame according to the target image corresponding to the second image region of each image frame to obtain a target video includes: Setting the target image corresponding to the second image region of each image frame as the background layer of the second image region of each image frame to obtain each filled image frame; Generating a target video according to each filled image frame.

4. The method according to claim 1, characterized in that, The obtaining color information of each of the moving particles includes: Determining theme color information according to the color information included in each image frame; Determining one or more reference color information having a color matching relationship with the theme color information; Determining color information of each of the moving particles according to the one or more reference color information.

5. The method according to claim 1, characterized in that, The background grid includes a plurality of grid cells, and auxiliary velocity information is stored at the grid points of each grid cell; the determining second position information of each moving particle in the background grid when each moving particle moves for N time steps according to fluid motion rules includes: Obtaining third position information of each moving particle when it moves for N - 1 time steps; Determine a target grid cell in the background grid where each moving particle is located according to the third position information of each moving particle; Obtain auxiliary velocity information stored at the lattice points of the target grid cell, where the auxiliary velocity information is determined according to the first velocity information of each moving particle located in the target grid cell, and the first velocity information is the velocity information of each moving particle when moving according to the fluid motion rules; Determine the second position information of each moving particle according to the third position information of each moving particle, the time step, and the auxiliary velocity information stored at the lattice points of the target grid cell.

6. The method according to claim 5, characterized in that, The method further includes: Obtain the first velocity information of each moving particle currently located in each grid cell when moving according to the fluid motion rules; Determine an influence factor corresponding to each moving particle according to the particle mass of each moving particle; Determine target velocity information according to the influence factor corresponding to each moving particle and the first velocity information of each moving particle; Update the auxiliary velocity information stored at the lattice points of each grid cell according to the target velocity information.

7. The method according to claim 6, wherein The obtaining the first velocity information of each moving particle currently located in each grid cell when moving according to the fluid motion rules includes: Determine a transmission control rule, a momentum control rule, and a force application control rule based on a fluid simulation model and configuration parameters, where the transmission control rule is used to indicate the change rate of the position information of the moving particle over time, the momentum control rule is determined based on the law of conservation of momentum, and the force application control rule is used to indicate the motion control of the applied force on the moving particle, and the configuration parameters include one or more of the type of applied force, initial position information, fluid viscosity, and fluid density; Determine the second velocity information of a target moving particle according to the transmission control rule, and the first position information and the third position information of the target moving particle, where the target moving particle is any moving particle currently located in each grid cell; Update the second velocity information of the target moving particle according to the momentum control rule and the force application control rule to obtain the first velocity information of the target moving particle when moving according to the fluid motion rules.

8. A video processing device, characterized in that, The apparatus includes: An obtaining unit, configured to obtain a plurality of image frames included in a video to be processed, where each image frame in the plurality of image frames includes a first image area, and the first image area contains target text; A processing unit, configured to preprocess the first image area of each image frame in the plurality of image frames to obtain a second image area of each image frame; The processing unit is further configured to determine, according to the fluid motion rule, the first position information of each of the plurality of moving particles when each moving particle moves for N time steps, where N is an integer greater than or equal to 0; wherein, if N is equal to 0, the initial position information of each moving particle in the background grid is determined as the first position information of each moving particle; if N is greater than 0, the second position information of each moving particle when moving for N time steps in the background grid is determined according to the fluid motion rule, and the second position information of each moving particle is determined as the first position information of each moving particle; the background grid is determined according to the first image region; Obtain the color information of each moving particle; Determine a target image corresponding to the second image region of the (N + 1)-th image frame in the plurality of image frames according to the first position information and the color information of each moving particle; the moving particle is a moving object with predetermined attribute characteristics; The processing unit is further configured to perform a filling process on the second image region of each image frame according to the target image corresponding to the second image region of each image frame, to obtain a target video.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the video processing method according to any one of claims 1-7 is implemented.

10. A computer device, characterized in that, The computer device includes a memory, a communication interface, and a processor, and the memory, the communication interface, and the processor are interconnected; the memory stores a computer program, and the processor calls the computer program stored in the memory to implement the video processing method according to any one of claims 1-7.

11. A computer program product, characterized in that, The computer program product includes a computer program or computer instructions, and when the computer program or computer instructions are executed by a processor, the video processing method according to any one of claims 1-7 is implemented.

Images